Model compression method and device, target detection method and device, equipment and storage medium

Abstract

The invention discloses a model compression method and device, a target detection method and device, equipment and a storage medium. The model compression method comprises the steps of obtaining a to-be-compressed model, and pruning the to-be-compressed model according to a preset pruning rate and a first parameter of a batch normalization layer BN in the to-be-compressed model to obtain a pruned model; deepening the network structure of the pruning model to obtain a deepened model, and training the deepened model to obtain a fine tuning model; and converting a first format of a second parameter in the fine tuning model into a second format to obtain a compression model. According to the technical scheme provided by the embodiment of the invention, the model is pruned and quantified, so that the volume of the model and the consumption of computing resources are effectively reduced, and meanwhile, the pruned model is finely adjusted and structurally optimized, so that the precision and the performance of the compressed model are ensured on the premise of greatly improving the compression ratio of the model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model compression method, a target detection method, a device, a equipment and a storage medium. Background Art

[0002] Machine learning and deep learning have been widely used in various fields, including image recognition, speech recognition, natural language processing, recommendation systems, and intelligent driving. Deep learning models, especially convolutional neural networks, have achieved great success in object detection. Classic object detection models include the R-CNN series (such as Faster R-CNN and Mask R-CNN), the YOLO series (such as YOLOv5 and YOLOv8), and the SSD object detection algorithm (Single Shot multibox Detector). These models can simultaneously locate and classify targets through end-to-end training, greatly improving the accuracy and efficiency of detection.

[0003] Currently, existing deep neural network models have high computational costs and large memory requirements, which hinder their deployment in resource-constrained devices (such as edge devices) and devices with strict latency requirements. With the surge in computing power requirements for deep learning models, model compression technology has emerged. Model compression technology can significantly reduce the number of parameters in deep learning models at the expense of extremely low model accuracy. Common model compression technologies include model pruning, model quantization, and knowledge distillation.

[0004] However, existing model compression methods have achieved certain results in reducing model size and reducing computing resource consumption, but the accuracy and performance of the model cannot be guaranteed. Summary of the invention

[0005] The present invention provides a model compression method, a target detection method, an apparatus, a device and a storage medium to solve the problem that the accuracy and performance of the model after compression cannot be guaranteed.

[0006] In a first aspect, the present invention provides a model compression method, comprising:

[0007] Obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model;

[0008] Deepening the network structure of the pruning model to obtain a deepened model, and training the deepened model to obtain a fine-tuning model;

[0009] A compressed model is obtained by converting a first format of a second parameter in the fine-tuning model into a second format, wherein a storage space occupied by the second parameter in the second format is smaller than a storage space occupied by the second parameter in the first format.

[0010] In a second aspect, the present invention provides a target detection method, comprising:

[0011] Acquire the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method described in the first aspect;

[0012] The data to be detected is input into the target compression model to obtain a model output result, and the target object is determined according to the model output result.

[0013] In a third aspect, the present invention provides a model compression device, comprising:

[0014] A pruning module, used for obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model;

[0015] A fine-tuning model determination module, used to deepen the network structure of the pruning model to obtain a deepened model, and train the deepened model to obtain a fine-tuning model;

[0016] The compression model determination module is used to obtain a compression model by converting a first format of a second parameter in the fine-tuning model into a second format, wherein the storage space occupied by the second parameter in the second format is smaller than the storage space occupied by the second parameter in the first format.

[0017] In a fourth aspect, the present invention provides a target detection device, comprising:

[0018] A data acquisition module, used to acquire the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method as described in the first aspect;

[0019] The target detection module is used to input the data to be detected into the target compression model to obtain a model output result, and determine the target object according to the model output result.

[0020] In a fifth aspect, the present invention provides an electronic device, the electronic device comprising:

[0021] at least one processor;

[0022] and a memory communicatively coupled to the at least one processor;

[0023] The memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the model compression method of the first aspect.

[0024] In a sixth aspect, the present invention provides an edge device, the edge device comprising:

[0025] at least one processor; and

[0026] a memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the target detection method of the second aspect.

[0028] In a seventh aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to enable a processor to implement the model compression method of the first aspect and / or the target detection method of the second aspect when executed.

[0029] The model compression scheme provided by the present invention obtains a model to be compressed, prunes the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model, deepens the network structure of the pruned model to obtain a deepened model, and trains the deepened model to obtain a fine-tuned model, and obtains a compressed model by converting a first format of a second parameter in the fine-tuned model into a second format, wherein the storage space occupied by the second parameter in the second format is smaller than the storage space occupied by the second parameter in the first format. By adopting the above technical scheme, the volume of the model and the consumption of computing resources are effectively reduced by pruning and quantizing the model, and at the same time, by fine-tuning and optimizing the structure of the pruned model, the accuracy and performance of the compressed model are guaranteed while greatly improving the compression rate of the model.

[0030] The target detection method provided by the present invention obtains the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method described in the above embodiment, the data to be detected is input into the target compression model to obtain the model output result, and the target object is determined according to the model output result. By adopting the above technical solution, the target recognition task is completed accurately and quickly using the compression model.

[0031] It should be understood that the content described in this section is not intended to identify the key or important features of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 is a flow chart of a model compression method provided according to Embodiment 1 of the present invention;

[0034] Figure 2 is a flow chart of a model compression method provided according to Embodiment 2 of the present invention;

[0035] Figure 3 is a flow chart of a target detection method provided according to Embodiment 3 of the present invention;

[0036] Figure 4 is a structural schematic diagram of a model compression device provided according to Embodiment 4 of the present invention;

[0037] Figure 5 is a schematic diagram of the structure of a target detection device provided according to Embodiment 5 of the present invention;

[0038] Figure 6 It is a structural schematic diagram of an electronic device provided according to Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In the description of the present invention, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are an "or" relationship. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Embodiment 1

[0042] Figure 1 A flowchart of a model compression method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a model is to be compressed. The method can be executed by a model compression device. The model compression device can be implemented in the form of hardware and / or software. The model compression device can be configured in an electronic device. The electronic device can be composed of two or more physical entities or one physical entity.

[0043] like Figure 1 As shown, the model compression method provided in the first embodiment of the present invention specifically includes the following steps:

[0044] S101. Obtain a model to be compressed, and prune the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model.

[0045] In this embodiment, the COCO dataset can be used to pre-train an initial model to obtain a model to be compressed. The parameters in the model to be compressed can be trained on the COCO dataset. Among them, the MS COCO (Microsoft Common Objects in Context) dataset contains 5 types of annotations: target detection, key point detection, material segmentation, panoramic segmentation, and image description. The annotation information is stored in JSON format, and the pre-processing accesses and operates the "annotation" through the COCO API. Among them, there are 80 categories for target detection. The main features of the target detection dataset include: multiple targets, occlusion and noise of some objects, objects of various sizes, more than 300,000 pictures, more than 2,000,000 instances, and 80 target categories.

[0046] Model pruning is usually the process of optimizing a trained model by reducing the number of parameters or computing resources, and should ensure that it has a significant impact on its predictive performance. The principle is to "prune" unimportant parts of the network, such as certain weights or neurons, thereby effectively reducing the complexity and scale of the network. The pruning rate can be preset to obtain a preset pruning rate. According to the preset pruning rate, neurons corresponding to some parameters (i.e., the first parameter) of the batch normalization layer BatchNorm (i.e., BN) in the model to be compressed can be deleted to achieve pruning of the model to be compressed and obtain a pruned model.

[0047] S102: deepening the network structure of the pruning model to obtain a deepened model, and training the deepened model to obtain a fine-tuning model.

[0048] In this embodiment, after obtaining the pruned model, its network structure can be modified and the network can be deepened to improve the quality of feature extraction of the pruned model and achieve the purpose of improving model performance. The model after the network structure is deepened may have lower accuracy than the model to be compressed, so it is necessary to fine-tune the deepened model to improve its accuracy. Specifically, the COCO dataset can be used to retrain the deepened model to obtain a fine-tuned model.

[0049] S103. Obtain a compressed model by converting a first format of a second parameter in the fine-tuning model into a second format, wherein a storage space occupied by the second parameter in the second format is smaller than a storage space occupied by the second parameter in the first format.

[0050] In this embodiment, the parameters of the (second) parameters in the fine-tuning model are usually stored in a set format (i.e., the first format), such as saved in float32 format, that is, each parameter needs to occupy 4 bytes of storage space. The first format can be converted to the second format, such as converting the float32 format to the int8 format, so that each parameter only needs to occupy 1 byte of storage space, which greatly reduces the calculation amount of the model. Among them, the compression model can be arranged in a resource-constrained electronic device, so as to achieve efficient target recognition on a resource-constrained device.

[0051] The model compression method provided by the embodiment of the present invention obtains a model to be compressed, prunes the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model, deepens the network structure of the pruned model to obtain a deepened model, and trains the deepened model to obtain a fine-tuned model, and obtains a compressed model by converting a first format of a second parameter in the fine-tuned model into a second format, wherein the storage space occupied by the second parameter in the second format is smaller than the storage space occupied by the second parameter in the first format. The technical solution of the embodiment of the present invention effectively reduces the size of the model and the consumption of computing resources by pruning and quantizing the model, and at the same time, by fine-tuning and optimizing the structure of the pruned model, the accuracy and performance of the compressed model are guaranteed while greatly improving the compression rate of the model.

[0052] Embodiment 2

[0053] Figure 2 This is a flow chart of a model compression method provided in the second embodiment of the present invention. The technical solution of the embodiment of the present invention is further optimized on the basis of the above-mentioned optional technical solutions, and a specific method for compressing the model is given.

[0054] Optionally, the model to be compressed includes a YOLO model; wherein, the model to be compressed is pruned according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model, including: sorting the values ​​of the first parameter of the batch normalization layer BN in the model to be compressed to obtain a sorting result, wherein the first parameter of the batch normalization layer BN in the model to be compressed is determined using an L1-norm algorithm; determining a target threshold using a preset pruning rate and the sum of the first parameter of the BN, and determining the first parameter less than the target threshold as a target parameter according to the sorting result; deleting the BN feature map channel corresponding to the target parameter and the convolution layer before the BN corresponding to the BN feature map channel to obtain a pruned model. The advantage of such a setting is that the importance of the BN feature map channel is comprehensively considered, the relatively important parameters of the model are retained, and the BN feature map channel and convolution layer associated with the BN parameter less than the target threshold are deleted, thereby not only achieving the purpose of reducing model parameters, but also avoiding excessive decline in model accuracy.

[0055] Optionally, the network structure of the pruned model is deepened to obtain a deepened model, including: adding multiple bottleneck layers BottleNeck in the first feature map fusion convolution layer of the pruned model to obtain an intermediate model; adding at least one BottleNeck in the second feature map fusion convolution layer of the intermediate model to obtain a deepened model, wherein the second feature map fusion convolution layer is a feature map fusion convolution layer in the pruned model except the first feature map fusion convolution layer. The advantage of this setting is that the model is deepened by adding BottleNeck, so that the model performs better in feature extraction, and the feature extraction quality and performance of the model are improved.

[0056] Optionally, before converting the first format of the second parameter in the fine-tuning model into a second format to obtain the compressed model, the method further includes: converting the fine-tuning model into a target format to obtain a converted fine-tuning model, wherein the target format includes a format for describing a computational graph; wherein converting the first format of the second parameter in the fine-tuning model into a second format to obtain the compressed model includes: converting the first format of the second parameter in the converted fine-tuning model into a second format to obtain the compressed model. The advantage of such a setting is that by converting the fine-tuning model into a target format, it is convenient to subsequently convert the compressed model into a model suitable for the chip of the edge device, and then deploy it on the edge device.

[0057] like Figure 2 As shown, a model compression method provided by Embodiment 2 of the present invention specifically includes the following steps:

[0058] S201. Obtain a model to be compressed, and sort the values ​​of a first parameter of a batch normalization layer BN in the model to be compressed to obtain a sorting result, wherein the first parameter of the batch normalization layer BN in the model to be compressed is determined using an L1-norm algorithm, and the model to be compressed includes a YOLO model.

[0059] Specifically, the parameters MODEL_BN of n batch normalization layers (i.e., BN) in the model to be compressed can be obtained, which can be expressed in the form of an n-dimensional vector [p1, p2, ..., pn], where p1, p2, ..., and pn respectively include multiple parameters corresponding to BN. The numerical values ​​of the (first) parameters in each BN can be sorted to obtain a sorting result.

[0060] S202: Determine a target threshold value by using a preset pruning rate and the sum of the first parameter of the BN, and determine a first parameter less than the target threshold value as a target parameter according to the sorting result.

[0061] Specifically, the target threshold threshold can be calculated according to the preset pruning rate α. The product of the preset pruning rate α and the sum of the first parameters of BN (i.e., the total number of all parameters included in p1, p2, ... and pn) can be determined as the target threshold. Each BN can be traversed according to the structure of the model to be compressed, and the parameter value of each BN can be compared with the target threshold threshold, and the first parameter less than the target threshold can be determined as the target parameter. Among them, α can be greater than zero and less than 1. When the target threshold threshold is 1, it means that the model is not pruned.

[0062] S203. Delete the BN feature map channel corresponding to the target parameter and the convolution layer before BN corresponding to the BN feature map channel to obtain a pruned model.

[0063] Specifically, for parameter values ​​greater than or equal to the target threshold threshold, the corresponding BN feature map channel and the corresponding convolution layer before BN are retained. For parameter values ​​less than the target threshold threshold, the corresponding BN feature map channel and the corresponding convolution layer before BN are deleted together. Among them, the BN feature map channel is the channel for determining the parameter value of BN.

[0064] S204, adding multiple bottleneck layers BottleNeck in the first feature map fusion convolution layer of the pruning model to obtain an intermediate model.

[0065] S205. Add at least one BottleNeck in the second feature map fusion convolution layer of the intermediate model to obtain a deepened model, wherein the second feature map fusion convolution layer is a feature map fusion convolution layer in the pruned model except the first feature map fusion convolution layer.

[0066] Optionally, the feature map fusion convolution layer includes a C2f layer.

[0067] Specifically, for the model pruning model obtained after pruning, multiple bottleneck layers BottleNeck can be added to the first feature map fusion convolution layer of the pruning model to obtain an intermediate model. Then at least one BottleNeck is added to the second feature map fusion convolution layer of the intermediate model to obtain a deepened model. For example, if there are 8 C2f layers in the pruning model, two bottleneck feature extraction layers BottleNeck can be added to the 2nd and 3rd C2f layers, and then one BottleNeck can be added to the remaining 6 C2f layers. This setting can ensure the accuracy of the model while reducing the number of parameters.

[0068] S206: Training the deepened model to obtain a fine-tuning model.

[0069] S207: Obtain a converted fine-tuning model by converting the fine-tuning model into a target format, wherein the target format includes a format for describing a computational graph.

[0070] For example, if the target format is the open neural network exchange format ONNX, the fine-tuning model can be converted into ONNX using the export function provided in the YOLO model, so as to facilitate its subsequent conversion into a model suitable for the chip of the edge device and then deployed on the edge device.

[0071] S208: Obtain a compressed model by converting a first format of a second parameter in the converted fine-tuning model into a second format.

[0072] The model compression method provided by the embodiment of the present invention comprehensively considers the importance of the BN feature map channel, retains the relatively important parameters of the model, and deletes the BN feature map channels and convolution layers associated with the BN parameters that are less than the target threshold, thereby not only achieving the purpose of reducing the model parameters, but also avoiding the excessive decline in model accuracy. The model is deepened by adding BottleNeck, so that the model performs better in feature extraction, thereby improving the model performance. By converting the fine-tuning model into the target format, it is convenient to subsequently convert the compressed model into a model suitable for the chip of the edge device, and then deploy it on the edge device.

[0073] Embodiment 3

[0074] Figure 3 A flowchart of a target detection method is provided for the third embodiment of the present invention. This embodiment is applicable to the case of target detection. The method can be executed by a target detection device. The target detection device can be implemented in the form of hardware and / or software. The target detection device can be configured in an electronic device. The electronic device can be composed of two or more physical entities or one physical entity.

[0075] like Figure 3 As shown, the target detection method provided by the third embodiment of the present invention specifically includes the following steps:

[0076] S301. Obtain the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method described in the above embodiment.

[0077] Object detection is one of the core problems in the field of computer vision. In order to obtain a complete image understanding, we should not only focus on classifying different images, but also try to accurately estimate the location of objects contained in each image. It is widely used in computer vision tasks such as image annotation, vehicle counting, activity recognition, face detection, face recognition, and video object co-segmentation.

[0078] S302: Input the data to be detected into the target compression model to obtain a model output result, and determine the target object according to the model output result.

[0079] The target detection method provided by the embodiment of the present invention obtains the data to be detected and the target compression model, wherein the target compression model is obtained by the model compression method described in the above embodiment, the data to be detected is input into the target compression model to obtain the model output result, and the target object is determined according to the model output result. The technical solution of the embodiment of the present invention uses the compression model to accurately and quickly complete the target recognition task.

[0080] Embodiment 4

[0081] Figure 4 This is a schematic diagram of the structure of a model compression device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device includes: a pruning module 401, a fine-tuning model determination module 402 and a compression model determination module 403, wherein:

[0082] A pruning module, used for obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model;

[0083] A fine-tuning model determination module, used to deepen the network structure of the pruning model to obtain a deepened model, and train the deepened model to obtain a fine-tuning model;

[0084] The compression model determination module is used to obtain a compression model by converting a first format of a second parameter in the fine-tuning model into a second format, wherein the storage space occupied by the second parameter in the second format is smaller than the storage space occupied by the second parameter in the first format.

[0085] The model compression device provided in the embodiment of the present invention effectively reduces the size of the model and the consumption of computing resources by pruning and quantizing the model. At the same time, by fine-tuning and structural optimization of the pruned model, the accuracy and performance of the compressed model are guaranteed while greatly improving the compression rate of the model.

[0086] Optionally, the model to be compressed includes a YOLO model.

[0087] Optionally, the pruning module includes:

[0088] A sorting unit, used to sort the values ​​of the first parameter of the batch normalization layer BN in the model to be compressed to obtain a sorting result, wherein the first parameter of the batch normalization layer BN in the model to be compressed is determined by using an L1-norm algorithm;

[0089] a target parameter determination unit, configured to determine a target threshold value by using a preset pruning rate and the sum of the first parameter of the BN, and determine the first parameter less than the target threshold value as the target parameter according to the sorting result;

[0090] The pruning model determination unit is used to delete the BN feature map channel corresponding to the target parameter and the convolution layer before BN corresponding to the BN feature map channel to obtain a pruning model.

[0091] Optionally, the fine-tuning model determination module includes:

[0092] An intermediate model determination unit, used for adding a plurality of bottleneck layers BottleNeck in the first feature map fusion convolution layer of the pruning model to obtain an intermediate model;

[0093] A deepening model determination unit is used to add at least one of the BottleNecks in the second feature map fusion convolution layer of the intermediate model to obtain a deepening model, wherein the second feature map fusion convolution layer is a feature map fusion convolution layer in the pruning model except the first feature map fusion convolution layer.

[0094] Furthermore, the feature map fusion convolution layer includes a C2f layer.

[0095] Optionally, the device further comprises:

[0096] A format conversion module is used to obtain a converted fine-tuning model by converting the fine-tuning model into a target format before obtaining the compressed model by converting the first format of the second parameter in the fine-tuning model into a second format, wherein the target format includes a format for describing a computational graph.

[0097] Optionally, the compression model determination module is specifically used to obtain the compression model by converting a first format of the second parameter in the converted fine-tuning model into a second format.

[0098] The model compression device provided in the embodiment of the present invention can execute the model compression method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] Embodiment 5

[0100] Figure 5 This is a schematic diagram of the structure of a model compression device provided in Embodiment 5 of the present invention. Figure 5 As shown, the device includes: a data acquisition module 501 and a target detection module 502, wherein:

[0101] A data acquisition module, used to acquire the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method described in the above embodiment;

[0102] The target detection module is used to input the data to be detected into the target compression model to obtain a model output result, and determine the target object according to the model output result.

[0103] The target detection device provided by the embodiment of the present invention uses a compression model to accurately and quickly complete the target recognition task.

[0104] Embodiment 6

[0105] Figure 6 A schematic diagram of the structure of an electronic device 60 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0106] like Figure 6 As shown, the electronic device 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0107] A number of components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0108] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 61 performs the various methods and processes described above, such as a model compression method.

[0109] In some embodiments, the model compression method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the model compression method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to perform the model compression method in any other suitable manner (e.g., by means of firmware).

[0110] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0112] The computer device provided above can be used to execute the model compression method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0113] Embodiment 7

[0114] The edge device may include at least one processor, and a memory connected to the at least one processor, such as a read-only memory (ROM), a random access memory (RAM), etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) or the computer program loaded from the storage unit to the random access memory (RAM). Various programs and data required for the operation of the edge device can also be stored in the RAM. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0115] The components shown herein, their connections and relationships, and their functions, are intended to be exemplary only, and are not intended to limit implementations of the inventions described and / or claimed herein.

[0116] Multiple components in the edge device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the edge device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0117] The processor may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor executes the various methods and processes described above, such as target detection methods.

[0118] In some embodiments, the target detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the edge device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the processor, one or more steps of the target detection method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the target detection method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] The computer device provided above can be used to execute the target detection method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0122] Embodiment 8

[0123] In the context of the present invention, a computer readable storage medium may be a tangible medium, wherein the computer executable instructions are used to perform a model compression method and / or an object detection method when executed by a computer processor. The model compression method includes:

[0124] Obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model;

[0125] Deepening the network structure of the pruning model to obtain a deepened model, and training the deepened model to obtain a fine-tuning model;

[0126] A compressed model is obtained by converting a first format of a second parameter in the fine-tuning model into a second format, wherein a storage space occupied by the second parameter in the second format is smaller than a storage space occupied by the second parameter in the first format.

[0127] Object detection methods include:

[0128] Acquire the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method described in the above embodiment;

[0129] The data to be detected is input into the target compression model to obtain a model output result, and the target object is determined according to the model output result.

[0130] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by an instruction execution system, device or equipment or used with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] The computer device provided above can be used to execute the model compression method and / or target detection method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0132] It is worth noting that in the embodiments of the above-mentioned model compression device and / or target detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0133] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A model compression method, characterized in that: include: Obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model; Deepening the network structure of the pruning model to obtain a deepened model, and training the deepened model to obtain a fine-tuning model; A compressed model is obtained by converting a first format of a second parameter in the fine-tuning model into a second format, wherein a storage space occupied by the second parameter in the second format is smaller than a storage space occupied by the second parameter in the first format.

2. The method according to claim 1, characterized in that The model to be compressed includes a YOLO model; wherein, pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model includes: Sorting the values ​​of the first parameter of the batch normalization layer BN in the model to be compressed to obtain a sorting result, wherein the first parameter of the batch normalization layer BN in the model to be compressed is determined by using an L1-norm algorithm; Determine a target threshold value by using a preset pruning rate and the sum of the first parameter of the BN, and determine the first parameter less than the target threshold value as the target parameter according to the sorting result; The BN feature map channel corresponding to the target parameter and the convolution layer before BN corresponding to the BN feature map channel are deleted to obtain a pruned model.

3. The method according to claim 1 or 2, characterized in that: The deepening of the network structure of the pruning model to obtain a deepened model includes: Adding multiple bottleneck layers BottleNeck in the first feature map fusion convolution layer of the pruning model to obtain an intermediate model; At least one BottleNeck is added to the second feature map fusion convolution layer of the intermediate model to obtain a deepened model, wherein the second feature map fusion convolution layer is a feature map fusion convolution layer in the pruned model except the first feature map fusion convolution layer.

4. The method according to claim 3, characterized in that The feature map fusion convolution layer includes a C2f layer.

5. The method according to claim 1, characterized in that Before obtaining the compressed model by converting the first format of the second parameter in the fine-tuning model into the second format, the method further includes: Obtaining a converted fine-tuning model by converting the fine-tuning model into a target format, wherein the target format includes a format for describing a computational graph; The step of converting the first format of the second parameter in the fine-tuning model into the second format to obtain the compressed model includes: A compressed model is obtained by converting the first format of the second parameter in the converted fine-tuning model into a second format.

6. A target detection method, characterized in that: include: Acquire the data to be detected and the target compression model, wherein the target compression model is obtained by using the model compression method according to any one of claims 1 to 5; The data to be detected is input into the target compression model to obtain a model output result, and the target object is determined according to the model output result.

7. A model compression device, characterized in that: include: A pruning module, used for obtaining a model to be compressed, and pruning the model to be compressed according to a preset pruning rate and a first parameter of a batch normalization layer BN in the model to be compressed to obtain a pruned model; A fine-tuning model determination module, used to deepen the network structure of the pruning model to obtain a deepened model, and train the deepened model to obtain a fine-tuning model; The compression model determination module is used to obtain a compression model by converting a first format of a second parameter in the fine-tuning model into a second format, wherein the storage space occupied by the second parameter in the second format is smaller than the storage space occupied by the second parameter in the first format.

8. A target detection device, characterized in that: include: A data acquisition module, used to acquire the data to be detected and a target compression model, wherein the target compression model is obtained by using the model compression method according to any one of claims 1 to 5; The target detection module is used to input the data to be detected into the target compression model to obtain a model output result, and determine the target object according to the model output result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the model compression method according to any one of claims 1 to 5.

10. An edge device, characterized in that: The edge device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the target detection method according to claim 6.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the model compression method as described in any one of claims 1 to 5 and / or the target detection method as described in claim 6 when executed.

Images