A method and system for compressing object detection models based on neural architecture search

Abstract

This invention provides a method and system for compressing object detection models based on neural architecture search. The method includes the following steps: preprocessing images in the object detection dataset to obtain a normalized dataset; converting the object detection model to be deployed into a supernet through neural architecture search; constructing a subnet sampler; the subnet sampler is used to uniformly sample each stage of the supernet; training the supernet based on the subnet sampler and the normalized dataset to obtain the supernet weights; searching for subnets to select the Pareto optimal solution of the target subnet architecture based on the search results, and extracting the weights of the target subnet architecture from the supernet weights for deployment, thereby achieving compression of the object detection model. This invention introduces neural architecture search technology, which can reasonably optimize the model architecture to be deployed using a small amount of computing resources, thereby improving the model inference speed and eliminating the complicated process of determining model structure hyperparameters and tuning model architecture.

Description

Technical Field

[0001] This invention relates to the field of physics, particularly to computer vision and machine learning technologies, especially to object detection technology, and specifically to a method and system for compressing object detection models based on neural architecture search. Background Technology

[0002] Object detection technology is a hot topic in the field of computer vision, with wide applications and demands in many real-world sectors, such as security monitoring systems, intelligent transportation, and industrial inspection. Thanks to the abundant data resources and increased computing power of the big data era, object detection algorithms, relying on supervised deep learning technology, have achieved accuracy surpassing human recognition in numerous detection tasks.

[0003] Currently, deep learning-based object detection technologies are typically trained on GPUs with ample memory and computing power, while pre-trained models are usually deployed on edge devices with limited memory and computing power. This significant difference between training and deployment environments necessitates operations such as model quantization and compression between training and final deployment to improve the inference speed of the model on edge devices.

[0004] In recent years, the emergence of Neural Architecture Search (NAS) technology has made automated machine learning (AutoML) a reality. NAS aims to automate architecture engineering by solving network architecture design problems. It can be mainly divided into Reinforcement Learning (RL) based NAS and weight-sharing (one-shot) NAS; weight-sharing NAS can be further divided into gradient-based multi-path NAS and single-path NAS. The core of NAS lies in ranking the sub-network architectures in the search space according to performance, making the design of the search space particularly critical. However, existing NAS algorithms still rely on expert design of the search space, and their search efficiency is very low. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for compressing object detection models based on neural architecture search, which solves the problems of existing object detection network architecture design relying on experience and difficulty in compressing model architecture.

[0006] To achieve the above and other related objectives, this invention provides a method for compressing a target detection model based on neural architecture search, comprising the following steps: preprocessing images in a target detection dataset to obtain a normalized dataset; converting the target detection model to be deployed into a supernet through neural architecture search; constructing a subnet sampler; the subnet sampler is used to uniformly sample each stage of the supernet; training the supernet based on the subnet sampler and the normalized dataset to obtain supernet weights; searching for subnets to select a target subnet architecture with Pareto optimality based on the search results, and extracting the weights of the target subnet architecture from the supernet weights for deployment, thereby achieving compression of the target detection model.

[0007] In one embodiment of the present invention, the preprocessing includes at least one of the following processing methods: image reconstruction, data enhancement, grayscale processing, and label mapping; wherein the data enhancement includes at least one of the following methods: color gamut transformation, flip transformation, mosaic data enhancement, and perspective fusion data enhancement.

[0008] In one embodiment of the present invention, the supernet includes five stages, each stage including a convolutional layer and multiple feature extraction modules; the convolutional layer and the multiple feature extraction modules are stacked in series.

[0009] In one embodiment of the present invention, the normalized dataset includes a training subset and a validation subset; training the supernet based on the subnet sampler and the normalized dataset includes training the supernet based on the subnet sampler and the training subset.

[0010] In one embodiment of the present invention, training the supernet based on the subnet sampler and the normalized dataset includes: validating the supernet using the validation subset; and constructing a loss function for backpropagation during supernet training.

[0011] In one embodiment of the present invention, the loss function includes cross-union ratio loss and binary cross-entropy loss.

[0012] This invention provides a target detection model compression system based on neural architecture search, comprising: a preprocessing module for preprocessing images in a target detection dataset to obtain a normalized dataset; a conversion module for converting the target detection model to be deployed into a supernet through neural architecture search; a construction module for constructing a subnet sampler; the subnet sampler is used to uniformly sample each stage of the supernet; a training module for training the supernet based on the subnet sampler and the normalized dataset to obtain supernet weights; and a deployment module for searching subnets to select a Pareto optimal target subnet architecture based on the search results, and extracting the weights of the target subnet architecture from the supernet weights for deployment to achieve compression of the target detection model.

[0013] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described object detection model compression method based on neural architecture search.

[0014] The present invention provides an electronic device, comprising: a processor and a memory; the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory, so that the electronic device performs the above-described target detection model compression method based on neural architecture search.

[0015] As described above, the target detection model compression method and system based on neural architecture search of the present invention has the following advantages:

[0016] Beneficial effects:

[0017] (1) Compared with the prior art, the present invention introduces neural architecture search technology, which can make reasonable optimization of the model architecture to be deployed using a small amount of computing power resources, thereby improving the model inference speed and eliminating the complicated process of determining model structure hyperparameters and model architecture tuning.

[0018] (2) Based on the single-path NAS algorithm in weight-sharing NAS, this invention constructs a target detection model compression method based on neural architecture search. It focuses on the single-path NAS algorithm in weight-sharing NAS. Based on the characteristic of highly shared weights in this type of algorithm, the idea of ​​NAS algorithm is introduced into the field of model compression. The target detection model to be deployed is converted into a supernet through neural architecture search. The enumeration search in the search space is transformed into pruning and compression of specific architecture modules and convolution channels, thereby realizing the compression and acceleration of the target detection model. Attached Figure Description

[0019] Figure 1 The flowchart shown is an embodiment of the target detection model compression method based on neural architecture search of the present invention.

[0020] Figure 2 The diagram shown is a structural schematic of the CSPDarknet of the present invention in one embodiment.

[0021] Figure 3 The diagram shown is a structural schematic of the CSPrayer of the present invention in one embodiment.

[0022] Figures 4 to 6 The diagram shown illustrates the structure of the subnet sampler of the present invention at different sampling rates at the stage level in one embodiment.

[0023] Figure 7 The diagram shows the evolutionary effect of hyperparameters in one embodiment during the training of the supernet according to the present invention.

[0024] Figure 8 The diagram shown is a structural schematic of an embodiment of the target detection model compression system based on neural architecture search of the present invention. Detailed Implementation

[0025] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0027] The present invention relates to a target detection model compression method and system based on neural architecture search. Compared with existing technologies, the present invention introduces neural architecture search technology, which can reasonably optimize the architecture of the model to be deployed using a small amount of computing resources, thereby improving the model inference speed and eliminating the complicated process of determining model structure hyperparameters and tuning model architecture. The present invention constructs a target detection model compression method based on neural architecture search based on the single-path NAS algorithm in weight-sharing NAS. Focusing on the single-path NAS algorithm in weight-sharing NAS, based on the characteristic of highly shared weights of this type of algorithm, the idea of ​​NAS algorithm is introduced into the field of model compression. The target detection model to be deployed is converted into a supernet through neural architecture search, and the enumeration search in the search space is transformed into pruning and compression of specific architecture modules and convolution channels, thereby realizing the compression and acceleration of the target detection model.

[0028] Automated Machine Learning (AutoML) automates the process of applying machine learning to data. Given a dataset, AutoML can iterate through different data transformations, machine learning algorithms, and hyperparameters to select the best model.

[0029] Neural Architecture Search (NAS) is the task of automatically finding one or more architectures for a given dataset that will generate models with good results (low loss) for that dataset. NAS is an aspect of AutoML, along with feature engineering, transfer learning, and hyperparameter optimization.

[0030] Reinforcement learning (RL), also known as reward learning, evaluation learning, or reinforcement learning, is a paradigm and methodology of machine learning used to describe and solve problems in which an agent learns strategies to maximize rewards or achieve specific goals during its interaction with the environment.

[0031] One-shot weight sharing means that there will be some overlap between candidate models. In other words, the entire search space can be viewed as a huge model structure (called a SuperNet), and each candidate model is a subnet of this SuperNet. In one-shot NAS, models share weights, so the weights of the shared parts can be directly inherited without repeated training each time.

[0032] The specific implementation scheme of the present invention will be described in detail below with reference to specific examples and accompanying drawings. The implementation scheme of the present invention is a method for compressing a target detection model based on neural architecture search.

[0033] The development platform for this example is a server running Ubuntu 18.04, with an NVIDIA 3090 GPU and an Intel i9-10850 CPU. The experimental environment includes Python 3.8, the deep learning framework PyTorch 1.11.1, CUDA Toolkit 11.3.1, and TorchVision 0.13.1.

[0034] like Figure 1 As shown, in one embodiment, the target detection model compression method based on neural architecture search includes the following steps:

[0035] Step S1: Preprocess the images in the object detection dataset to obtain a normalized dataset.

[0036] In one embodiment, the preprocessing includes, but is not limited to, any one of the following processing methods: image reconstruction, data enhancement, grayscale processing, and label mapping; wherein the data enhancement includes, but is not limited to, any one of the following methods: color gamut transformation, flip transformation, mosaic data enhancement, and perspective fusion data enhancement.

[0037] Specifically, the images in the target detection data are processed by color gamut transformation, flip transformation, mosaic data enhancement, and perspective fusion data enhancement, and the label corresponding to each target box is determined, outputting normalized data F(f img ,f label ), where f img f represents the data-augmented training subset; label This represents the location label of the detected object corresponding to the image in the training subset.

[0038] In one embodiment, before preprocessing the images in the target detection dataset in step S1, it is necessary to first obtain the target detection dataset.

[0039] In this embodiment, the disclosed target detection dataset VOC has a total of 20 detection categories, including 16,551 training images (forming a training subset) and 4,952 validation images (forming a validation subset).

[0040] In one embodiment, all VOC images are reconstructed into 512×512×3 RGB three-channel images (corresponding to the image reconstruction described above). Grayscale bars are constructed to pad redundant parts of the image (corresponding to the grayscale processing described above). The data augmentation methods used (corresponding to the data augmentation described above) are color gamut transformation (HSV), flip transformation, mosaic data augmentation, and perspective blending (mix-up) data augmentation. Label bounding boxes are mapped to the corresponding positions in the data-augmented images (corresponding to the label mapping described above). Finally, the dataset is encapsulated into a class using `torch.utils.data.dataset.Dataset` in PyTorch for use in training batches of data F. i (f img ,f label Iteration of ).

[0041] Step S2: Transform the target detection model to be deployed into a Supernet through neural architecture search. Specifically, the target detection model to be deployed is transformed into a dynamic structure of Supernet with variable convolutional channel number and module stacking number.

[0042] In one embodiment, the supernet includes five stages, each of which includes a convolutional layer and multiple feature extraction modules.

[0043] Specifically, the convolutional layer is stacked in series with multiple feature extraction modules.

[0044] In one embodiment, each stage of the supernet i Includes a step size of 2 and C output channels. i The convolution, and B i The feature extraction modules (which may be residual modules such as Resblock and bottleneck modules such as Bottleneck) are stacked in series.

[0045] In this embodiment, a supernet N_blocks is set. i The range of values ​​for is (1, B). i This allows the supernet to proceed through the stage according to the index of the number of feature extraction modules during the forward pass. i The first N_blocks i One feature extraction module; set the supernet value range to (C i / 2, C i This allows the supernet to perform forward propagation according to the index of the number of convolutional channels for that stage. i Each feature extraction module, after passing through the first N_channels i One convolutional channel.

[0046] In one embodiment, the object detection model YoloX_s is used as the model to be deployed and as the search space, as shown in Table 1.

[0047] like Figure 2 As shown, the backbone network of the YOLOX model is CSPDarknet, which includes structures such as Focus, CSPlayer, Bottleneck, and SPPBottleneck.

[0048] The Focus operator performs a single downsampling of the input image through slicing; the Bottleneck operator includes a 1×1 convolutional layer, a 3×3 convolutional layer, a regularized BN layer, and an activation function; SPPBottleneck concatenates features processed by 5×5, 9×9, and 13×13 max-pooling kernels, and finally fuses the features through a 1×1 convolutional layer; CSPDarknet contains 5 stages, each stage containing a convolution with a stride of 2 for double downsampling, followed by a CSPlayer (see...). Figure 3 Information integration is carried out.

[0049] This invention transforms the original fixed quantities of the number of convolution channels of each operator in each stage of the backbone network and the number of Bottleneck stacks in CSPLAYER into searchable variables to form a search space.

[0050] Table 1 Search Space

[0051]

[0052]

[0053] Step S3: Construct the subnet sampler.

[0054] It should be noted that the subnet sampler is used to uniformly sample each stage of the supernet, and can extract the subnet StaticNet from the supernet.

[0055] Constructing a subnet sampler for each stage of the supernet using the hyperparameter N_channels i With N_blocks i Uniform sampling is performed according to the sampling rate (max, min, middle).

[0056] Specifically, when the sampling rate is max, the maximum value of the hyperparameter is sampled; when the sampling rate is min, the minimum value of the hyperparameter is sampled; and when the sampling rate is middle, the value of the hyperparameter is randomly selected.

[0057] The subnet architecture can be constructed from the supernet by sampling these 5 sets (10 in total) of hyperparameters through the subnet sampler.

[0058] Step S4: Train the supernet based on the subnet sampler and the normalized dataset to obtain the supernet weights.

[0059] In one embodiment, the normalized dataset includes a training subset and a validation subset; training the supernet based on the subnet sampler and the normalized dataset includes training the supernet based on the subnet sampler and the training subset.

[0060] In one embodiment, training the supernet based on the subnet sampler and the normalized dataset includes: validating the supernet using the validation subset; and constructing a loss function for backpropagation during supernet training.

[0061] In one embodiment, the loss function includes cross-union ratio loss and binary cross-entropy loss.

[0062] Specifically, the supernet is connected to the target feature fusion network and the target detection head, wherein the target feature fusion network is a channel aggregation network; the target detection head includes a classification head, a confidence head and a localization head.

[0063] The classification head consists of three sets of convolutional layers, regularization layers, and activation function layers. The classification head outputs cls_pred, which has the number of detected object categories and is used to classify the categories of the predicted candidate boxes. The confidence head outputs obj_pred, which has 1 output channel and is used to predict the probability of an object being in the candidate box. The localization head outputs bb_pred, which has 4 output channels and is used to localize the detected objects.

[0064] The supernet is trained according to the sandwich principle, with F data in each batch. i (f img ,f label Used four times. First, train the max subnet once (e.g., Figure 4 As shown, SuperNet itself is used, and the network output logit is reserved as the teacher result (teacher_pred). Teacher_pred consists of teacher position prediction T_bb_pred, teacher confidence prediction T_obj_pred, and teacher classification prediction T_cls_pred, and the parameters in the detection head are locked. Then, the min subnet (i.e., the structure with the smallest number of parameters in the entire search space) is trained again. Figure 5 (as shown) and two middle subnets (such as) Figure 6 (as shown); where, in the last three training iterations, f is no longer fed to f. label Instead of learning directly, it performs distillation training on the teacher_pred soft tag.

[0065] During distillation training, the parameters in the detector head need to be locked after obtaining `teacher_pred` from the max subnet. Then, knowledge distillation is performed on the prediction results `logits` of the min and middle subnets using `teacher_pred`. For the confidence and classification parts, a binary cross-entropy (BCE) loss is applied after applying SoftMax to `teacher_pred`. For the regression part, a direct-to-follower algorithm is used. label Learning. Every certain number of iterations, calculate the loss of SuperNet on the validation set. val When it stops decreasing, stop training.

[0066] The loss function consists of three parts:

[0067] Loss_distill=L iou(bb_pred,reg_target) +L BCE(obj_pred,T_obj_pred) +L BCE(cls_pred,T_cls_pred) ;

[0068] Among them, L iou To compare the loss between intersection and union, L BCEFor binary cross-entropy loss, reg_target, obj_target, and cls_target are determined by f. label We obtain: bb_preds and reg_targets are the model candidate box prediction results and candidate box labels, respectively; obj_pred and obj_targets are the model candidate box confidence prediction results and confidence labels, respectively; class_pred and class_targets are the candidate box class prediction results and class labels, respectively.

[0069] In this embodiment, the number of epochs is set to 200. Data augmentation is enabled for the first 180 epochs, and the SGD optimizer is used to drive network optimization (with an initial learning rate of 5×10⁻⁶). -3 , momentum=0.937, weight_decay=5×10 -4 The cosine annealing learning rate formula is used, with the first 5 epochs serving as a warm-up. After completing the required number of iterations, the final supernet weights are obtained.

[0070] For the confidence and class components, after applying SoftMax to obj_pred and cls_pred respectively, the model is learned from T_obj_pred and T_cls_pred based on the binary cross-entropy (BCE) loss. For the regression component, the model is learned directly from reg_target. Every certain number of iterations, the subnet sampler constructed in step S3 is used to sample the subnet architecture three times according to the sampling rates (max, min, middle), and the validation set loss is calculated for each subnet. val When the validation set loss of the three is... val Stop training when the condition stabilizes.

[0071] It should be noted that the batch data of each iteration first passes through the supernet itself, S. max (i.e., the largest subnet), and its prediction result logits max As a teacher, the result is teacher_pred. Subnet S min With S middle The prediction results logits min With logits middle Distillation learning is performed on the teacher result teacher_pred.

[0072] Step S5: Search subnets to select the target subnet architecture with Pareto optimal solution based on the search results, and extract the weights of the target subnet architecture from the supernet weights for deployment to achieve compression of the target detection model.

[0073] In this embodiment, an improved evolutionary algorithm is used to heuristically enumerate subnets in the search space and evaluate their validation set loss. val With time delay.

[0074] Specifically, firstly, the initial population (number of subnets) required for the evolutionary search ier0 is defined as population0. Then, the number of floating-point operations per second (FLOPs) for the maximum and minimum subnets in the search space is calculated, and these are divided into four intervals. Each interval requires an initial number of subnets equal to population0 / 4. Random subnet sampling is initiated using a sampler until the initial population meets the requirements. Subsequently, an evolutionary algorithm is used to cross-pollinate and mutate N_channels and N_blocks, as well as the test image resolution, to construct the subnet architecture, inheriting the supernet weights of the slices, and evaluating its loss. val Record its FLOPs.

[0075] The crossover process takes two network architectures A and B as inputs, and for one of the network architectures A, at each stage... i The parameters (N_A_blocks) i N_A_channels i There is a certain probability that p is in the same stage of another network architecture. i The parameters (N_B_blocks) i N_B_channels i Replace and output a new network architecture C; the mutation process takes a network architecture D as input and outputs a new network architecture C for each stage. i The parameters (N_D_blocks) i N_D_channels i There is a certain probability that p will randomly change to other values ​​within the search space, and a new network architecture E will be output.

[0076] Evaluation of the validation set loss of the new subnet architecture obtained from crossover mutation. val Calculate the FLOPs and add them to the population. In each iteration, the best subnets at the Pareto front are used as input to the evolutionary algorithm for crossover and mutation, for a total of K iterations.

[0077] Each time, the best subnets at the Pareto front are evolved, and their hyperparameters are cross-referenced and mutated, until the evolution stops after 25 generations. See the diagram showing the evolution from generation 0 to generation 25. Figure 7 .

[0078] The X-axis represents the number of floating-point operations per second for the subnet, and the Y-axis represents the negative average loss of the subnet on the validation set.

[0079] Finally, choose the appropriate Pareto optimal solution. Figure 7 The subnet architecture (connecting points on the network) is defined. The weight of the subnet is extracted from the supernet weights obtained in step S4 based on N_channels and N_blocks, and then deployed after fine-tuning.

[0080] In summary, this invention provides a target detection model compression method based on neural architecture search. Addressing the issues of complex hyperparameter design in the backbone network of target detection networks and insufficient performance of edge devices, this method transforms the fixed structure of the model to be deployed into a search space with variable channel width and module number through neural architecture search technology. Without significantly reducing model accuracy, pruning techniques are used to optimize the channel width and module number, thereby achieving compression and acceleration of the target detection model.

[0081] This invention introduces neural network architecture search technology, which can optimize the architecture of the model to be deployed using a small amount of computing resources, thereby improving the model inference speed and eliminating the complicated process of determining model structure hyperparameters and tuning model architecture.

[0082] This invention relates to a method for compressing object detection models based on neural architecture search. It is an implementation method based on convolutional neural networks. This invention can remove redundant convolutional channels and modules in the backbone network of the object detection model, improve the inference speed of the model, and at the same time, the model accuracy of the network is not significantly reduced.

[0083] This invention introduces a one-shot-single-path NAS series as a compression method for object detection model architecture. It can automatically search for the hyperparameters of each stage of the object detection network, which is more economical and efficient in terms of parameter quantity compared to networks designed by human experience. Through knowledge distillation technology, the prediction results of the supernet are distilled into the subnet, so that the subnet reduces the number of model parameters and computational cost without significantly reducing the accuracy of the supernet. At the same time, during the search process, the validation set loss is used as the subnet accuracy performance evaluation index, which improves the subnet search speed.

[0084] This application introduces the concept of one-shot-single-path NAS, which transforms the target detection model to be deployed into a supernet. After distillation training, a subnet with redundant convolutional channels and modules removed is obtained for deployment, achieving a balance between performance and inference speed. At the same time, the search method of NAS is improved to enhance the efficiency of searching subnets and fully explore the entire search space.

[0085] It should be noted that the scope of protection of the target detection model compression method based on neural architecture search described in this invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0086] The storage medium of this invention stores a computer program, which, when executed by a processor, implements the aforementioned object detection model compression method based on neural architecture search. The storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disk, USB flash drive, memory card, or optical disk.

[0087] One or more storage media may be used in any combination. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0088] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0089] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0090] In one embodiment, computer program code for performing the operations of the present invention is written using a deep learning framework based on Python.

[0091] The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, these computer program instructions create means for implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0093] These computer program instructions may also be stored in a computer-readable medium, which causes a computer, other programmable data processing apparatus, or other device to operate in a particular manner, thereby producing an article of manufacture that includes instructions for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0094] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus provide a process for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The electronic device of the present invention includes a processor and a memory.

[0096] The memory is used to store computer programs; preferably, the memory includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0097] The processor is connected to the memory and is used to execute the computer program stored in the memory, so that the electronic device performs the above-described target detection model compression method based on neural architecture search.

[0098] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0099] In one embodiment, the electronic device is a server.

[0100] like Figure 8 As shown, in one embodiment, the target detection model compression system based on neural architecture search of the present invention includes:

[0101] The preprocessing module 81 is used to preprocess the images in the object detection dataset to obtain a normalized dataset.

[0102] Transformation module 82 is used to transform the target detection model to be deployed into a supernet through neural architecture search.

[0103] Module 83 is used to construct a subnet sampler; the subnet sampler is used to uniformly sample each stage of the supernet.

[0104] Training module 84 is used to train the supernet based on the subnet sampler and the normalized dataset to obtain the supernet weights.

[0105] Deployment module 85 is used to search subnets, select the target subnet architecture with Pareto optimal solution based on the search results, and extract the weights of the target subnet architecture from the supernet weights for deployment, so as to achieve compression of the target detection model.

[0106] It should be noted that the structure and principle of the preprocessing module, the conversion module, the construction module, the training module and the deployment module correspond one-to-one with the steps (steps S1 to S5) in the above-mentioned object detection model compression method based on neural architecture search, so they will not be repeated here.

[0107] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, module x can be a separate processing element, or it can be integrated into a chip within the system. Alternatively, it can be stored as program code in the system's memory, and its function can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0108] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-On-a-Chip (SOC).

[0109] It should be noted that the target detection model compression system based on neural architecture search of the present invention can implement the target detection model compression method based on neural architecture search of the present invention. However, the implementation device of the target detection model compression method based on neural architecture search of the present invention includes, but is not limited to, the structure of the target detection model compression system based on neural architecture search listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of the present invention are included within the protection scope of the present invention.

[0110] In summary, the target detection model compression method and system based on neural architecture search of this invention, compared with the prior art, introduces neural architecture search technology, which can reasonably optimize the architecture of the model to be deployed using a small amount of computing resources, thereby improving the model inference speed and eliminating the complicated process of determining model structure hyperparameters and tuning the model architecture. This invention constructs a target detection model compression method based on neural architecture search, focusing on the single-path NAS algorithm in weight-sharing NAS. Based on the highly shared weight characteristic of this type of algorithm, the idea of ​​NAS algorithm is introduced into the field of model compression. Through neural architecture search, the target detection model to be deployed is converted into a supernet, and the enumeration search in the search space is transformed into pruning and compression of specific architecture modules and convolution channels, realizing the compression and acceleration of the target detection model. Therefore, this invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0111] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A neural architecture search based target detection model compression method, characterized in that, The method comprises the following steps: Step S1, preprocessing images in a target detection dataset to obtain a normalized dataset; Step S2, converting a target detection model to be deployed into a super network through neural architecture search; Step S3, constructing a subnet sampler; The subnet sampler is used for uniformly sampling each stage of the super network; Step S4, training the super network based on the subnet sampler and the normalized dataset to obtain super network weights; Specifically, the super network is connected to a target feature fusion network and a target detection head, wherein the target feature fusion network is a channel aggregation network; and the target detection head comprises a classification head, a confidence head and a positioning head; The super network is trained according to the sandwich rule; Step S5, searching subnets to select a target subnet architecture of a Pareto optimal solution based on a search result, and taking out weights of the target subnet architecture from the super network weights for deployment to realize compression of the target detection model; Specifically, first define the evolutionary search The initial population is needed Then, calculate the floating-point operations per second (FLOPs) of the maximum subnet max and the minimum subnet min of the search space, and divide them into four intervals, and the number of initial subnets required for each interval is / 4, and use the sampler to start random subnet sampling until the number of initial populations meets the requirements, and then use the evolutionary algorithm to and , and test the image resolution to cross and mutate, construct the subnet architecture, inherit the super-network weight of the slice, and evaluate its , and record its FLOPs; Wherein, the crossover process inputs two network architectures A and B, and for the parameters of one network architecture A at each stage there is a certain probability p to be replaced by the parameters of the other network architecture at the same stage , and a new network architecture C is output; the mutation process inputs one network architecture D, and for the parameters of D at each stage there is a certain probability p to be randomly changed to other values within the search space range, and a new network architecture E is output; evaluating the validation set loss of the new subnetwork architecture resulting from the crossover variation , compute its FLOPs and join the population , the elite subnetworks on the Pareto front are used as input to the evolutionary algorithm for crossover and mutation, for K iterations Each time, an excellent subnet on a Pareto frontier is evolved, and hyperparameters therein are crossed and mutated. Finally, select the appropriate sub-network architecture of the Pareto optimal solution, according to With The weight of the sub-network is obtained from the super-network weight in step S4, and is deployed after fine-tuning.

2. The neural architecture search based target detection model compression method of claim 1, wherein, The preprocessing at least comprises any one of the following processing modes: image reconstruction, data enhancement, grayscale processing and label mapping; wherein the data enhancement at least comprises any one of the following modes: color gamut transformation, flip transformation, mosaic data enhancement and perspective fusion data enhancement.

3. The neural architecture search based target detection model compression method of claim 1, wherein, The super network comprises five stages, each of which comprises a convolutional layer and a plurality of feature extraction modules; and the convolutional layer and the plurality of feature extraction modules are connected in series and stacked.

4. The neural architecture search based target detection model compression method of claim 1, wherein, The normalized dataset comprises a training subset and a verification subset; The training of the super network based on the subnet sampler and the normalized dataset comprises training the super network based on the subnet sampler and the training subset.

5. The neural architecture search based target detection model compression method of claim 4, wherein, The training of the super network based on the subnet sampler and the normalized dataset comprises: The verification subset is used for verifying the super network; A loss function is constructed for back propagation of the super network training.

6. The neural architecture search based target detection model compression method of claim 5, wherein, The loss function comprises an intersection over union loss and a binary cross entropy loss. 7.A neural architecture search based target detection model compression system, characterized in that, The method comprises: A preprocessing module is configured to preprocess images in a target detection dataset to obtain a normalized dataset; A conversion module is configured to convert a target detection model to be deployed into a super network through neural architecture search; A construction module is configured to construct a subnet sampler; The subnet sampler is used for uniformly sampling each stage of the super network; A training module is configured to train the super network based on the subnet sampler and the normalized dataset to obtain super network weights; Specifically, the super network is connected to a target feature fusion network and a target detection head, wherein the target feature fusion network is a channel aggregation network; and the target detection head comprises a classification head, a confidence head and a positioning head; The super network is trained according to the sandwich rule; A deployment module is configured to search subnets to select a target subnet architecture of a Pareto optimal solution based on a search result, and take out weights of the target subnet architecture from the super network weights for deployment to realize compression of the target detection model; Each time, an excellent subnet on a Pareto frontier is evolved, and hyperparameters therein are crossed and mutated. ​ ​ ​ Specifically, first define the evolutionary search The initial population is needed Then, calculate the floating-point operations per second (FLOPs) of the maximum subnet max and the minimum subnet min of the search space, and divide them into four intervals, and the number of initial subnets required for each interval is / 4, and use the sampler to start random subnet sampling until the number of initial populations meets the requirements, and then use the evolutionary algorithm to and , and test the image resolution to cross and mutate, construct the subnet architecture, inherit the super-network weight of the slice, and evaluate its , and record its FLOPs; Wherein, the crossover process inputs two network architectures A and B, and for the parameters of one network architecture A at each stage there is a certain probability p to be replaced by the parameters of the other network architecture at the same stage , and a new network architecture C is output; the mutation process inputs one network architecture D, and for the parameters of which at each stage there is a certain probability p to be randomly changed to other values within the search space range, and a new network architecture E is output; evaluating the validation set loss of the new subnetwork architecture resulting from the crossover variation , compute its FLOPs and join the population , the elite subnetworks on the Pareto front in each iteration are taken as the input of the evolutionary algorithm for crossover and variation, and the iteration is performed for K times. ​ Finally, the sub-network architecture of the appropriate Pareto optimal solution is selected, and the weights of the sub-network are obtained from the super-network weights according to and The weights of the sub-network are obtained from the super-network weights at step S4, and the sub-network is deployed after fine-tuning.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the neural architecture search-based target detection model compression method in any one of claims 1 to 6.

9. An electronic device, comprising: Comprise: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device executes the neural architecture search-based target detection model compression method in any one of claims 1 to 6.

Images