A method for detecting general objects based on a quantized neural network

Abstract

The application provides a method for detecting general objects based on a quantized neural network, and belongs to the technical field of object detection. The application directly obtains bit configurations of weights and feature maps of each layer of a general object detector by establishing a mixed precision configuration and searching the same, thereby building and implementing quantization of the general object detector, effectively improving the accuracy of the quantized general object detector, and optimizing detection of object images. The application can be widely applied to deployment of various general object detectors, and then applied to various application scenarios such as intelligent security, automatic driving, and unmanned aerial vehicle surveying.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of object detection, relates to computer vision and deep learning technology, and particularly relates to a method for detecting general objects by using a quantized neural network based on mixed precision configuration search. BACKGROUND

[0002] General object detection is one of the most important tasks in the field of computer vision, and has been widely applied to fields such as intelligent security, autonomous driving, unmanned aerial vehicle surveying, and the like. In real application scenarios, a general object detector often needs to rely on neural network quantization to compress the weights and feature maps of each layer of the detector from the original 32 bits to a lower bit (such as 4 bits) to reduce the storage and calculation of the general object detector, and thus effectively reduce the threshold for deploying the general object detector.

[0003] Although the technology for quantized general object detectors has made great progress at the present stage, it often directly uses a unified fixed bit configuration or relies on expert hand design of the bit configuration of each layer, which makes the performance of the general object detector after neural network quantization greatly different from that of the original model. Therefore, the existing neural network quantization algorithm for general object detection still has certain technical bottlenecks, and the accuracy is limited compared with the original full-precision detector. SUMMARY

[0004] In order to overcome the deficiencies of the prior art, the present application provides a method for detecting general objects by using a quantized neural network based on mixed precision configuration search, which directly obtains the bit configuration of the weights and feature maps of each layer of the general object detector by establishing and searching a mixed precision configuration, thereby building and realizing the quantization of the general object detector, effectively improving the accuracy of the quantized general object detector, and optimizing the detection of object images.

[0005] The technical solution of the present application is as follows:

[0006] A method for detecting general objects by using a quantized neural network, which directly obtains the bit configuration of the weights and feature maps of each layer of the general object detector by establishing and searching a mixed precision configuration, thereby building and realizing the quantization of the general object detector, effectively improving the accuracy of the quantized general object detector, and optimizing the detection of object images. The method comprises the following steps:

[0007] 1) Obtain the bit configuration of the weights and feature maps of each layer allowed by the general object detector, and construct a search space accordingly;

[0008] 2) Obtain the number of convolution layers L of the general object detector, construct an encoding representing the bit configuration of each layer weight and feature map, and each encoding is a list of length L, and each element in the list is a two-dimensional tuple representing the weight bit number and feature map bit number of the corresponding convolution layer;

[0009] 3) Based on the search space constructed above, a mixed precision quantization super network is established, which replaces each layer of the general object detector with a quantization layer, which supports free forward calculation of each bit of weight and feature map;

[0010] 4) Train the mixed precision quantization super network using the corresponding task training set. For each iteration, first sample the maximum bit configuration network and train it, then sample the minimum bit configuration network and train it, and finally randomly sample two bit configuration networks and train the network;

[0011] 5) Construct a search algorithm iterator. The search can use any existing algorithm, such as random search, evolutionary algorithm search, etc. Finally, the bit configuration of each layer weight and feature map that optimizes the accuracy of the general object detector will be obtained;

[0012] 6) Use the bit configuration of each layer weight and feature map obtained in step 5) to configure the bit number of each convolution layer of the general object detector, and construct the quantized general object detector;

[0013] 7) Train the general object detector constructed in step 6) using the training set, and finally apply it to the detection of general objects.

[0014] The above method of quantizing neural network detection of general objects based on mixed precision configuration search can be widely applied to the deployment of various general object detectors, and further applied to intelligent security, autonomous driving, unmanned aerial vehicle surveying and other application scenarios, and improve the accuracy of the quantized general object detector.

[0015] Compared with the prior art, the beneficial effects of the present application are:

[0016] The present application directly obtains the bit configuration of each layer weight and feature map of the general object detector by establishing and searching the mixed precision configuration, and accordingly builds and realizes the quantization of the general object detector, effectively improves the accuracy of the quantized general object detector, and optimizes the detection of object images.

[0017] The method solves the scheme of directly using a unified fixed bit configuration or relying on expert manual design of bit configurations of each layer in the conventional quantized neural network in general object detection, and realizes quantization of the general object detector through search of the mixed precision configuration, thereby effectively improving the performance of the quantized detector. The application can be applied to any general object detector based on a convolutional neural network (such as YoloV3, Faster R-CNN, etc.). For the general object detection task corresponding to the application, the experimental results on the Pascal VOC dataset show that the average 4-bit quantized model obtained by executing the above method can improve the mAP value from 71.2% to 73.0% compared with the fixed 4-bit model, with the full-precision mAP value of the YoloV3-MobileNetV1 detector being 75.3%. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flow chart of the application.

[0019] Figure 2 is a schematic diagram of a quantization layer in a mixed precision quantization super network according to an embodiment of the application. DETAILED DESCRIPTION

[0020] The application will be further described by examples in conjunction with the drawings, but the scope of the application is not limited in any way.

[0021] The application proposes a method for detecting general objects by a quantized neural network based on search of mixed precision configuration, and the overall process is as shown in Figure 1 . Among them, the mixed precision configuration refers to the bit configuration of each layer weight and feature map.

[0022] The existing quantized neural network often directly uses a unified fixed bit configuration or relies on an expert manually designed bit configuration scheme for each layer when detecting general objects. In order to adjust the resource configuration of each layer from the overall dimension and effectively improve the accuracy of the quantized detector, the application proposes to use a mixed precision configuration search to realize the quantization of the detector. Specifically, the application first establishes a mixed precision configuration search space, which is usually dependent on the limitations of the deployment platform, and in this embodiment, four bits are taken as an example to introduce. The corresponding code of the mixed precision configuration is constructed, assuming that there are L convolutional layers, each code is an L-length list, and each element in the list is a two-dimensional tuple, which represents the weight bit number and the feature map bit number of the layer. According to this, each code can represent a mixed precision bit configuration. Assuming that the bit configuration of a detector is 4 bits for the weight and 6 bits for the feature map of each layer, the corresponding code will be [(4, 6), (4, 6), …, (4, 6)]. Then a mixed precision quantization super network is established based on the constructed mixed precision configuration search space, which replaces each layer of convolution in the detector with a quantization layer, such as Figure 2As shown, the bit configuration of each layer of the detector in the figure is configured as an example of 4 bits of weights and 6 bits of feature maps, and thereby supports the free forward calculation of each bit of weights and feature maps to realize the support of various bit configurations of weights and feature maps. Then, the mixed precision super network is directly trained using the corresponding task training set. For each iteration, first, the maximum bit configuration network (i.e. [(6, 6), (6, 6), …, (6, 6)]) is sampled and trained, then the minimum bit configuration network (i.e. [(2, 2), (2, 2), …, (2, 2)]) is sampled, and finally two bit configuration networks are randomly sampled for training. Then, the search algorithm iterator is constructed, taking the evolutionary algorithm iterator as an example, and the evolutionary algorithm is used to search for the optimal precision of the mixed precision configuration. In the evolutionary algorithm, 50 mixed precision configurations are first randomly generated. For each mixed precision configuration, the corresponding calculation amount is calculated, and the individuals that do not meet the target calculation amount are filtered out and regenerated. Then, for all individuals in the population, the batch normalization layer (BN) statistics are recalculated by forward reasoning a certain number (such as 1000) of training set data, and the performance is directly tested on the validation set to obtain the performance and record it, and the top 10 individual configurations are retained. Then, the evolutionary iteration begins. First, 25 new mixed precision configurations are obtained by mutation. In each mutation, a random selection of the top 10 individual configurations retained in the above population is performed, and a random mutation (i.e. with a 10% probability of becoming another random candidate bit) is performed on each element in the code. At the same time, for each newly generated configuration, the calculation amount is calculated, and the individuals that do not meet the target calculation amount are filtered out and regenerated. Then, another 25 new mixed precision configurations are obtained by crossover. In each crossover, two random selections of the 10 individual configurations retained in the above population are performed, and a random crossover (i.e. randomly selecting an element from the corresponding elements of the two individuals) is performed on each element in the code. At the same time, for each newly generated configuration, the calculation amount is calculated, and the individuals that do not meet the target calculation amount are filtered out and regenerated. Then, a new population is constructed, which contains the 50 individuals obtained by the above mutation and crossover steps, and the above evaluation, mutation and crossover steps are repeated for 20 iterations. At the end of the evolutionary algorithm, the performance of each individual in the final population is evaluated, and the individual with the optimal precision is selected as the output result of the evolutionary algorithm search, i.e. for all individuals in the current population, the batch normalization layer (BN) statistics are recalculated by forward reasoning a certain number (such as 1000) of training set data, and then the performance is directly tested on the validation set to obtain the performance and record it, and the individual with the optimal precision is retained. After the above search algorithm search, the mixed precision configuration with the optimal precision of the general object detector is obtained, and then the bit number of each convolutional layer of the general object detector is set according to the mixed precision configuration to construct the corresponding quantized general object detector.Finally, the trained quantized general object detector is obtained by training the quantized general object detector using the training set, and then applied to the detection of general objects.

[0023] It should be noted that the object of the embodiments disclosed herein is to illustrate the invention. The person skilled in the art can understand that various replacements and modifications can be made without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the disclosed embodiments, and the scope of protection of the invention is defined by the scope of the claims.

Claims

1. A method for detecting general objects based on quantized neural networks, comprising the following steps: 1) obtaining the bit configurations of each layer weight and feature map allowed by the general object detector, and constructing a search space accordingly; 2) obtaining the number of convolutional layers L of the general object detector, and constructing an encoding representing the bit configuration of each layer weight and feature map, each encoding being a list of length L, each element in the list being a two-dimensional tuple representing the number of weight bits and the number of feature map bits of the corresponding convolutional layer; 3) establishing a mixed-precision quantization hypernetwork based on the search space constructed above, which replaces each convolutional layer in the general object detector with a quantization layer that supports free forward calculation of each bit of the weight and each bit of the feature map; 4) training the mixed-precision quantization hypernetwork using the training set corresponding to the task; 5) constructing a search algorithm iterator to ultimately obtain the bit configuration of each layer weight and feature map that optimizes the accuracy of the general object detector; 6) configuring the bit number of each convolutional layer of the general object detector using the bit configuration of each layer weight and feature map that optimizes the accuracy obtained in step 5) to construct a quantized general object detector; 7) training the general object detector constructed in step 6) using the training set, and finally applying it to the detection of general objects. 2.The method of claim 1, wherein, In step 4), for each iteration, first sample the maximum bit configuration network and train it, then sample the minimum bit configuration network and train it, and finally randomly sample two bit configuration networks and train them for each iteration. 3.The method of claim 1, wherein, The search in step 5) uses random search or evolutionary algorithm search.

Images