A method for building an adaptive target detection model based on model parameter mapping

Abstract

The present invention discloses a method for constructing an adaptive target detection model based on model parameter mapping. Firstly, the model parameters of a pre-trained seed neural network model are directly mapped to individuals in a new population through the model parameter mapping method as pre-trained weights of the new individuals, thereby improving the convergence speed of each individual and further improving the search efficiency of the entire neural network architecture search algorithm. Secondly, the present invention develops an evolutionary neural network architecture search algorithm, which can customize the backbone network model structure based on a given task target detection data set, and further adaptively design a target detection model, effectively improving the feature extraction capability of the backbone network and further improving the detection capability of the target detection model.

Description

Technical Field

[0001] The present invention relates to target detection model design and optimization technology, and in particular to a method for constructing an adaptive target detection model based on model parameter mapping. Background Art

[0002] Deep neural networks have achieved excellent performance in computer vision tasks. Among them, target detection tasks are widely used in various real-world scenarios, such as autonomous driving, facial key point detection, and full-scene intelligent monitoring systems. At present, the mainstream target detection algorithm based on deep learning has two main steps: first, extract key feature information from the input image through the backbone network structure, then generate a detection frame based on the extracted feature information, and train the target detection model to make the detection frame fit the target to be detected as closely as possible. Because the feature extraction ability of the backbone network is crucial, it has a great impact on the performance of the target detection model. At present, advanced target detection models usually use image classification models designed by human experts as the backbone network.

[0003] However, image classification models are usually custom designed for image classification tasks. Since image classification tasks tend to judge what information is in the image and focus on global information, the input is an image and the output is a label / category, which represents the classification of the image. Therefore, the output of the classification task is structured and has only one label. The output of target detection is all targets in the image, including target category information and target location information in the image, and it is uncertain how many targets there are in the image. Therefore, directly using the image classification model as the backbone network may not be the optimal solution for target detection tasks.

[0004] Recently, the emergence of Neural Architecture Search (NAS) technology has provided new ideas for the design of target detection models. NAS technology models the design of neural network models as a combinatorial optimization problem. It can automatically design neural network models based on task datasets using optimization algorithms. This method has achieved remarkable results in image classification, segmentation, target detection, tracking and other fields. However, in order to make the entire model converge more easily, the backbone network structure of the target detection model usually needs to be pre-trained on the imagenet dataset. This brings a heavy computational burden to the design of the entire NAS algorithm for target detection, which greatly hinders the promotion of the algorithm. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose an adaptive target detection model construction method based on model parameter mapping, and use the NAS algorithm based on the evolutionary algorithm to adaptively design the target detection model. Through model parameter mapping, the pre-trained weight parameters of the seed network are mapped to the new individuals in the population, so that the new individuals obtain pre-trained weights, avoiding their pre-training on imagenet, and thus each individual can obtain a more accurate fitness value during pre-evaluation, solving the problem of low search efficiency and insufficient performance of the searched model in the target detection neural network architecture search algorithm.

[0006] To solve the above problems, the present invention provides the following technical solutions:

[0007] The present application provides a method for constructing an adaptive target detection model based on model parameter mapping, comprising the following steps:

[0008] Step S1: given a labeled target detection dataset, divide the dataset into training dataset D train , validation dataset D val And the test dataset D test , the training data set D train , randomly divided into two parts: D train1 , D train2 ;

[0009] Step S2, define the seed target detection network, define the search space and encoding scheme:

[0010] Step S2-1, select mobilenet v2 as the seed network, and use the imagenet dataset to pre-train the seed network until the seed network converges.

[0011] Step S2-2: Use the mobilenet v2 seed network as the backbone network of the RetinaNet target detection framework and train the training set D train1 Train until convergence to obtain the seed target detection network mobilenet v2-RetinaNet.

[0012] Step S2-3, using the Mobilenet v2 network as the template neural network architecture, using the evolutionary algorithm to redesign the structures of the five modules of Mobilenet v2, namely, bottleneck-2, bottleneck-3, bottleneck-4, bottleneck-5, and bottleneck-6, including the size of the convolution kernel (k) of the convolution layer in the bottleneck, which includes 3 optional values ​​k={3, 5, 7} and the channel expansion factor t of the bottleneck structure, which includes 3 optional values ​​t={1, 4, 6}. Therefore, there are 9 combinations of the expansion factor parameter t and the convolution kernel parameter k, which are represented by integers 1, 2, 3, 4, 5, 6, 7, 8, and 9, respectively. In addition, each bottleneck module needs to be stacked m times repeatedly, where m is an integer with a value of 2≤m≤4. The bottleneck-2 module group is sub-network module 1, the bottleneck-3 group is sub-network module 2, the bottleneck-4 group is sub-network module 3, the bottleneck-5 group is sub-network module 4, and the bottleneck-6 group is sub-network module 5. Each sub-network module needs to be encoded using a 4-bit integer. The skip layer represents the skip connection and is encoded by the integer 0. A mobilenet v2 backbone network structure can be encoded as a 20-bit integer gene sequence.

[0013] Step S3, construct an evolutionary neural network architecture search algorithm based on target detection:

[0014] Step S3-1, algorithm initialization, set the initial population size M, crossover probability Pc, mutation probability Pm, and maximum evolutionary generations G.

[0015] Step S3-2, based on the encoding scheme defined in step S2-3, initialize M chromosomes as the initial population.

[0016] Step S3-3, decode the M chromosomes into M neural network architectures, and use them as the backbone network of the RetinaNet target detection framework to obtain M candidate RetinaNet target detection models.

[0017] Step S3-4, the seed target detection network mobilenet v2-RetinaNet pre-trained model parameters W obtained in step S2-2 seed Mapped to the M candidate RetinaNet target detection models in step S3-3, respectively, as the initial parameters W of each candidate RetinaNet target detection model ini , the model parameter mapping process is as follows:

[0018] 1) Parameter mapping of model depth:

[0019] Parameter mapping is performed in units of sub-network modules. The sub-network module of mobilenet v2-RetinaNet contains i bottleneck structures (2≤i≤4), and the candidate RetinaNet target detection model to be mapped contains 1 bottleneck structure (2≤l≤4). The parameter mapping formula for each bottleneck is:

[0020] f(i)=min(i,l),

[0021]

[0022] 2) Model width parameter mapping:

[0023] The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is Among them, the parameters out and in represent the bottleneck input and output channels, h, w, h i and w i is the spatial size of the feature map, then the width mapping formula is:

[0024] If: r≤p:

[0025]

[0026] If r>p:

[0027]

[0028] 3) Convolution kernel parameter mapping

[0029] The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is Among them, k and k 1 is the size of the convolution kernel. Then the formula for convolution kernel mapping is:

[0030]

[0031]

[0032] Step S3-5, the individuals whose model parameters are mapped in step S3-4 are based on the training data set D train2 Train and validate the dataset D val The Loss value of each individual is calculated as the fitness value of the individual.

[0033] Step S3-6, using roulette to select several individuals to form a parent population;

[0034] Step S3-7, using a single-point crossover operator to perform pairwise crossover on the individuals in the parent population to obtain M new individuals to form the offspring population;

[0035] Step S3-8, using integer polynomial mutation to perform mutation operation on each individual in the offspring population;

[0036] Step S3-9, using the model parameter mapping method given in step S3-4, the pre-trained parameters of mobilenet v2-RetinaNet are mapped to each individual in the offspring population.

[0037] Step S3-10, based on the validation data set D val , calculate the fitness value of each individual in the offspring population;

[0038] Step S3-11, merge the individuals in the offspring population and the parent population, select M new individuals according to the fitness value of the individuals and use environmental selection to form the next generation population, and feed back to step S3-6 until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure.

[0039] Step S4, decode the optimal individual output in step S3 into the corresponding neural network as the backbone network of RetinaNet, and obtain the target detection model RetinaNet-nas based on the dataset D train The network structure is fully trained until the model converges.

[0040] Compared with the prior art, the above application has the following advantages or beneficial effects:

[0041] The present invention firstly maps the model parameters of the pre-trained seed neural network model directly to the individuals in the new population through the model parameter mapping method as the pre-trained weights of the new individuals, thereby improving the convergence speed of each individual and thus improving the search efficiency of the entire neural network architecture search algorithm. Secondly, the present invention develops an evolutionary neural network architecture search algorithm, which can customize the backbone network model structure based on a given task target detection data set, and then adaptively design a target detection model, effectively improving the feature extraction capability of the backbone network, and thus improving the detection capability of the target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of a candidate target detection model of the present invention;

[0043] Figure 2 It is the algorithm flow chart of the present invention;

[0044] Figure 3 It is a schematic diagram of the model parameter mapping of the present invention;

[0045] Figure 4 It is a schematic diagram of the training convergence state of the seed target detection network mobilenet v2-RetinaNet of the present invention;

[0046] Figure 5 It is a search schematic diagram of the evolutionary neural network architecture search algorithm of the present invention;

[0047] Figure 6 It is the optimal individual complete training schematic diagram of the present invention. DETAILED DESCRIPTION

[0048] The present embodiment will be further described below in conjunction with the accompanying drawings:

[0049] Step S1: given a labeled target detection dataset, divide the dataset into training dataset D val And the test dataset D test , the training data set D train The samples were randomly divided into two groups: D train1 , D train2 ;

[0050] Step S2, define the seed target detection network, define the search space and encoding scheme:

[0051] Step S2-1, select mobilenet v2 as the seed network, and use the imagenet dataset to pre-train the seed network until the seed network converges.

[0052] Step S2-2: Use the mobilenet v2 seed network as the backbone network of the RetinaNet target detection framework and train the training set D train1 The training is performed until convergence, and the seed target detection network mobilenet v2-RetinaNet is obtained. The training process of the seed target detection network mobilenet v2-RetinaNet is as follows: Figure 4 shown.

[0053] Step S2-3, using the Mobilenet v2 network as the template neural network architecture, and using the evolutionary algorithm to redesign the structures of the five modules of Mobilenet v2, namely, bottleneck-2, bottleneck-3, bottleneck-4, bottleneck-5, and bottleneck-6, including the size of the convolution kernel (k) of the convolution layer in the bottleneck, which includes 3 optional values ​​k={3, 5, 7} and the channel expansion factor t of the bottleneck structure, which includes 3 optional values ​​t={1, 4, 6}. Therefore, there are 9 combinations of the expansion factor parameter t and the convolution kernel parameter k, which are represented by integers 1, 2, 3, 4, 5, 6, 7, 8, and 9, respectively. In addition, each bottleneck module needs to be repeatedly stacked m times, and m is an integer with a value of 2≤m≤4. The bottleneck-2 module group is sub-network module 1, the bottleneck-3 group is sub-network module 2, the bottleneck-4 group is sub-network module 3, the bottleneck-5 group is sub-network module 4, and the bottleneck-6 group is sub-network module 5. Each sub-network module needs to be encoded using a 4-bit integer. The skip layer represents the skip connection and is encoded by the integer 0. A mobilenet v2 backbone network structure can be encoded as a 20-bit integer gene sequence. In this embodiment, the sub-network modules redesigned by the evolutionary neural network architecture search algorithm are shown in Table 1, and the bottleneck structure encoding is shown in Table 2

[0054] Table 1 Sub-network modules redesigned by the evolutionary neural network architecture search algorithm

[0055]

[0056] Table 2 Coding information

[0057] Bottleneck Configuration coding k=3, t=1 1 k=5, t=4 2 k=7, t=6 3 k=3, t=1 4 k=5, t=4 5 k=7, t=6 6 k=3, t=1 7 k=5, t=4 8 k=7, t=6 9 Skip 0

[0058] Step S3, construct an evolutionary neural network architecture search algorithm based on target detection. The algorithm flow chart is as follows: Figure 2 As shown:

[0059] Step S3-1, algorithm initialization, set the initialization population size M=40, crossover probability Pc=0.95, mutation probability Pm=0.2, and maximum evolutionary generation G=30.

[0060] Step S3-2, based on the encoding scheme defined in step S2-3, initialize M chromosomes as the initial population.

[0061] Step S3-3, decode the M chromosomes into M neural network architectures, and use them as the backbone network of the RetinaNet target detection framework to obtain M candidate RetinaNet target detection models.

[0062] Step S3-4, such as Figure 3 As shown, the seed target detection network mobilenet v2-RetinaNet pre-trained model parameters W obtained in step S2-2 are seed , mapped to the M candidate RetinaNet target detection models in step S3-3, as the initial parameters W of each candidate RetinaNet target detection model ini The structure diagram of the candidate RetinaNet target detection model after mapping is as follows Figure 1 As shown. Figure 3 In the figure, the model parameter mapping sequence starts from convolution layer 2d and completes the parameter mapping in sequence. The mapping order is: convolution layer 2d, bottleneck, sub-network 1, sub-network 2, sub-network 3, sub-network 4, sub-network 5, bottleneck, convolution layer, FPN model. The model parameter mapping process is as follows:

[0063] 1) Parameter mapping of model depth:

[0064] Parameter mapping is performed in units of sub-network modules. The sub-network module of mobilenetv2-RetinaNet contains i bottleneck structures (2≤i≤4), and the candidate RetinaNet target detection model to be mapped contains 1 bottleneck structure (2≤l≤4). The parameter mapping formula for each bottleneck is:

[0065] f(i)=min(i,l),

[0066]

[0067] 2) Model width parameter mapping:

[0068] The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is Among them, the parameters out and in represent the bottleneck input and output channels, h, w, h i and w i is the spatial size of the feature map, then the width mapping formula is:

[0069] If: r≤p:

[0070]

[0071] If r>p:

[0072]

[0073] 3) Convolution kernel parameter mapping

[0074] The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is Among them, k and k 1 is the size of the convolution kernel. Then the formula for convolution kernel mapping is:

[0075]

[0076]

[0077] Step S3-5, the individuals whose model parameters are mapped in step S3-4 are based on the training data set D train2 Train and validate the dataset D val The Loss value of each individual is calculated as the fitness value of the individual.

[0078] Step S3-6, using roulette to select several individuals to form a parent population;

[0079] Step S3-7, using a single-point crossover operator to perform pairwise crossover on the individuals in the parent population to obtain M new individuals to form the offspring population;

[0080] Step S3-8, using integer polynomial mutation to perform mutation operation on each individual in the offspring population;

[0081] Step S3-9, using the model parameter mapping method given in step S3-4, the pre-trained parameters of mobilenet v2-RetinaNet are mapped to each individual in the offspring population.

[0082] Step S3-10, based on the validation data set D val , calculate the fitness value of each individual in the offspring population;

[0083] Step S3-11, merge the individuals in the offspring population and the parent population, select M new individuals according to the fitness value of the individuals and use environmental selection to form the next generation population, and feed back to step S3-6 until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure.

[0084] Step S4, decode the optimal individual output in step S3 into the corresponding neural network as the backbone network of RetinaNet, and obtain the target detection model RetinaNet-nas based on the dataset D train The network structure is fully trained until the model converges.

[0085] In order to verify the advantages of the present invention, the following comparison is performed:

[0086] The present invention uses the Microsoft COCO dataset, with 118,287 training set images and 5,000 test set images. The present invention randomly extracts 5,000 images from the training set as the validation set to verify the effect of the evolved neural network architecture. The optimization search process of the evolved neural network architecture search is as follows: Figure 5 The optimal individual output by the evolutionary neural network architecture search algorithm undergoes a complete training on the entire training set. The complete training process of the optimal individual is shown in Figure 6 After training, the individual was tested on the test set of the Microsoft COCO dataset to test its detection ability, using the maAP value when IoU = 0.5:0.95 as the evaluation index, and compared with the currently more common centralized method. The specific experimental results are as follows:

[0087] Table 3 Specific experimental results

[0088]

[0089] From the results in Table 3, it can be seen that the backbone network model MobilenetV2-NAS optimized by the algorithm of the present invention has a mAP value that not only exceeds the Mobilenet V2 model, but also exceeds the other two neural network architecture search algorithms DetNAS and FNA. Therefore, the algorithm proposed in the present invention has a more significant improvement compared with the prior art and has good practical value and promotion value in the field of target detection.

[0090] Although the above describes the specific implementation methods of the present invention, it does not limit the scope of protection of the present invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing an adaptive target detection model based on model parameter mapping, characterized in that: The following steps are involved: Step S1: Given a labeled target detection dataset, which is a picture, divide the dataset into a training dataset D train , validation dataset D val And the test dataset D test , the training data set D train The samples were randomly divided into two groups: D train1 , D train2 ; Step S2, defining a seed target detection network, defining a search space and an encoding scheme, including: Step S2-1, select mobilenet v2 as the seed network, and use the imagenet dataset to pre-train the seed network until the seed network converges; Step S2-2: Use the mobilenet v2 seed network as the backbone network of the RetinaNet target detection framework and train the training set D train1 Train until convergence to obtain the seed target detection network mobilenet v2-RetinaNet; Step S2-3, using the mobilenet v2 network as a template neural network architecture; Step S3, constructing an evolutionary neural network architecture search algorithm based on target detection, including: Step S3-1, algorithm initialization, set the initial population size M, crossover probability Pc, mutation probability Pm, and maximum evolutionary generations G; Step S3-2, based on the encoding scheme defined in step S2-3, initialize M chromosomes as the initial population; Step S3-3, decoding the M chromosomes into M neural network architectures and using them as the backbone network of the RetinaNet target detection framework to obtain M candidate RetinaNet target detection models; Step S3-4, the seed target detection network mobilenet v2-RetinaNet pre-trained model parameters W obtained in step S2-2 seed Mapped to the M candidate RetinaNet target detection models in step S3-3, respectively, as the initial parameters W of each candidate RetinaNet target detection model ini ; Step S3-5, the individuals whose model parameters are mapped in step S3-4 are based on the training data set D train2 Train and validate the dataset D val Calculate the Loss value of each individual as the fitness value of the individual; Step S3-6, using roulette to select several individuals to form a parent population; Step S3-7, using a single-point crossover operator to perform pairwise crossover on the individuals in the parent population to obtain M new individuals to form the offspring population; Step S3-8, using integer polynomial mutation to perform mutation operation on each individual in the offspring population; Step S3-9, using the model parameter mapping method given in step S3-4, mapping the pre-trained parameters of mobilenet v2-RetinaNet to each individual in the offspring population; Step S3-10, based on the validation data set D val , calculate the fitness value of each individual in the offspring population; Step S3-11, merge the individuals in the offspring population and the parent population, select M new individuals according to the fitness value of the individuals and use environmental selection to form the next generation population, and feed back to step S3-6 until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure; Step S4, decode the optimal individual output in step S3 into the corresponding neural network as the backbone network of RetinaNet, and obtain the target detection model RetinaNet-nas based on the dataset D train The neural network is fully trained until the model converges.

2. The method for constructing an adaptive target detection model based on model parameter mapping according to claim 1, characterized in that: The model parameter mapping steps are: 1) Parameter mapping of model depth: Parameter mapping is performed in units of sub-network modules. The sub-network module of mobilenet v2-RetinaNet contains i bottleneck structures (2≤i≤4). The candidate RetinaNet target detection model to be mapped contains l bottleneck structures (2≤l≤4). The parameter mapping formula for each bottleneck is: f(i)=min(i,l), 2) Model width parameter mapping: The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is Among them, the parameters out and in represent the bottleneck input and output channels, h, w, h i and w i is the spatial size of the feature map, then the width mapping formula is: If: r≤p: If r>p: 3) Convolution kernel parameter mapping The mapping is performed in units of bottleneck modules. The width parameter in the bottleneck module in mobilenet v2-RetinaNet is The width parameter of the bottleneck in the candidate RetinaNet target detection model to be mapped is The formula for convolution kernel mapping is: Among them, k and k 1 is the size of the convolution kernel.

Images