An efficient evolutionary neural network architecture search method for object detection
Through an evolutionary algorithm, searching for the optimal backbone network architecture of the object detection task in the hypernetwork search space, solving the problems of high computing cost and low efficiency in the existing technology, and achieving more efficient feature extraction and network architecture optimization.
Patent Information
- Application Number
- CN202110675709.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-18
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-06-18
AI Technical Summary
In the prior art, when using neural networks designed based on image classification tasks as the backbone network for object detection, the calculation cost is high and not optimized enough. The traditional neural network architecture search technology requires re-training on large data sets, resulting in inefficiency.
The population is initialized by an evolutionary algorithm mechanism, and the hyper-network search space is searched, combined with image classification and object detection data sets are pre-trained, and the evolutionary algorithm is used to search the optimal backbone network architecture to avoid each individual's re-training and save computational costs.
It improves the feature extraction capability of the backbone network, reduces the computing cost, improves the efficiency of the object detection task, and realizes better neural network architecture search.
Smart Images

Figure CN113920514B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of digital image processing, and in particular relates to an efficient evolutionary neural network architecture search method for target detection. Background Art
[0002] In object detection networks, it is usually necessary to use a backbone network to extract features from images. Most commonly used backbone networks are designed based on image classification tasks, such as VggNet, ResNet, and GoogleNet. The goal of image classification tasks is to identify the objects contained in the image and classify different images into different categories.
[0003] Target detection requires first locating the target object, and then determining the category information of the current target object. Therefore, directly using a neural network designed for image classification tasks as the backbone network may not be the optimal solution for target detection tasks. In addition, the backbone network usually needs to be pre-trained on a large dataset. However, traditional neural network architecture search technology requires sampling neural networks from the search space, then training their weights from scratch, and then evaluating their performance, which is computationally expensive for large datasets. Summary of the invention
[0004] The purpose of the present invention is to solve the problems existing in the above-mentioned prior art and provide an efficient evolutionary neural network architecture search method for target detection. By using the method of the present invention, a backbone network can be efficiently designed for target detection tasks.
[0005] In order to solve the above technical problems, the present invention provides an efficient evolutionary neural network architecture search method for target detection, which is characterized by comprising the following steps:
[0006] Step S1: given a labeled image classification dataset, divide the dataset into a training dataset C train , validation dataset C val And the test dataset C test ; Given a labeled target detection dataset, divide the dataset into training dataset D train , validation dataset D val And the test dataset D test ;
[0007] Step S2, population initialization, the initial population includes M individuals, each individual includes an input layer, encapsulates several computing modules, and uses a fully connected layer as an output layer. M individuals together form a super network, where M is a natural number greater than 1;
[0008] Step S3, perform multiple random re-sampling of individuals in the population, using the training data set C trainThe sampled individuals are trained and the training weights are retained for each individual. train After the traversal is completed, the super network obtains the weight W A ;
[0009] Step S4: All individuals inherit the relevant weights obtained from the training of the hypernetwork, based on the validation set C val Evaluate individual performance and record the best individual based on the validation set C val Loss value
[0010] Step S5, repeat steps (2)-(4) until the hypernetwork model converges;
[0011] Step S6: Fuse feature pyramid networks (FPN) for each individual, and each individual becomes a target detection model. train , fine-tune the weights of all individuals until the hypernetwork model converges, and save the optimal hypernetwork weights
[0012] Step S7, using the mAP value as the fitness function to evaluate the fitness of each individual;
[0013] Step S8, using roulette to select several individuals to form a parent population;
[0014] Step S9, using a crossover operator to perform a pairwise crossover on the individuals in the parent population to obtain M new individuals to form a child population;
[0015] Step S10, based on the verification data set D val , the offspring individuals inherit the super network weights Calculate the fitness value of each individual in the offspring population;
[0016] Step S11, merge the individuals in the offspring population and the parent population, select M new individuals according to the fitness value of the individuals and use environmental selection to form the next generation population, and feed back to step (8) until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure.
[0017] Preferably, in step S31, the individuals in the population are randomly sampled multiple times, that is, each mini-batch randomly samples an individual from the initial population, and the individual w i The probability of being sampled is:
[0018]
[0019] Among them, M is the total number of individuals in the population, that is, each individual is sampled uniformly.
[0020] Step S32, use the training data set to train the sampled individuals and save the optimized weight w*(a). The optimization goal is to minimize the neural network architecture a based on the validation set C val The loss value The objective function is:
[0021]
[0022]
[0023] Among them, A is the search space, that is, the hypernetwork, a is the sampled neural network architecture, w is the unoptimized weight, and the neural network architecture a is trained by the training data set, by minimizing the training set C train Loss value Get the optimized weight w*(a).
[0024] Preferably, in step S41, all C train After the traversal is completed, the relevant weights obtained from the training are used to evaluate the trained individual C val The performance on the above, and record the best individual based on the loss value The loss function here is the cross entropy loss function. The calculation formula is:
[0025]
[0026]
[0027] Among them, w A is the weight of the hypernetwork A after training and optimization, For the neural network architecture a based on the validation set C val The loss value of , the optimization goal here is to find The neural network architecture with the smallest value is selected and the loss value is recorded to determine whether the hypernetwork has converged.
[0028] Preferably, step S61 is to fuse feature pyramid networks (FPN) for each individual, using D train , fine-tune the weights of all individuals. The fine-tuning method is:
[0029] Step S62, perform multiple random re-sampling of individuals in the population, using the training data set D train The sampled individuals are trained, and the training weights are updated and retained for each individual. The optimization goal is to minimize the validation set D val The loss value The calculation formula is as follows:
[0030]
[0031]
[0032] Among them, w p (w)* is the neural network architecture a based on the training set C train The optimized weights, w*(a), are the neural network architecture a through minimization of the validation set D val The loss value is the weight after optimization is completed.
[0033] Preferably, in step S71, the Map value of each individual in the population is calculated as the fitness value F(x i ), where i = (1, 2, 3, ..., M);
[0034] Preferably, step S81 calculates the probability P(x i ), the formula is:
[0035]
[0036] Preferably, in step S82, the cumulative probability of any individual is calculated using the formula:
[0037]
[0038] Preferably, in step S83, a random number is randomly generated in the interval [0, 1], and the random number is associated with any individual in any population, and the position of the random number in the interval is used to determine the number of times the individual is selected.
[0039] Preferably, a single-point crossover operator is used to perform pairwise crossover on individuals in the parent population, including the following steps:
[0040] Preferably, in step S91, the computing modules of two individuals are aligned, an intersection is randomly selected, and the computing modules after the intersection are exchanged.
[0041] Compared with the prior art, the above application has the following advantages or beneficial effects: the present invention relies on the initial population in the evolutionary algorithm mechanism to establish a hypernetwork search space, and through the hypernetwork-based neural network architecture search technology, adaptively constructs a backbone network for specific target detection problems, so that the backbone network has better feature extraction capabilities. In addition, the present invention establishes a search space for a neural network architecture through a hypernetwork, and pre-trains the hypernetwork using a large image classification data set, and then uses an evolutionary algorithm as a search strategy to search for the optimal backbone network in the pre-trained hypernetwork, avoiding the need for each individual to use an image classification data set for pre-training, thereby saving a lot of computing costs and improving search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a schematic diagram of a neural network architecture of the present invention.
[0043] Figure 2 It is a schematic diagram of encoding and decoding of a computing module of the present invention.
[0044] Figure 3 It is a schematic diagram of a computing node of the present invention.
[0045] Figure 4 Schematic diagram of the crossover operator of the present invention. DETAILED DESCRIPTION
[0046] The present invention is further described in detail with reference to the accompanying drawings and in conjunction with specific embodiments.
[0047] Embodiment: In order to achieve the above-mentioned object, the present application provides an efficient evolutionary neural network architecture search method for target detection, comprising the following steps:
[0048] Step S1: Given a labeled image classification dataset ImageNet, divide the dataset into a training dataset C train , validation dataset C val And the test dataset C test ; Given a labeled target detection dataset Coco, divide the dataset into a training dataset D train , validation dataset D val And the test dataset D test ;
[0049] Step S2, population initialization, the initial population contains M individuals, each individual contains an input layer, encapsulates several computing modules, and uses a fully connected layer as the output layer. Each individual is a neural network, and M individuals together form a super network, where M is a natural number greater than 1; the input layer is a convolution layer with a convolution kernel of 3.
[0050] like Figure 1 As shown in Figure 1, it contains three types of computing modules. Figure 2 As shown, each computing module contains 2 input nodes, 1 output node and 8 computing nodes. The input node is used to receive the output of the previous computing module, and the output node is a Concat node, which is the output of the current computing module. In the current computing module, if the output of some computing nodes is not used as the input of other computing nodes, the output of the computing nodes will be spliced together along the depth direction through the Concat operation as the output of the computing module.
[0051] like Figure 3As shown in the figure, each computing node contains two inputs and two neural network operations. Each computing node can only use the output of its predecessor node as the input of this node. The optional neural network operations are: depthwise separable convolution layer with convolution kernel of 3, depthwise separable convolution layer with convolution kernel of 5, maximum pooling layer, average pooling layer, dilated convolution with convolution kernel of 3, and dilated convolution with convolution kernel of 5.
[0052] Therefore, a computing node can be encoded with a 4-bit vector (x1, x2, O1, O2), where x1 and x2 represent the two inputs of the computing node: input 1 and input 2 respectively, and O1 and O2 represent the two neural network operations contained in the node at this stage. The outputs of the two neural network operations are added together and used as the output of the computing node.
[0053] Step S3, perform multiple random re-sampling of individuals in the population, using the training data set C train The sampled individuals are trained and the training weights are retained for each individual. train After the traversal is completed, the super network obtains the weight W A ;
[0054] Step S31, perform multiple random re-sampling of individuals in the population, that is, each mini-batch randomly samples an individual from the initial population, and the individual w i The probability of being sampled is:
[0055]
[0056] Among them, M is the total number of individuals in the population, that is, each individual is sampled uniformly.
[0057] Step S32, use the training data set to train the sampled individuals and save the optimized weight w*(a). The optimization goal is to minimize the neural network architecture a based on the validation set C val The loss value The objective function is:
[0058]
[0059]
[0060] Among them, A is the search space, that is, the hypernetwork, a is the sampled neural network architecture, w is the unoptimized weight, and the neural network architecture a is trained by the training data set, by minimizing the training set C train Loss value Get the optimized weight w*(a).
[0061] Step S4: All individuals inherit the relevant weights obtained from the training of the hypernetwork, based on the validation set Cval Evaluate individual performance and record the best individual based on the validation set C val Loss value
[0062] Step S41, all C train After the traversal is completed, the relevant weights obtained from the training are used to evaluate the trained individual C val The performance on the above, and record the best individual based on the loss value The loss function here is the cross entropy loss function. The calculation formula is:
[0063]
[0064]
[0065] Among them, w A is the weight of the hypernetwork A after training and optimization, For the neural network architecture a based on the validation set C val The loss value of , the optimization goal here is to find The neural network architecture with the smallest value is selected and the loss value is recorded to determine whether the hypernetwork has converged.
[0066] Step S5, repeat steps (2)-(4) until the hypernetwork model converges;
[0067] Step S6, for each individual, feature pyramid networks (FPN) are integrated, so each individual becomes a target detection model. When integrating the FPN network, a set of convolution layers with a convolution kernel of 1 is used by default to reduce the number of channels and match the FPN network. First, 1×1 convolution is performed on the calculation module 3 to reduce the number of channels to obtain P7, and then P6, P5, P4, P3, and P2 are obtained by performing 2 times nearest neighbor upsampling in sequence, where the feature maps of the P6, P5, P4, P3, and P2 layers need to be the same length and width as the corresponding calculation module feature maps in order to perform element-by-element addition. After the upsampling is completed, the FPN network fuses the image feature information at each level, that is, the shallow features with high resolution but insufficient semantic information and the deep features with rich semantic information but low resolution are integrated. Among them, in the P3-P7 layers, each layer has 9 anchors, and the sizes of the anchors are defined as: 32×32, 64×64, 128×128, 256×256, 512×512, and the ratios are set to 1:2, 1:1, 2:1. The prediction value output by the FPN network includes the prediction value (Dets), which contains the 4-bit regression vector (x, y, w, h) of the object category and the position of the border, and the score of the object, which is generated by the classification layer and the regression layer. The label value (GTs) contains the 4-bit true value vector of the object category and the position of the border. For each anchor, there is an N-dimensional one-bit effective encoding (one-hot) vector, where N is the number of object categories. For the classification layer, each anchor must have N categories, that is, an N-dimensional vector, representing the probability of each category. Based on one-hot encoding, the highest probability value is set to 1, and the remaining N-1 categories are set to 0. In addition, the 4-bit regression vector (x, y, w, h) of each anchor is output by the regression layer to predict the offset between the anchor and its corresponding label box position. The classification layer and regression layer parameters are not shared.
[0068] Using D train , fine-tune the weights of all individuals until the hypernetwork model converges, and save the optimal hypernetwork weights The fine-tuning method is:
[0069] Step S62, perform multiple random re-sampling of individuals in the population, using the training data set D train The sampled individuals are trained, and the training weights are updated and retained for each individual. The optimization goal is to minimize the validation set D val The loss value The calculation formula is as follows:
[0070]
[0071]
[0072] Among them, w p (w)* is the neural network architecture a based on the training set C train The optimized weights, w*(a), are the neural network architecture a through minimization of the validation set D val The weight after loss value optimization is completed
[0073] Step S7, using the mAP value as the fitness function to evaluate the fitness of each individual;
[0074] Step S71, calculate the Map value of each individual in the population as the fitness value F(x i ), where i = (1, 2, 3, ..., M);
[0075] Step S8, using roulette to select several individuals to form a parent population;
[0076] Step S81, calculate the probability P(x i ), the formula is:
[0077]
[0078] Step S82, calculate the cumulative probability of any individual, the formula is:
[0079]
[0080] Step S83, randomly generate a random number in the interval [0, 1], and correspond the random number to any individual in any population, and use the position of the random number in the interval to determine the number of times the individual is selected.
[0081] Step S9, using a crossover operator to perform a pairwise crossover on the individuals in the parent population to obtain M new individuals to form a child population; using a single-point crossover operator to perform a pairwise crossover on the individuals in the parent population, including the following steps:
[0082] Step S91, align the computing modules of two individuals, randomly select an intersection, and exchange the computing modules after the intersection.
[0083] Step S10, based on the verification data set D val , the offspring individuals inherit the super network weights Calculate the fitness value of each individual in the offspring population;
[0084] Step S11, merge the individuals in the offspring population and the parent population, and select M new individuals according to the fitness value of the individuals by using environmental selection. The selection method is to sort the individuals in the population according to the fitness value, select the top M individuals as the next generation population, and feed back to step (8) until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure.
Claims
1. An efficient evolutionary neural network architecture search method for target detection, characterized in that: The following steps are involved: Step S1: given a labeled image classification dataset, divide the dataset into a training dataset C train , validation dataset C val And the test dataset C test ; Given a labeled target detection dataset, divide the dataset into training dataset D train , validation dataset D val And the test dataset D test ; Given the evolutionary algorithm parameters, including the number of individuals in the population M, the crossover probability P c , evolutionary algebra G; Step S2, population initialization, the initial population contains M individuals, each individual is a neural network, each individual contains an input layer, encapsulates several computing modules, and uses a fully connected layer as an output layer. M individuals together form a super network, where M is a natural number greater than 1; Step S3, perform multiple random re-sampling of individuals in the population, using the training data set C train The sampled individuals are trained and the training weights are retained for each individual. train After the traversal is completed, the super network obtains the weight W A ; Step S4: All individuals inherit the relevant weights obtained from the training of the hypernetwork, based on the validation set C val Evaluate individual performance and record the best individual based on the validation set C val Loss value Step S5, repeating steps S2-S4 until the hypernetwork model converges; Step S6, fusion feature pyramid network for each individual, each individual becomes a target detection model, using D train , fine-tune the weights of all individuals until the hypernetwork model converges, and save the optimal hypernetwork weights Step S7, using the mAP value as the fitness function to evaluate the fitness of each individual; Step S8, using roulette to select several individuals to form a parent population; Step S9, using a crossover operator to perform a crossover between two individuals in the parent population to obtain M new individuals to form a child population; Step S10, based on the verification data set D val , the offspring individuals inherit the super network weights Calculate the fitness value of each individual in the offspring population; Step S11, merge the individuals in the offspring population and the parent population, select M new individuals according to the fitness value of the individuals and use environmental selection to form the next generation population, and feed back to step S8 until the preset evolutionary generation is reached, and output the individual with the best fitness value as the optimal backbone network structure.
2. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: The individuals in the population are randomly sampled multiple times, that is, each mini-batch randomly samples an individual from the initial population, and the individual w i The probability of being sampled is: Among them, M is the total number of individuals in the population, that is, each individual is sampled uniformly.
3. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 2, characterized in that: Use the training data set to train the sampled individuals and save the optimized weights w * (a), the optimization goal is to minimize the neural network architecture a based on the validation set C val The loss value The objective function is: Among them, A is the search space, that is, the hypernetwork, a is the sampled neural network architecture, w is the unoptimized weight, and the neural network architecture a is trained by the training data set, by minimizing the training set C train Loss value Get the optimized weight w * (a).
4. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: All C train After the traversal is completed, the relevant weights obtained from the training are used to evaluate the trained individual C val The performance on the above, and record the best individual based on the loss value The loss function here is the cross entropy loss function, and the calculation formula is: Among them, w A is the weight of the hypernetwork A after training and optimization, For the neural network architecture a based on the validation set C val The loss value of , the optimization goal here is to find The neural network architecture with the smallest value is selected and the loss value is recorded to determine whether the hypernetwork has converged.
5. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: For each individual, the feature pyramid network is integrated and D train , fine-tune the weights of all individuals.
6. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: The individuals in the population are randomly sampled multiple times, using the training data set D train The sampled individuals are trained, and the training weights are updated and retained for each individual. The optimization goal is to minimize the validation set D val The loss value The calculation formula is as follows: Among them, w p (w) * For the neural network architecture a based on the training set C train The optimized weight, w * (a) For neural network architecture a, minimize the validation set D val The loss value is the weight after optimization is completed.
7. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: Calculate the Map value of each individual in the population as the fitness value F(x i ), where i = (1, 2, 3, …, M).
8. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: Calculate the probability P(x) that any individual in the parent population will be inherited by the next generation i ), the formula is: 。 9. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: Calculate the cumulative probability of any individual, the formula is: Step S83, randomly generate a random number in the interval [0,1], and correspond the random number to any individual in any population, and use the position of the random number in the interval to determine the number of times the individual is selected.
10. The method for searching an efficient evolutionary neural network architecture for target detection according to claim 1, characterized in that: A single-point crossover operator is used to perform pairwise crossover on individuals in the parent population, including the following steps: aligning the computational modules of two individuals, randomly selecting a crossover point, and exchanging the computational modules after the crossover point.
