Efficient neural network architecture search method and device for 3D target detection algorithm
By employing a supernet model and multiple evaluation strategies to generate a performance predictor in a 3D object detection algorithm, and combining it with the Pareto evolution algorithm, the problems of high computational resource consumption and low accuracy are solved, and efficient, low-latency neural network architecture search is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2023-06-30
- Publication Date
- 2026-07-28
AI Technical Summary
In 3D object detection algorithms, existing technologies suffer from problems such as high computational resource consumption, low architecture search accuracy, and high latency, which especially affect real-time data processing on vehicle-side platforms with limited computing resources.
We employ a pre-trained supernet model and generate performance predictors for neural network sub-architectures using single-evaluation and zero-evaluation strategies. We then combine a delay predictor and a Pareto evolution algorithm to search for the optimal neural network architecture.
It improves the efficiency and accuracy of neural network architecture search, reduces the consumption of computing resources and latency, and is suitable for vehicle-side platforms with limited computing resources.
Smart Images
Figure CN116796821B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D target detection technology, and in particular to an efficient neural network architecture search method and apparatus for 3D target detection algorithms. Background Technology
[0002] Deep learning is one of the most influential methods in the field of machine learning. With the continuous improvement of machine computing power, deep neural networks have been widely used in various fields. Among them, convolutional neural networks (CNNs) are feedforward neural networks inspired by the receptive field mechanism in biology. They feature local connectivity, weight sharing, and convergence, and are generally composed of convolutional layers, pooling layers, fully connected layers, and softmax layers. The architecture design of a CNN has a significant impact on the expressive power of the entire model. As the difficulty of tasks increases, the scale of models is constantly growing, and the amount of expert work required to design the model structure is also increasing. Therefore, manually designing neural networks is becoming increasingly difficult, necessitating the use of computers to automatically search for suitable neural network schemes for design.
[0003] In related technologies, the application of 3D perception algorithms is becoming increasingly widespread. Autonomous vehicles need to acquire accurate 3D information about dangerous targets and obstacles such as vehicles and pedestrians in the road environment to ensure driving safety.
[0004] However, the application of 3D perception algorithms in practical deployments introduces significant latency, particularly noticeable on vehicle-side computing platforms with limited computing resources, making it less than ideal for real-time data processing in autonomous driving. Specifically, in 2D vision tasks such as classification and object detection, the backbone network, responsible for feature extraction, consumes the most inference time. Furthermore, techniques like quantization, pruning, and neural network architecture search are commonly used in 2D vision tasks to compress the backbone network and reduce latency while maintaining high accuracy. However, in 3D vision, there is currently no neural network search solution that can be directly applied to 3D point cloud perception algorithms, a problem that urgently needs to be addressed. Summary of the Invention
[0005] This application provides an efficient neural network architecture search method and apparatus for 3D object detection algorithms, in order to solve the problems of high computational resource consumption, low network architecture search accuracy, and high latency in the search process.
[0006] The first aspect of this application provides an efficient neural network architecture search method for 3D object detection algorithms, comprising the following steps:
[0007] Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated.
[0008] Based on the single-round evaluation strategy and the zero-round evaluation strategy, the performance of each neural network sub-architecture is evaluated respectively. The optimal neural network sub-architecture is obtained based on the evaluation results, and an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture; and
[0009] The system acquires latency data of at least one neural network sub-architecture, trains a latency predictor based on the latency data, and searches for the at least one neural network sub-architecture using a preset Pareto evolution algorithm based on the performance predictor and the latency predictor to obtain the optimal neural network architecture.
[0010] According to one embodiment of this application, the sampling of at least one neural network sub-architecture based on a pre-trained supernet model, and the generation of a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture, includes:
[0011] Based on the pre-trained supernet model, at least one neural network sub-architecture is sampled in the supernet model, and the weight parameters of the at least one neural network sub-architecture are obtained through the weight parameter sharing of the supernet model.
[0012] Based on the weight parameters of the at least one neural network sub-architecture, a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated.
[0013] According to one embodiment of this application, before obtaining the optimal neural network sub-architecture based on the evaluation result, the method further includes:
[0014] Based on the single-evaluation strategy and the zero-evaluation strategy of the at least one neural network sub-architecture, a single-evaluation predictor and a zero-evaluation predictor are trained respectively to obtain the training results of the single-evaluation predictor and the training results of the zero-evaluation predictor.
[0015] According to one embodiment of this application, training the single-evaluation predictor and the zero-evaluation predictor respectively to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor includes:
[0016] The at least one neural network sub-architecture is serialized into an architecture option sequence, and the architecture option sequence is converted into a dense vector by a preset encoder;
[0017] Based on the dense vector, the at least one neural network sub-architecture is evaluated using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain performance pairs of at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs of at least one neural network sub-architecture based on the zero-evaluation strategy.
[0018] The single-evaluation predictor and the zero-evaluation predictor are trained using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
[0019] According to one embodiment of this application, the step of independently training the optimal neural network sub-architecture using the independent evaluation strategy to obtain a performance predictor for the at least one neural network sub-architecture includes:
[0020] Based on the dense vectors, the optimal neural network sub-architecture is evaluated using the independent evaluation strategy to obtain the performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy.
[0021] The optimal neural network sub-architecture is trained using the independent evaluation strategy to obtain the training result of the independent evaluation strategy. Based on the training result of the independent evaluation strategy, the training result of the single-evaluation predictor and the training result of the zero-evaluation predictor are weighted and fused to obtain the performance predictor of the at least one neural network sub-architecture.
[0022] According to one embodiment of this application, the step of searching for the at least one neural network sub-architecture based on the performance predictor and the delay predictor, combined with the preset Pareto evolution algorithm, to obtain the optimal neural network architecture includes:
[0023] Based on the Pareto evolution algorithm, at least one neural network sub-architecture is sampled to form a population, and the population is iterated.
[0024] Select at least one neural network sub-architecture in the population that meets the preset conditions as the parent, mutate the parent to obtain the mutated at least one neural network sub-architecture, evaluate and train the mutated at least one neural network sub-architecture, add the evaluation and training results to the population, and record the training results.
[0025] Based on the training results recorded, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
[0026] According to the embodiments of this application, an efficient neural network architecture search method for 3D object detection algorithms is proposed. Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and its single-round evaluation strategy and zero-round evaluation strategy are generated. The performance of each neural network sub-architecture is evaluated separately to obtain the optimal neural network sub-architecture. Simultaneously, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. Latency data of at least one neural network sub-architecture is obtained to train a latency predictor. Based on the performance predictor and the latency predictor, a preset Pareto evolutionary algorithm is used to search for at least one neural network sub-architecture to obtain the optimal neural network architecture. This solves the problems of high computational resource consumption and low accuracy and high latency in network architecture search. By employing multiple evaluation strategies to train the corresponding performance predictors, the search efficiency and accuracy of network architecture are improved.
[0027] A second aspect of this application provides an efficient neural network architecture search device for 3D object detection algorithms, comprising:
[0028] A generation module is used to sample at least one neural network sub-architecture based on a pre-trained supernet model, and generate a single-evaluation strategy and a zero-evaluation strategy for the at least one neural network sub-architecture.
[0029] A training module is used to evaluate the performance of each neural network sub-architecture based on the single-round evaluation strategy and the zero-round evaluation strategy, obtain the optimal neural network sub-architecture based on the evaluation results, and independently train the optimal neural network sub-architecture using an independent evaluation strategy to obtain a performance predictor for at least one neural network sub-architecture; and
[0030] An acquisition module is used to acquire latency data of the at least one neural network sub-architecture, train a latency predictor based on the latency data, and search for the at least one neural network sub-architecture based on the performance predictor and the latency predictor, combined with a preset Pareto evolution algorithm, to obtain the optimal neural network architecture.
[0031] According to one embodiment of this application, the generation module is specifically used for:
[0032] Based on the pre-trained supernet model, at least one neural network sub-architecture is sampled in the supernet model, and the weight parameters of the at least one neural network sub-architecture are obtained through the weight parameter sharing of the supernet model.
[0033] Based on the weight parameters of the at least one neural network sub-architecture, a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated.
[0034] According to one embodiment of this application, before obtaining the optimal neural network sub-architecture based on the evaluation result, the training module is further configured to:
[0035] Based on the single-evaluation strategy and the zero-evaluation strategy of the at least one neural network sub-architecture, a single-evaluation predictor and a zero-evaluation predictor are trained respectively to obtain the training results of the single-evaluation predictor and the training results of the zero-evaluation predictor.
[0036] According to one embodiment of this application, the training module is specifically used for:
[0037] The at least one neural network sub-architecture is serialized into an architecture option sequence, and the architecture option sequence is converted into a dense vector by a preset encoder;
[0038] Based on the dense vector, the at least one neural network sub-architecture is evaluated using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain performance pairs of at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs of at least one neural network sub-architecture based on the zero-evaluation strategy.
[0039] The single-evaluation predictor and the zero-evaluation predictor are trained using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
[0040] According to one embodiment of this application, the training module is specifically used for:
[0041] Based on the dense vectors, the optimal neural network sub-architecture is evaluated using the independent evaluation strategy to obtain the performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy.
[0042] The optimal neural network sub-architecture is trained using the independent evaluation strategy to obtain the training result of the independent evaluation strategy. Based on the training result of the independent evaluation strategy, the training result of the single-evaluation predictor and the training result of the zero-evaluation predictor are weighted and fused to obtain the performance predictor of the at least one neural network sub-architecture.
[0043] According to one embodiment of this application, the acquisition module is specifically used for:
[0044] Based on the Pareto evolution algorithm, at least one neural network sub-architecture is sampled to form a population, and the population is iterated.
[0045] Select at least one neural network sub-architecture in the population that meets the preset conditions as the parent, mutate the parent to obtain the mutated at least one neural network sub-architecture, evaluate and train the mutated at least one neural network sub-architecture, add the evaluation and training results to the population, and record the training results.
[0046] Based on the training results recorded, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
[0047] According to embodiments of this application, an efficient neural network architecture search device for 3D object detection algorithms is provided. Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and its single-round evaluation strategy and zero-round evaluation strategy are generated. The performance of each neural network sub-architecture is evaluated separately to obtain the optimal neural network sub-architecture. Simultaneously, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. Latency data of at least one neural network sub-architecture is obtained to train a latency predictor. Based on the performance predictor and the latency predictor, a preset Pareto evolutionary algorithm is used to search for at least one neural network sub-architecture to obtain the optimal neural network architecture. This solves the problems of high computational resource consumption and low accuracy and high latency in network architecture search. By employing multiple evaluation strategies to train the corresponding performance predictors, the search efficiency and accuracy of network architecture are improved.
[0048] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the efficient neural network architecture search method for 3D object detection algorithms as described in the above embodiments.
[0049] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement an efficient neural network architecture search method for 3D object detection algorithms as described in the above embodiments.
[0050] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0051] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0052] Figure 1This is a flowchart illustrating an efficient neural network architecture search method for 3D object detection algorithms provided in an embodiment of this application.
[0053] Figure 2 This is a schematic diagram illustrating the training of a performance predictor according to an embodiment of this application;
[0054] Figure 3 This is a schematic diagram illustrating the training process of a delay predictor according to an embodiment of this application;
[0055] Figure 4 This is a schematic diagram of the overall search according to one embodiment of this application;
[0056] Figure 5 This is a schematic diagram of a search box for an efficient neural network architecture for a 3D object detection algorithm according to an embodiment of this application;
[0057] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0058] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0059] The following describes, with reference to the accompanying drawings, an efficient neural network architecture search method and apparatus for 3D object detection algorithms according to embodiments of this application. Addressing the problems mentioned in the background art, such as the high computational resource consumption and low accuracy / high latency in network architecture search, this application provides an efficient neural network architecture search method for 3D object detection algorithms. In this method, based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and its single-evaluation strategy and zero-evaluation strategy are generated. The performance of each neural network sub-architecture is evaluated separately to obtain the optimal neural network sub-architecture. Simultaneously, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. Latency data of at least one neural network sub-architecture is obtained to train a latency predictor. Based on the performance predictor and the latency predictor, a preset Pareto evolutionary algorithm is used to search for at least one neural network sub-architecture to obtain the optimal neural network architecture. This solves the problems of high computational resource consumption and low accuracy / high latency in network architecture search. By employing multiple evaluation strategies to train the corresponding performance predictors, the search efficiency and accuracy of the network architecture are improved.
[0060] Before introducing the embodiments of this application, we will first introduce the network architecture search methods in related technologies. Related technologies face certain challenges in the process of searching for network architectures, such as a large network search space and long evaluation time for each network architecture. The key component corresponding to a large network search space is the search space itself, and the key components corresponding to the long evaluation time for each network architecture are the evaluation strategy and the search strategy. Regarding the problem of a large network search space, those skilled in the art still need to consume a large amount of computing resources for network searches, which cannot meet the needs of practical applications. Therefore, corresponding solutions are proposed to address the long evaluation time for network architectures.
[0061] Specifically, in terms of network search strategies, predictors can be used to predict the performance of network architectures. These predictors are small, trained neural networks that can evaluate the performance of different network architectures in a very short time, greatly improving search efficiency. During network training, network architecture-performance pairs are needed as training data. However, existing predictor-based methods are often inefficient due to the long evaluation time required for each architecture. Using independent training strategies for evaluation consumes a significant amount of time and resources. For example, if an independent model requires 80 training rounds, the training data for the predictor typically requires thousands of models, which would require hundreds of GPUs (Graphics Processing Units) per day. Therefore, the computational cost is high and the efficiency is low.
[0062] Furthermore, regarding evaluation strategies, the single-round evaluation strategy is a relatively efficient method. The general process involves defining a set of neural network architectures in the search space, constructing a supernet, and sampling sub-networks in the search space each time. These sub-network architectures directly inherit weights from the supernet for training and inference. The evaluation of network architectures is essentially a process of acquiring model weights and performing tests. The single-round evaluation strategy is characterized by training only one supernet, which can then evaluate all its sub-network architectures through parameter sharing, saving the time of training each sub-network architecture individually to acquire weights. However, on the one hand, sub-network architectures directly extracted from the supernet rarely possess high accuracy and cannot be used directly; on the other hand, the results (ground truth) of sub-network architectures and independent training strategies often do not have a high correlation, thus failing to guarantee that the final searched model is optimal. The zero-round evaluation strategy is currently the fastest evaluation strategy, a meta-learning-based neural network architecture search technique that accelerates the search for the optimal neural network architecture by learning commonalities between different tasks. However, the zero-round evaluation strategy sacrifices accuracy for evaluation speed to a greater extent, making it difficult to guarantee the quality of the evaluation.
[0063] Therefore, this application adopts a predictor-based NAS (Network Attached Storage) method, combining the advantages of different evaluation strategies. First, a single-evaluation strategy and a zero-evaluation strategy are used as evaluation methods to train corresponding performance predictors. Then, only a small number of ground truth values (results of independent evaluation strategies) are used to fine-tune the results of the performance predictors. Second, a latency predictor is trained to save time spent on actual deployment and latency testing for each sampled network architecture. Finally, the performance predictor and latency predictor are used in conjunction with the Pareto evolutionary algorithm for multi-objective search to find the optimal network architecture with high performance and low latency. This leverages the high efficiency of single-evaluation and zero-evaluation, improves the accuracy of results through a small number of independent evaluations, and combines the efficient search strategy of predictors and evolutionary algorithms to significantly reduce computational overhead while ensuring search results. To make the implementation of this application clearer, the specific implementation process will be described in the following specific embodiments.
[0064] Figure 1 This is a flowchart illustrating an efficient neural network architecture search method for 3D object detection algorithms according to an embodiment of the present invention.
[0065] like Figure 1 As shown, the efficient neural network architecture search method for 3D object detection algorithms includes the following steps:
[0066] In step S101, based on the pre-trained supernet model, at least one neural network sub-architecture is sampled, and a single-round evaluation strategy and a zero-round evaluation strategy for at least one neural network sub-architecture are generated.
[0067] Furthermore, in some embodiments, based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and a single-round evaluation strategy and a zero-round evaluation strategy for at least one neural network sub-architecture are generated, including: based on a pre-trained supernet model, at least one neural network sub-architecture is sampled in the supernet model, and the weight parameters of at least one neural network sub-architecture are obtained through supernet model weight parameter sharing; and a single-round evaluation strategy and a zero-round evaluation strategy for at least one neural network sub-architecture are generated according to the weight parameters of at least one neural network sub-architecture.
[0068] Specifically, such as Figure 2As shown, in this embodiment, a set of neural network architectures is first defined in the search space, and a supernet model is constructed based on the set. When the supernet model converges, a neural network sub-architecture is randomly sampled in the supernet model to obtain the weight parameters of at least one neural network sub-architecture. After sampling each neural network sub-architecture, only the weight parameters of that neural network sub-architecture are updated, while other weight parameters remain unchanged, so as to complete the training and inference of the supernet model. Thus, a single-evaluation strategy and a zero-evaluation strategy for at least one neural network sub-architecture are generated based on the weight parameters of at least one neural network sub-architecture.
[0069] For example, this application uses CNN (Convolutional Neural Network) as an example. CNN can choose from three dimensions: kernel size, width, and depth. Each dimension can have different options, and the neural network sub-architecture is composed of combinations of options in different dimensions. Assume a CNN network has u units, and the options for the kernel size, width, and depth of each unit are k, w, and d = {x1, x2, ..., x...} d If there are}, then the size N of the search space is:
[0070]
[0071] This shows that the weights of all N neural network sub-architectures are contained in the supernet model. Each neural network sub-architecture only needs to share the corresponding weight parameters from the supernet model for testing during evaluation. The test results will be used to train a single-evaluation predictor, thereby improving the correlation between the performance of the neural network sub-architecture and the ground truth.
[0072] It should be noted that the weight parameter sharing of the supernet model in this application embodiment is selected according to the absolute value of the weight. However, since it is difficult to guarantee the absolute performance of the neural network sub-architecture in different tasks, a more reliable method is to find the neural network sub-architecture with the best relative performance from the supernet model and then train it independently.
[0073] In step S102, the performance of each neural network sub-architecture is evaluated based on the single evaluation strategy and the zero evaluation strategy, and the optimal neural network sub-architecture is obtained according to the evaluation results. The optimal neural network sub-architecture is then trained independently using the independent evaluation strategy to obtain a performance predictor for at least one neural network sub-architecture.
[0074] Furthermore, in some embodiments, before obtaining the optimal neural network sub-architecture based on the evaluation results, the method further includes: training a single-evaluation predictor and a zero-evaluation predictor based on a single-evaluation strategy and a zero-evaluation strategy of at least one neural network sub-architecture, respectively, to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
[0075] Specifically, in this application embodiment, based on at least one sampled neural network sub-architecture, the performance of each neural network sub-architecture is evaluated using a single-evaluation strategy and a zero-evaluation strategy, respectively. Based on the evaluation results, a single-evaluation predictor and a zero-evaluation predictor are trained respectively to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
[0076] Further, in some embodiments, training a single-evaluation predictor and a zero-evaluation predictor respectively to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor includes: serializing at least one neural network sub-architecture into an architecture option sequence, and converting the architecture option sequence into a dense vector using a preset encoder; evaluating at least one neural network sub-architecture using a single-evaluation strategy and a zero-evaluation strategy respectively based on the dense vector to obtain performance pairs of at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs of at least one neural network sub-architecture based on the zero-evaluation strategy; training the single-evaluation predictor and the zero-evaluation predictor using the single-evaluation strategy and the zero-evaluation strategy respectively to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor respectively.
[0077] The preset encoder can be selected by those skilled in the art based on actual training needs, and no specific limitations are made here.
[0078] Specifically, in the process of training a single-evaluation predictor and a zero-evaluation predictor using a single-evaluation strategy and a zero-evaluation strategy for at least one neural network sub-architecture, the first step is to serialize at least one neural network sub-architecture into an architecture option sequence. This architecture option sequence is then converted into a dense vector using a preset encoder, such as an LSTM (Long Short-Term Memory) encoder, which converts the architecture option sequence into a dense vector of dimension 20. This maps discrete architecture information to a continuous representation space. After encoding, the similarity between at least one neural network sub-architecture can be measured by calculating the distance between the vectors. Next, based on the dense vectors, at least one neural network sub-architecture is evaluated using both the single-evaluation strategy and the zero-evaluation strategy, respectively. This yields performance pairs for at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs for at least one neural network sub-architecture based on the zero-evaluation strategy. The single-evaluation predictor and the zero-evaluation predictor are then trained using the single-evaluation strategy and the zero-evaluation strategy, respectively, resulting in training results for the single-evaluation predictor and the zero-evaluation predictor. Finally, the optimal neural network sub-architecture is obtained based on the training results.
[0079] Furthermore, in some embodiments, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. This includes: evaluating the optimal neural network sub-architecture using an independent evaluation strategy based on dense vectors to obtain a performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy; training the optimal neural network sub-architecture using the independent evaluation strategy to obtain the training result of the independent evaluation strategy; and weightedly fusing the training result of the single-evaluation predictor and the training result of the zero-evaluation predictor based on the training result of the independent evaluation strategy to obtain a performance predictor for at least one neural network sub-architecture.
[0080] Specifically, in this embodiment, since the independent evaluation strategy requires a long time to obtain the performance of the neural network sub-architecture, this embodiment evaluates the optimal neural network sub-architecture obtained above based on the dense vector through the independent evaluation strategy, and obtains the performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy, so as to fine-tune the single-evaluation predictor and the zero-evaluation predictor to improve the accuracy of the predictor.
[0081] Furthermore, in this embodiment, the optimal neural network sub-architecture is trained through an independent evaluation strategy to obtain the training results of the independent evaluation strategy. Based on the training results of the independent evaluation strategy, the training results of the single evaluation predictor and the training results of the zero evaluation predictor are weighted and fused using MLP (Multi-Layer Perceptron) and softmax to obtain the performance predictor of at least one neural network sub-architecture.
[0082] In step S103, latency data of at least one neural network sub-architecture is obtained, a latency predictor is trained based on the latency data, and at least one neural network sub-architecture is searched based on the performance predictor and the latency predictor, combined with a preset Pareto evolution algorithm, to obtain the optimal neural network architecture.
[0083] Furthermore, in some embodiments, based on the performance predictor and the delay predictor, and combined with a preset Pareto evolutionary algorithm, at least one neural network sub-architecture is searched to obtain the optimal neural network architecture. This includes: sampling at least one neural network sub-architecture to form a population based on the Pareto evolutionary algorithm, and iterating the population; selecting at least one neural network sub-architecture in the population that meets preset conditions as the parent, and mutating the parent to obtain at least one mutated neural network sub-architecture, and evaluating and training the at least one mutated neural network sub-architecture, adding the evaluation and training results to the population, and recording the training results; and based on the recorded training results, selecting the neural network sub-architecture with the highest score in the training results as the optimal neural network architecture.
[0084] The preset Pareto evolution algorithm and preset conditions can be relevant algorithms and conditions adopted by those skilled in the art according to actual training needs, and are not specifically limited here.
[0085] Specifically, such as Figure 3 As shown, to save time in actually deploying and testing the latency of each sampled neural network architecture, this embodiment of the application also needs to train the latency predictor. Since the training process of the latency predictor is relatively simple, this embodiment of the application can first randomly sample 100 neural network sub-architectures and deploy them to the edge chip to test the latency of each neural network sub-architecture to obtain latency data, and then train the latency predictor based on the latency data; secondly, based on the performance predictor and the latency predictor, a multi-objective joint search is performed using a preset Pareto evolution algorithm to obtain at least one neural network sub-architecture to obtain the optimal neural network architecture.
[0086] Evolutionary algorithms are a class of computational models built upon the laws of natural evolution, simulating biological evolution mechanisms to optimize problems. Their core idea is to iterate through a population, selecting superior individuals as parents, and then subjecting them to mutations to continuously update and optimize the population.
[0087] Furthermore, in the search of neural network architectures, a neural network architecture is an individual, and the performance of the neural network architecture is the condition for judging whether the individual is excellent. Mutation refers to randomly changing one or more options in the option sequence of the neural network architecture to generate a new neural network sub-architecture. The number of genes mutated each time needs to be specified in advance, and the number of mutations is generally related to the length of the option sequence of the neural network architecture.
[0088] Specifically, such as Figure 4 As shown, in this embodiment of the application, based on a performance predictor and a delay predictor, and combined with a preset Pareto evolutionary algorithm, a multi-objective joint search for at least one neural network sub-architecture is performed. First, P neural network architectures are randomly sampled from the search space to form a population, and the population is iterated. Second, in each subsequent iteration, S neural network architectures are extracted from the population and trained using the performance predictor for performance evaluation. The neural network architecture with the highest score is selected as the parent, and the parent is mutated to obtain at least one neural network sub-architecture. Each neural network sub-architecture is then evaluated and trained and added to the population, and the training results are recorded. Finally, based on the recorded training results, after the population is updated, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
[0089] Furthermore, to reduce the repetitiveness of extracting neural network architectures each time, this embodiment of the application can remove the oldest neural network architecture from the population after adding the neural network sub-architecture, keeping the population size unchanged. Therefore, a Pareto front set can be maintained using performance predictors and latency predictors to achieve multi-objective (performance, latency) optimization and select the optimal neural network architecture that meets the requirements of high accuracy and low latency.
[0090] According to the embodiments of this application, an efficient neural network architecture search method for 3D object detection algorithms is proposed. Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and its single-round evaluation strategy and zero-round evaluation strategy are generated. The performance of each neural network sub-architecture is evaluated separately to obtain the optimal neural network sub-architecture. Simultaneously, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. Latency data of at least one neural network sub-architecture is obtained to train a latency predictor. Based on the performance predictor and the latency predictor, a preset Pareto evolutionary algorithm is used to search for at least one neural network sub-architecture to obtain the optimal neural network architecture. This solves the problems of high computational resource consumption and low accuracy and high latency in network architecture search. By employing multiple evaluation strategies to train the corresponding performance predictors, the search efficiency and accuracy of network architecture are improved.
[0091] Next, referring to the accompanying drawings, we describe an efficient neural network architecture search device for 3D object detection algorithms proposed according to embodiments of this application.
[0092] Figure 5 This is a block diagram of an efficient neural network architecture search device for 3D object detection algorithms according to an embodiment of this application.
[0093] like Figure 5 As shown, the efficient neural network architecture search device 10 based on 3D target detection algorithm includes: generation module 100, training module 200 and acquisition module 300.
[0094] The generation module 100 is used to sample at least one neural network sub-architecture based on a pre-trained supernet model, and generate a single-evaluation strategy and a zero-evaluation strategy for at least one neural network sub-architecture.
[0095] Training module 200 is used to evaluate the performance of each neural network sub-architecture based on a single-evaluation strategy and a zero-evaluation strategy, respectively, and to obtain the optimal neural network sub-architecture based on the evaluation results. An independent evaluation strategy is then used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture.
[0096] The acquisition module 300 is used to acquire latency data of at least one neural network sub-architecture, train a latency predictor based on the latency data, and search for at least one neural network sub-architecture based on the performance predictor and the latency predictor, combined with a preset Pareto evolution algorithm, to obtain the optimal neural network architecture.
[0097] Furthermore, in some embodiments, the generation module 100 is specifically used for:
[0098] Based on a pre-trained hypernet model, at least one neural network sub-architecture is sampled in the hypernet model, and the weight parameters of at least one neural network sub-architecture are obtained through weight parameter sharing of the hypernet model.
[0099] Generate a single-round evaluation strategy and a zero-round evaluation strategy for at least one neural network sub-architecture based on the weight parameters of at least one neural network sub-architecture.
[0100] Furthermore, in some embodiments, before obtaining the optimal neural network sub-architecture based on the evaluation results, the training module 200 is also used for:
[0101] Based on at least one neural network sub-architecture, a single-evaluation predictor and a zero-evaluation predictor are trained respectively, and the training results of the single-evaluation predictor and the zero-evaluation predictor are obtained.
[0102] Furthermore, in some embodiments, the training module 200 is specifically used for:
[0103] At least one neural network sub-architecture is serialized into an architecture option sequence, and the architecture option sequence is converted into a dense vector by a pre-defined encoder;
[0104] Based on the dense vector, at least one neural network sub-architecture is evaluated by a single evaluation strategy and a zero evaluation strategy respectively, to obtain the performance pair of at least one neural network sub-architecture based on the single evaluation strategy and the performance pair of at least one neural network sub-architecture based on the zero evaluation strategy.
[0105] The single-evaluation predictor and the zero-evaluation predictor are trained using a single-evaluation strategy and a zero-evaluation strategy, respectively, and the training results of the single-evaluation predictor and the zero-evaluation predictor are obtained.
[0106] Furthermore, in some embodiments, the training module 200 is specifically used for:
[0107] Based on the dense vector, the optimal neural network sub-architecture is evaluated using an independent evaluation strategy to obtain the performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy.
[0108] The optimal neural network sub-architecture is trained using the independent evaluation strategy to obtain the training result of the independent evaluation strategy. Based on the training result of the independent evaluation strategy, the training result of the single evaluation predictor and the training result of the zero evaluation predictor are weighted and fused to obtain the performance predictor of at least one neural network sub-architecture.
[0109] Furthermore, in some embodiments, the acquisition module 300 is specifically used for:
[0110] At least one neural network sub-architecture is sampled to form a population based on the Pareto evolution algorithm, and the population is iterated.
[0111] Select at least one neural network sub-architecture in the population that meets the preset conditions as the parent, mutate the parent to obtain at least one mutated neural network sub-architecture, evaluate and train the at least one mutated neural network sub-architecture, add the evaluation and training results to the population, and record the training results.
[0112] Based on the recorded training results, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
[0113] According to embodiments of this application, an efficient neural network architecture search device for 3D object detection algorithms is provided. Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and its single-round evaluation strategy and zero-round evaluation strategy are generated. The performance of each neural network sub-architecture is evaluated separately to obtain the optimal neural network sub-architecture. Simultaneously, an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture. Latency data of at least one neural network sub-architecture is obtained to train a latency predictor. Based on the performance predictor and the latency predictor, a preset Pareto evolutionary algorithm is used to search for at least one neural network sub-architecture to obtain the optimal neural network architecture. This solves the problems of high computational resource consumption and low accuracy and high latency in network architecture search. By employing multiple evaluation strategies to train the corresponding performance predictors, the search efficiency and accuracy of network architecture are improved.
[0114] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:
[0115] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.
[0116] When the processor 602 executes the program, it implements the efficient neural network architecture search method for 3D target detection algorithms provided in the above embodiments.
[0117] Furthermore, electronic devices also include:
[0118] Communication interface 603 is used for communication between memory 601 and processor 602.
[0119] The memory 601 is used to store computer programs that can run on the processor 602.
[0120] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0121] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0122] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.
[0123] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0124] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described efficient neural network architecture search method for 3D object detection algorithms.
[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0126] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0127] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for efficient neural network architecture search for 3D object detection algorithms, characterized in that, Includes the following steps: Based on a pre-trained supernet model, at least one neural network sub-architecture is sampled, and a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated. Based on the single-round evaluation strategy and the zero-round evaluation strategy, the performance of each neural network sub-architecture is evaluated respectively. The optimal neural network sub-architecture is obtained based on the evaluation results, and an independent evaluation strategy is used to independently train the optimal neural network sub-architecture to obtain a performance predictor for at least one neural network sub-architecture; and The system obtains latency data of at least one neural network sub-architecture, trains a latency predictor based on the latency data, and searches for the at least one neural network sub-architecture based on the performance predictor and the latency predictor, combined with a preset Pareto evolution algorithm, to obtain the optimal neural network architecture. The step of independently training the optimal neural network sub-architecture using the independent evaluation strategy to obtain a performance predictor for at least one neural network sub-architecture includes: Based on the dense vectors, the optimal neural network sub-architecture is evaluated using the independent evaluation strategy to obtain the performance pair of the optimal neural network sub-architecture corresponding to the independent evaluation strategy. The optimal neural network sub-architecture is trained using the independent evaluation strategy to obtain the training result of the independent evaluation strategy. Based on the training result of the independent evaluation strategy, the training result of the single-evaluation predictor and the training result of the zero-evaluation predictor are weighted and fused to obtain the performance predictor of the at least one neural network sub-architecture.
2. The method according to claim 1, characterized in that, The pre-trained supernet model samples at least one neural network sub-architecture and generates a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture, including: Based on the pre-trained supernet model, at least one neural network sub-architecture is sampled in the supernet model, and the weight parameters of the at least one neural network sub-architecture are obtained through the weight parameter sharing of the supernet model. Based on the weight parameters of the at least one neural network sub-architecture, a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated.
3. The method according to claim 1, characterized in that, Before obtaining the optimal neural network sub-architecture based on the evaluation results, the following steps are also included: Based on the single-evaluation strategy and the zero-evaluation strategy of the at least one neural network sub-architecture, a single-evaluation predictor and a zero-evaluation predictor are trained respectively to obtain the training results of the single-evaluation predictor and the training results of the zero-evaluation predictor.
4. The method according to claim 3, characterized in that, The step of training the single-evaluation predictor and the zero-evaluation predictor respectively to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor includes: The at least one neural network sub-architecture is serialized into an architecture option sequence, and the architecture option sequence is converted into a dense vector by a preset encoder; Based on the dense vector, the at least one neural network sub-architecture is evaluated using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain performance pairs of at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs of at least one neural network sub-architecture based on the zero-evaluation strategy. The single-evaluation predictor and the zero-evaluation predictor are trained using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
5. The method according to claim 1, characterized in that, The process of searching for the at least one neural network sub-architecture based on the performance predictor and the delay predictor, combined with the preset Pareto evolution algorithm, to obtain the optimal neural network architecture includes: Based on the Pareto evolution algorithm, at least one neural network sub-architecture is sampled to form a population, and the population is iterated. Select at least one neural network sub-architecture in the population that meets the preset conditions as the parent, mutate the parent to obtain the mutated at least one neural network sub-architecture, evaluate and train the mutated at least one neural network sub-architecture, add the evaluation and training results to the population, and record the training results. Based on the recorded training results, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
6. A high-efficiency neural network architecture search device for 3D object detection algorithms, characterized in that, include: A generation module is used to sample at least one neural network sub-architecture based on a pre-trained supernet model, and generate a single-evaluation strategy and a zero-evaluation strategy for the at least one neural network sub-architecture. A training module is used to evaluate the performance of each neural network sub-architecture based on the single-round evaluation strategy and the zero-round evaluation strategy, obtain the optimal neural network sub-architecture based on the evaluation results, and independently train the optimal neural network sub-architecture using an independent evaluation strategy to obtain a performance predictor for at least one neural network sub-architecture; and An acquisition module is used to acquire latency data of the at least one neural network sub-architecture, train a latency predictor based on the latency data, and search for the at least one neural network sub-architecture based on the performance predictor and the latency predictor, combined with a preset Pareto evolution algorithm, to obtain the optimal neural network architecture. The training module is specifically used for: evaluating the optimal neural network sub-architecture using the independent evaluation strategy based on the dense vector, to obtain a performance pair of the optimal neural network sub-architecture based on the independent evaluation strategy; training the optimal neural network sub-architecture using the independent evaluation strategy to obtain the training result of the independent evaluation strategy; and weightedly fusing the training result of the single-evaluation predictor and the training result of the zero-evaluation predictor based on the training result of the independent evaluation strategy to obtain a performance predictor for the at least one neural network sub-architecture.
7. The apparatus according to claim 6, characterized in that, The generation module is specifically used for: Based on the pre-trained supernet model, at least one neural network sub-architecture is sampled in the supernet model, and the weight parameters of the at least one neural network sub-architecture are obtained through the weight parameter sharing of the supernet model. Based on the weight parameters of the at least one neural network sub-architecture, a single-round evaluation strategy and a zero-round evaluation strategy for the at least one neural network sub-architecture are generated.
8. The apparatus according to claim 6, characterized in that, Before obtaining the optimal neural network sub-architecture based on the evaluation results, the training module is further configured to: Based on the single-evaluation strategy and the zero-evaluation strategy of the at least one neural network sub-architecture, a single-evaluation predictor and a zero-evaluation predictor are trained respectively to obtain the training results of the single-evaluation predictor and the training results of the zero-evaluation predictor.
9. The apparatus according to claim 8, characterized in that, The training module is specifically used for: The at least one neural network sub-architecture is serialized into an architecture option sequence, and the architecture option sequence is converted into a dense vector by a preset encoder; Based on the dense vector, the at least one neural network sub-architecture is evaluated using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain performance pairs of at least one neural network sub-architecture based on the single-evaluation strategy and performance pairs of at least one neural network sub-architecture based on the zero-evaluation strategy. The single-evaluation predictor and the zero-evaluation predictor are trained using the single-evaluation strategy and the zero-evaluation strategy, respectively, to obtain the training results of the single-evaluation predictor and the zero-evaluation predictor.
10. The apparatus according to claim 6, characterized in that, The acquisition module is specifically used for: Based on the Pareto evolution algorithm, at least one neural network sub-architecture is sampled to form a population, and the population is iterated. Select at least one neural network sub-architecture in the population that meets the preset conditions as the parent, mutate the parent to obtain the mutated at least one neural network sub-architecture, evaluate and train the mutated at least one neural network sub-architecture, add the evaluation and training results to the population, and record the training results. Based on the training results recorded, the neural network sub-architecture with the highest score in the training results is selected as the optimal neural network architecture.
11. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the efficient neural network architecture search method for 3D object detection algorithms as described in any one of claims 1-5.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the efficient neural network architecture search method for 3D object detection algorithms as described in any one of claims 1-5.