An efficient differentiable neural architecture search method based on topology optimization

Abstract

The application discloses a kind of high-efficiency differentiable neural architecture search methods based on topological structure optimization, comprising the following steps: according to computer vision task classification form, determine model structure and search network, form the computational graph of searchable structure; The architecture parameters of computational graph are determined by structure parameters and network parameters;Iterative optimization is carried out to the topological structure of computational graph, redundant structure is eliminated, and structure parameters are optimized;Carry out convolution operation;Optimize network parameters by convolution kernel reparameterization;And by convolution kernel standardization, decouple network parameters and structure parameters;Gradient descent algorithm is obtained after the optimized computational graph runs, and the optimal architecture parameter is determined;Train model structure, complete computer vision task classification.This method simplifies the network topological structure of differentiable neural architecture search, and sets the optimization algorithm of simplified structure targetedly;Help to complete network light weight and neural architecture search in complex visual task application.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and computer vision, and more specifically to an efficient differentiable neural architecture search method based on topology optimization. Background Technology

[0002] Neural Architecture Search (NAS) is a method for automatically designing neural network structures. It can search for network structures that are suitable for the characteristics of a given dataset and search space, avoiding the complexity and inconvenience of manually adjusting the network structure.

[0003] In practical applications, neural architecture search methods can significantly reduce the workload of manually designing and debugging network architectures, lowering the cost of model development and optimization. Simultaneously, through neural architecture search, models can maintain high efficiency and superior performance across various tasks, no longer limited by manual design, and better adapt to diverse and ever-changing application requirements. Therefore, research based on neural architecture search is of significant importance in realizing the automated design, optimization, and evolution of deep learning models. Exploring efficient neural architecture search methods can further promote the development of deep learning technology in fields such as image processing and natural language processing, providing more efficient, flexible, and reliable solutions for achieving highly intelligent and intelligent applications.

[0004] Currently, neural architecture search can be mainly divided into two categories: discrete neural architecture search and differentiable neural architecture search. Discrete neural architecture search includes reinforcement learning-based neural architecture search and evolutionary algorithm-based neural architecture search. The characteristic of this type of search method is that its search space is a discrete set. The search space refers to the predefined range of model structures that the algorithm is allowed to explore when running a neural architecture search method. The discrete search space makes it difficult for different network structures to share training information, so when attempting to search for new network structures, old network parameters cannot be reused or only a portion can be reused. Algorithms typically require multiple complete retraining processes to evaluate the merits of different network architectures in the search space; therefore, this type of algorithm is generally very time-consuming, often requiring several days of operation from a large number of graphics computing units to complete a single search.

[0005] Differentiable neural architecture search transforms a difficult discrete search problem into an optimization problem solvable by gradient descent. This method requires significant computational resources because it simultaneously computes multiple possible network branch structures. Topology, a combination of operators and branches within a neural network, can be analyzed and simplified through detailed analysis. This significantly increases the efficiency of the neural architecture search process, reducing computational resources and time consumption, allowing for the search of large-scale branch structures with minimal computation.

[0006] Differentiable neural architecture search simultaneously computes features on all possible network branches during the search process and merges them into a single feature output through a weighted sum based on the branch weights. Gradient descent optimizes both network weights and branch weights, ultimately selecting the optimal network architecture based on the highest branch weight. Differentiable neural architecture search is generally more efficient than discrete neural architecture search, but because it requires simultaneous computation of all possible branches, current differentiable neural architecture search methods consume many times more GPU memory resources than ordinary network structures, and incur greater computational cost and time. The larger the number of optional operators in the search space, the more combinations of model structures the algorithm can explore. As the number of optional operators increases, the computational cost of current techniques also increases linearly, making it particularly difficult to expand the number of optional operators in existing algorithms.

[0007] Therefore, providing an efficient search method for differentiable neural architectures based on topology optimization, while avoiding greater computational cost and time consumption, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides an efficient differentiable neural architecture search method based on topology optimization, which can solve the problems of high computational resource consumption, high memory resource occupation, difficulty in expanding the search space, excessive time consumption, and low parameter reusability in complex vision tasks, so as to reduce computational resource consumption, speed up the search, and improve search efficiency and effectiveness.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] An efficient differentiable neural architecture search method based on topology optimization includes the following steps:

[0011] S1. Based on the form of the computer vision task, determine the model structure and search network to form a computational graph with searchable structure. The computation graph The architecture parameters are determined by the structural parameters and the network parameters;

[0012] S2, Regarding the computation graph The topology is iteratively optimized to eliminate redundant structures and optimize the structural parameters.

[0013] S3. Perform convolution operations on the optimized topology; optimize the network parameters by reparameterizing the convolution kernel; and decouple the network parameters and the structural parameters by standardizing the convolution kernel; to obtain the optimized computation graph.

[0014] S4. The optimized computation graph Input an image classification dataset or an image semantic segmentation dataset, run the gradient descent algorithm to determine the optimal architecture parameters, and complete the search for the optimal architecture of the classification or segmentation model; train the model structure corresponding to the optimal architecture parameters to complete the computer vision classification or semantic segmentation task.

[0015] Furthermore, regarding the computation graph The topology is iteratively optimized to eliminate redundant structures; including:

[0016] Search for parallel structures in the topology to obtain common front modules, common back modules, and common floating modules on different branches;

[0017] For the common front-end module and common rear-end module structure in the topology, an optimization mode of partial module sharing is applied to merge all the common front-end modules and all the common rear-end modules.

[0018] For the common suspended module structure in the topology, the optimization mode of shared suspended modules is applied to merge all the common suspended modules.

[0019] Furthermore, the optimization modes shared by the aforementioned modules include:

[0020] When the input and output parts of some candidate operations share common convolutional layers, nonlinear layers, or regularization layers, the operation process is as follows:

[0021]

[0022] Where x is the feature map of the input neural architecture search module in the convolutional neural network, i is the index of the candidate operation, and o s1 (·) represents a common computation module that is decoupled from the candidate operation input part. s2 (·) represents a common computation module that is decoupled from the candidate operation output. For the unique operation module of the i-th candidate operation, o i (·) represents other candidate operations that do not share common features. This represents the set of candidate operation indices for which there is no shared computation module. This represents the set of candidate operation indices that exist in a common operation module;

[0023] By adjusting the structural parameter α j The candidate operation weights, calculated using the softmax function, represent the selection probability of each candidate operation, where M is the number of candidate operations and i and j are the indices of the candidate operations. To isolate shared components, the operations of shared modules are treated as a single operation, denoted as o. u (x); then the module is decomposed into two cascaded operation selection branches, denoted as g(x):

[0024]

[0025] in and The candidate computation probabilities are derived from two independent sets of structural parameters; considering the potential prior bias introduced by the two sets of parameters, the new structural parameters are reparameterized using the original structural parameters, letting...

[0026]

[0027] Then g(x) is restated as:

[0028]

[0029] in β j This indicates the probability that a candidate operation is a non-shared operation.

[0030] Furthermore, the optimization modes shared by the suspended modules include:

[0031] When the intermediate dangling parts of some candidate operations contain shared convolutional, nonlinear, or regularization layer operation modules, the original topological structure is expressed as follows:

[0032]

[0033] in For independent computation modules before and after the suspended module, o s This is a suspended shared computing module. By adjusting the structural parameter α j The candidate operation weights, calculated using the softmax function, represent the selection probability of each candidate operation, where M is the number of candidate operations and i and j are the indices of the candidate operations. The optimized computation process is as follows:

[0034]

[0035] Where h(x) is the intermediate result selected in the first operation, and g(x) is the result selected in the second operation.

[0036] Furthermore, the convolution kernel reparameterization includes: converting the convolution operation into an equivalent standard convolution operation, and swapping the order of operations for the weighted sum and the convolution operation; wherein, the conversion process for different types of convolution kernels is as follows:

[0037] Standard convolution: First, the shape of the merged convolution kernel is determined by the maximum width and height of all candidate convolution kernels. Then, all candidate convolution kernels are zero-padded into the merged shape, and weighted summation can be performed.

[0038] Dilated convolution: First, initialize a compact convolution kernel with a shape of k×k, where k is the size of the convolution kernel. Then, fill in zeros between columns and rows to convert it into a dilated convolution kernel, which can then perform standard weighted summation operations.

[0039] Depthwise separable convolution: consists of a k×k grouped convolution and a 1×1 standard convolution; let C I C O The number of input and output feature map channels; grouped convolution has C I Let there be a k×k convolution kernel, denoted as . Standard convolution has C I ·C O A convolution kernel of shape 1×1, denoted as Depthwise separable convolution is represented as

[0040]

[0041] Where * denotes convolution, x i This represents the component of the i-th channel of the input feature map. y represents the component of the i-th channel of the feature map after grouped convolution operation. j This represents the component of the j-th channel of the output feature map; considering the 1×1 convolution kernel as a scalar, the above equation can be rewritten as...

[0042]

[0043] in This is equivalent to a standard convolution kernel.

[0044] Furthermore, the kernel standardization includes:

[0045] Remember K i The original convolutional kernel is used as the basis for standardization, which yields the standardized convolutional kernel.

[0046]

[0047] Here, mean and std represent the calculation of mean and standard deviation.

[0048] As can be seen from the above technical solution, the present invention discloses an efficient differentiable neural architecture search method based on topology optimization. Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0049] 1. Reduced computational resource requirements: This invention proposes a merged optimization of convolution operators in the search space. By utilizing the weighting of linear operators and structural parameters, as well as the property of interchangeable operation order, a convolution operator search without additional computation is achieved, thereby significantly reducing the huge computational resources required to simultaneously compute all possible branches.

[0050] 2. Improved search stability: This invention introduces convolution kernel reparameterization and normalization algorithms. These algorithms decouple model parameters from structural parameters, achieving stable and fast model architecture search.

[0051] 3. A generalized and efficient search method: Based on this invention, a model-independent plug-and-play module can be proposed. Through an iterative topology optimization algorithm, it can be applied to various search spaces and network structures, providing support for neural architecture search needs in various fields.

[0052] This invention simplifies the network topology of differentiable neural architecture search from the perspective of optimizing search efficiency, and sets up targeted optimization algorithms for the simplified structure. It provides a theoretical and technical foundation for the application of network lightweighting and neural architecture search in complex visual tasks, and provides theoretical support and technical verification for research in the academic field of artificial intelligence and computer vision, as well as the implementation of related products in the industry. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 The flowchart of the efficient differentiable neural architecture search method based on topology optimization provided by this invention is shown.

[0055] Figure 2 The diagram shows the optimization process for shared modules provided by this invention; in the diagram, (a) is a schematic diagram of the local topology before optimization; and (b) is a schematic diagram of the local topology after optimization.

[0056] Figure 3The diagram shows the optimization process for the suspended module provided by this invention; (a) is a schematic diagram of the local topology before optimization; and (b) is a schematic diagram of the local topology after optimization.

[0057] Figure 4 This is a diagram illustrating the kernel reparameterization process provided by the present invention.

[0058] Figure 5 A comparison chart of the accuracy and efficiency of various neural architecture search methods provided by this invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] This invention discloses an efficient differentiable neural architecture search method based on topology optimization, comprising the following steps:

[0061] S1. Based on the form of the computer vision task, determine the model structure and search network to form a computational graph with searchable structure. Calculation graph The architecture parameters are determined by the structural parameters and the network parameters;

[0062] S2, Calculation graph The topology is iteratively optimized to eliminate redundant structures and optimize structural parameters;

[0063] S3. Perform convolution operations on the optimized topology; optimize network parameters by reparameterizing the convolution kernel; and decouple network parameters and structural parameters by standardizing the convolution kernel; thus obtaining the optimized computational graph.

[0064] S4. Optimized computation graph Input an image classification dataset or an image semantic segmentation dataset, run the gradient descent algorithm to determine the optimal architecture parameters, and complete the search for the optimal architecture of the classification or segmentation model; train the model structure corresponding to the optimal architecture parameters to complete the computer vision classification or semantic segmentation task.

[0065] The efficient differentiable neural architecture search method based on topology optimization according to the present invention can be applied to computer vision classification tasks. For example, in image classification tasks, a convolutional neural network (CNN) can be identified as the search network, and a searchable computation graph can be constructed. Specifically, this method can be used to search for a CNN architecture suitable for image classification tasks. First, a basic CNN architecture, such as VGG or ResNet, can be identified and used as the search network. Then, a searchable computation graph can be constructed, containing multiple convolutional layers, pooling layers, and fully connected layers. Next, the topology of the computation graph can be iteratively optimized to eliminate redundant structures and optimize structural parameters. For example, structural parameters can be optimized by reducing the number of convolutional kernels or changing the size of the convolutional kernels. Then, convolution operations can be performed on the optimized topology, and network parameters can be optimized by reparameterizing and normalizing the convolutional kernels, decoupling network parameters and structural parameters. Finally, the gradient descent algorithm can be run to determine the optimal architecture parameters, and the model structure corresponding to the optimal architecture parameters can be trained to complete the image classification task.

[0066] This method can be applied to various computer vision tasks, such as object detection and semantic segmentation. By searching for the optimal neural network architecture, the performance of the model can be improved while reducing its computational complexity, thus achieving efficient differentiable neural architecture search.

[0067] In this embodiment, it is first necessary to clarify the application scenario and task requirements of this method.

[0068] The method of this invention can not only complete simple tasks such as image classification and text classification, but also complete more complex visual tasks at the same cost.

[0069] The search for differentiable neural architectures is a type of optimization task. In this embodiment, the task requirements are defined by the following formula:

[0070]

[0071] Where α is the structural parameter, which is a set of parameters characterizing the network structure obtained by encoding the structure of the network to be searched. * Let be the optimal structural parameters to be found; w be the learnable parameters of the network itself, including the parameters in all possible network structural branches; given the training set D = {x} i ,y i}, where t1 and t2 are two subsets obtained by further dividing the given dataset. and These are the loss functions for the given network parameters and structural parameters on these two datasets, respectively.

[0072]

[0073] Where CE represents the cross-entropy loss function, f(x,w,α) is the function mapping relationship expressed by the model itself, determined by the structural parameter α and the model parameter w, and x,y are the samples and their corresponding labels in the dataset. Since neural architecture search methods do not depend on a specific task or network structure, the specific form of x,y is determined by the specific task. For example, when performing neural architecture search on an image classification task, The input RGB image is represented by W and H, respectively, while y is an enumeration value representing the corresponding image category label; when performing neural architecture search for image semantic segmentation tasks, Represents the input RGB image, This represents the semantic information annotation of the input image pixel by pixel.

[0074] In this embodiment, the method of the present invention will be described in detail, and the specific steps are as follows:

[0075] S1. Based on the form of the computer vision task, determine the model structure and search network to form a computational graph with searchable structure. Calculation graph The architecture parameters are determined by the structural parameters and the network parameters;

[0076] like Figure 1 As shown, before the search begins, based on the application scenario and task requirements of this embodiment, the model structure and search space are first defined to form a computational graph with searchable structures. A computation graph is an abstraction of structure and parameters that defines a unique computational logic. This logic is represented graphically and called a computation graph. The computation graph is a directed acyclic graph (DAG) that contains the computational flow, computational methods, and interdependencies between all operators in the basic network unit.

[0077] In differentiable neural architecture search, the search unit is a directed acyclic graph composed of nodes and edges, i.e., a computational graph. Nodes represent a feature map, and edges represent a searchable path. Candidate edge operations include parameterless zeroing, identity, average pooling, and max pooling operations, as well as four convolution operations: 3×3 and 5×5 separable convolutions, and 3×3 and 5×5 dilated convolutions. Each operation can be represented as a combination of multiple concatenated basic modules. Specifically, candidate convolutional operations consist of a non-linear activation function, convolution operations, and batch normalization operations, while separable convolutions are composed of multiple such concatenated layers. The topology of each candidate convolutional operation begins with a non-linear activation function and ends with batch normalization; therefore, these modules can be considered common features of the topologies of multiple candidate operations, allowing for simplification.

[0078] S2, Calculation graph The topology is iteratively optimized to eliminate redundant structures and optimize structural parameters;

[0079] Unoptimized computation graphs contain three main types of redundant structures: common pre-modules, common post-modules, and common dangling modules. Common pre-modules and common post-modules refer to parallel structures in the computation graph where different branches share the same pre-module and post-module, differing only in their intermediate modules. For common pre-modules and common post-modules, a partial module-sharing optimization pattern is designed to merge pre-modules and post-modules without affecting the search of multi-branch structures. Common dangling modules refer to parallel structures in the computation graph where different branches share the same intermediate module, but their pre-modules and post-modules are different. For common dangling modules, a shared dangling module optimization pattern is designed to merge dangling modules while preserving the structural parameter dependencies between different pre- and post-modules. The optimization is achieved by iteratively calling methods for partial module sharing and dangling module sharing. Optimization is performed to obtain the optimized network computation graph. It can greatly simplify the calculation.

[0080] Specifically, it includes:

[0081] (1) Search for parallel structures in the topology to obtain common front modules, common back modules and common floating modules on different branches;

[0082] Redundant structures can be summarized and classified into the following two categories:

[0083] Category 1: Topologies with common front and rear modules, but different suspended modules;

[0084] Category 2: Topologies with a common suspended module but different front and / or rear modules.

[0085] (2) For the common front module and common back module structure in the topology, the optimization mode of partial module sharing is applied to merge all common front modules and all common back modules.

[0086] like Figure 2 As shown, operators with common start operations and common end operations are simplified.

[0087] The optimization mode shared by some modules is as follows:

[0088] When the input and output parts of some candidate operations share common convolutional layers, nonlinear layers, or regularization layers, the operation process is as follows:

[0089]

[0090] Where x is the feature map of the input neural architecture search module in the convolutional neural network, i is the index of the candidate operation, and o s1 (·) represents a common computation module that is decoupled from the candidate operation input part. s2 (·) represents a common computation module that is decoupled from the candidate operation output. For the unique operation module of the i-th candidate operation, o i (·) represents other candidate operations that do not share common features. For the indicator set, This represents the set of candidate operation indices for which there is no shared computation module. This represents the set of candidate operation indices that exist in a common operation module;

[0091] By adjusting the architecture parameter α j The candidate operation weights, calculated using the softmax function, represent the selection probability of each candidate operation, where M is the number of candidate operations and i and j are the indices of the candidate operations. To isolate shared components, the operations of shared modules are treated as a single operation, denoted as o. u (x); then the module is decomposed into two cascaded operation selection branches, denoted as g(x):

[0092]

[0093] in and The candidate computation probabilities are derived from two independent sets of structural parameters; considering the potential prior bias introduced by the two sets of parameters, the new structural parameters are reparameterized using the original structural parameters, letting...

[0094]

[0095] Then g(x) is restated as:

[0096]

[0097] in β j This indicates the probability that a candidate operation is a non-shared operation.

[0098] (3) For the common suspended module structure in the topology, apply the optimization mode of shared suspended modules to merge all common suspended modules.

[0099] like Figure 3As shown, in the actual search space, there are also cases where modules share intermediate suspended parts. To improve efficiency, this invention proposes to rewrite the topology into two cascaded operation selection branches and share architectural parameters.

[0100] Specifically, when the intermediate dangling parts of some candidate operations contain shared convolutional, nonlinear, or regularization layer operation modules, the original topology can be expressed as follows:

[0101]

[0102] in For independent computation modules before and after the suspended module, o s This is a suspended shared computing module. By adjusting the structural parameter α j The candidate operation weights, calculated using the softmax function, represent the selection probability of each candidate operation, where M is the number of candidate operations and i and j are the indices of the candidate operations. The optimized computation process is as follows:

[0103]

[0104] Where h(x) is the intermediate result selected in the first operation, and g(x) is the result selected in the second operation. In the new operation process, o s It only requires one calculation, simplifying the computation.

[0105] S3. Perform convolution operations on the optimized topology; optimize network parameters by reparameterizing the convolution kernel; and decouple network parameters and structural parameters by standardizing the convolution kernel; thus obtaining the optimized computational graph.

[0106] (1) Convolution kernel reparameterization

[0107] Discrete convolution is a linear operation, meaning that for any real numbers β1, β2, β1x*h1 + β2x*h2 = x*(β1h1 + β2h2) holds true. Therefore, after topology optimization, the operation of the convolution operator can be further simplified. Each convolution operation can be transformed into an equivalent standard convolution operation according to specific rules, and their weighted sums and the order of operations can be interchanged. After this interchange, only one convolution operation is needed to obtain the weighted sum of the results of multiple convolution operators. For example... Figure 4 As shown, this section provides a detailed explanation of how to perform different types of convolution kernel transformations.

[0108] In standard convolution, adding equal amounts of zero padding to the convolution kernel and the input feature map does not numerically affect the output. For standard convolution kernels of different shapes, the shape of the merged kernel is first determined by the maximum width and height of all candidate kernels. Then, all candidate kernels are zero-paded into the merged shape, and weighted summation can be performed.

[0109] For dilated convolution, a k×k convolution kernel with a dilation rate of d has a receptive field with a side length of (k-1)d+1. It is equivalent to a standard convolution of ((k-1)d+1)×((k-1)d+1), except that the kernel has non-zero values ​​only at d pixel intervals. First, initialize a compact convolution kernel of shape k×k, where k is the kernel size. Then, fill in zeros between columns and rows to convert it to a dilated convolution kernel, allowing for standard weighted summation operations.

[0110] A depthwise separable convolution consists of a k×k grouped convolution and a 1×1 standard convolution. Let C... I C O The number of input and output feature map channels; grouped convolution has C I Let there be a k×k convolution kernel, denoted as . Standard convolution has C I ·C O A convolution kernel of shape 1×1, denoted as Depthwise separable convolution can be represented as

[0111]

[0112] Where * denotes convolution, x i This represents the component of the i-th channel of the input feature map. y represents the component of the i-th channel of the feature map after grouped convolution operation. j This represents the component of the j-th channel of the output feature map. Considering the 1×1 convolution kernel as a scalar, the above equation can be rewritten as...

[0113]

[0114] in This is equivalent to a standard convolution kernel.

[0115] (2) Convolution kernel standardization

[0116] The NAS method based on reparameterization increases the instability of the search process because the structural parameters are directly multiplied with the learnable convolution kernel parameters when selecting operators, forming a coupled bilinear operation.

[0117] To address this problem, this invention proposes a direct method to eliminate coupling between structural parameter model parameters. Let K... iThe original convolution kernel is used as the basis for standardization, resulting in...

[0118]

[0119] Here, mean and std represent the calculation of mean and standard deviation; the normalized kernel is insensitive to the scaling and translation of the original kernel, thus effectively alleviating the coupling problem between the kernel and the architecture parameters.

[0120] S4. Optimized computation graph Input an image classification dataset or an image semantic segmentation dataset, run the gradient descent algorithm to determine the optimal architecture parameters, and complete the search for the optimal architecture of the classification or segmentation model; train the model structure corresponding to the optimal architecture parameters to complete the computer vision classification or semantic segmentation task.

[0121] In an embodiment using a convolutional neural network (CNN) as a classification model for an image classification dataset or a segmentation model for an image semantic segmentation dataset, the following steps are taken: First, the gradient descent algorithm is used to optimize the architecture parameters of the CNN, such as the kernel size, the number of convolutional layers, and the number of pooling layers. Then, the optimized parameters are used to train the CNN model. Finally, the trained model is used to complete the image classification or semantic segmentation task.

[0122] The gradient descent method optimizes both network weights and branch weights, and ultimately selects the optimal network architecture based on the highest branch weight.

[0123] The gradient descent algorithm in step S4 iteratively optimizes the architecture parameters of the computation graph G' to achieve optimal model performance in computer vision classification tasks. By continuously adjusting the architecture parameters and updating the model weights during training, a more adaptive and higher-performing neural network structure can be obtained, thereby improving the classification accuracy of computer vision tasks.

[0124] This invention proposes an efficient differentiable neural architecture search method based on topology optimization. Starting from practical problems, it solves the problems of slow search speed and high computational cost for differentiable neural architectures. Compared with current neural architecture search methods, this invention has the following advantages:

[0125] 1) Reduced computational resource requirements: During the search process, this invention proposes to merge and optimize convolution operators in the search space. By utilizing the weighting of linear operators and structural parameters, as well as the property of interchangeable operation order, convolution operator search without additional computation is achieved, thereby significantly reducing the huge computational resources required to simultaneously compute all possible branches.

[0126] 2) Improved search stability: To address the instability caused by the coupling between structural and model parameters during the search process, this invention introduces convolution kernel reparameterization and normalization algorithms. These algorithms decouple model parameters from structural parameters, achieving stable and fast model architecture search.

[0127] 3) Generalized and Efficient Search Method: Existing search acceleration methods are often specially designed for specific models or search spaces, making them difficult to generalize. This invention proposes a model-independent plug-and-play module that, through an iterative topology optimization algorithm, can be applied to various search spaces and network structures, supporting the neural architecture search needs of various fields.

[0128] Based on the above advantages, the improved method proposed in this invention can significantly improve the efficiency and stability of neural architecture search, providing strong support for the design of various complex neural network models.

[0129] For example, the effectiveness of the proposed search method is verified on an image classification task under the common NASBench201 benchmark. Since the proposed method is an improvement on a general topology, it can be widely applied to various existing neural architecture search methods and achieve efficiency improvements. The experiment uses the NASBench201 standard search space to search the network architecture on the CIFAT-10 dataset, and then trains and verifies the accuracy of the obtained network architecture on the ImageNet-16 (IN-16) dataset. The Top-1 accuracy of image classification is used as the accuracy evaluation metric, while the GPU memory and computation time resources consumed in the search are used as a secondary evaluation metric, and scatter plots are generated. Experimental results are as follows: Figure 5 As shown, the efficient optimization strategy proposed in this invention can increase search efficiency in various neural architecture search methods, and can improve search accuracy in most algorithms. Figure 5 The data points at the origin of the middle arrow represent the original efficiency and accuracy of the search method, while the data points at the end of the arrow represent the efficiency and accuracy achieved by the search method after optimization by the method proposed in this invention. The term represents the search strategy obtained by combining the β regularization method proposed in β-DARTS with PC-DARTS; β-DARTS* represents the result obtained by using the special hyperparameter settings proposed in β-DARTS during the search process, while β-DARTS represents the result obtained by using the standard hyperparameter settings of NASBench201.

[0130] Due to limitations in computational resources, most neural architecture search methods were originally limited to simple tasks such as image classification and text classification. However, optimized and efficient neural architecture search methods can perform searches on more complex tasks at the same cost. This provides a theoretical and technical foundation for lightweight networks and the application of neural architecture search in complex visual tasks.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0132] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A highly efficient differentiable neural architecture search method based on topology optimization, characterized in that, Includes the following steps: S1. Based on the form of the computer vision task, determine the model structure and search network to form a computational graph with searchable structure. The computation graph The architecture parameters are determined by the structural parameters and the network parameters; S2, Regarding the computation graph The topology is iteratively optimized to eliminate redundant structures and optimize the structural parameters. S3. Perform convolution operation on the optimized topology with structural parameters; The network parameters are optimized by reparameterizing the convolution kernels; and the network parameters and structural parameters are decoupled by normalizing the convolution kernels, resulting in an optimized computational graph. ; S4. The optimized computation graph Input an image classification dataset or an image semantic segmentation dataset, run the gradient descent algorithm to determine the optimal architecture parameters, and complete the search for the optimal architecture of the classification or segmentation model; train the model structure corresponding to the optimal architecture parameters to complete the computer vision classification or semantic segmentation task. Among them, the computation graph The topology is iteratively optimized to eliminate redundant structures; including: Search for parallel structures in the topology to obtain common front modules, common back modules, and common floating modules on different branches; For the common front-end module and common rear-end module structure in the topology, an optimization mode of partial module sharing is applied to merge all the common front-end modules and all the common rear-end modules. For the common suspended module structure in the topology, the optimization mode of shared suspended modules is applied to merge all the common suspended modules.

2. The efficient differentiable neural architecture search method based on topology optimization according to claim 1, characterized in that, The optimization modes shared by some of the modules include: When the input and output parts of some candidate operations share common convolutional layers, nonlinear layers, or regularization layers, the operation process is as follows: ； Where x is the feature map of the input neural architecture search module in the convolutional neural network, and i is the index of the candidate operation. This is a common computation module that decouples the candidate operation input part from the candidate operation itself. This is a common computation module that is decoupled from the candidate computation output. This is a unique computation module for the i-th candidate operation. For other candidate operations that do not share common features, This represents the set of candidate operation indices for which there is no shared computation module. This represents the set of candidate operation indices that exist in a common operation module; By adjusting the structural parameters The candidate operation weights, calculated using the softmax function, represent the probability of selecting each candidate operation, where M is the number of candidate operations, and i and j are the indices of the candidate operations. To isolate shared components, the operations of shared modules are treated as a single operation, denoted as... The module is then decomposed into two cascaded operation selection branches, denoted as... : ； ； in and The candidate computation probabilities are derived from two independent sets of structural parameters; considering the potential prior bias introduced by the two sets of parameters, the new structural parameters are reparameterized using the original structural parameters, letting... ； but Rephrased as: ； in, This indicates the probability that a candidate operation is a non-shared operation.

3. The efficient differentiable neural architecture search method based on topology optimization according to claim 1, characterized in that, The optimization modes shared by the suspended modules include: When the intermediate dangling parts of some candidate operations contain shared convolutional, nonlinear, or regularization layer operation modules, the original topological structure is expressed as follows: ； in , These are independent computation modules before and after the suspended module. This is a suspended shared computing module. By adjusting the structural parameters The candidate operation weights, calculated using the softmax function, represent the selection probability of each candidate operation, where M is the number of candidate operations and i and j are the indices of the candidate operations. The optimized computation process is as follows: ； ； in The intermediate result selected for the first calculation. The result selected for the second operation.

4. The efficient differentiable neural architecture search method based on topology optimization according to claim 1, characterized in that, The convolution kernel reparameterization includes: converting the convolution operation into an equivalent standard convolution operation, and swapping the order of operations for the weighted sum and convolution operation; wherein, the conversion process for different types of convolution kernels is as follows: Standard convolution: First, the shape of the merged convolution kernel is determined by the maximum width and height of all candidate convolution kernels. Then, all candidate convolution kernels are zero-padded into the merged shape, and weighted summation can be performed. Dilated convolution: First, initialize a shape as... A compact convolution kernel, where k is the size of the convolution kernel, is then filled with zeros between columns and rows to convert it into a dilated convolution kernel, which can then be used for standard weighted summation operations. Depthwise separable convolution: consisting of a Grouped convolution and a The standard convolutional structure; let The number of input and output feature map channels; grouped convolution has The shape is The convolution kernel, denoted as Standard convolution has The shape is The convolution kernel, denoted as Then depthwise separable convolution is represented as: ； ； Where * denotes convolution. This represents the component of the i-th channel of the input feature map. This represents the component of the i-th channel of the feature map after grouped convolution. This represents the component of the j-th channel of the output feature map; consider Since the convolution kernel is a scalar, the above equation can be rewritten as: ； in This is the equivalent standard convolution kernel.

5. The efficient differentiable neural architecture search method based on topology optimization according to claim 1, characterized in that, The kernel standardization includes: remember The original convolutional kernel is used as the basis for standardization, which yields the standardized convolutional kernel. ; ； Here, mean and std represent the calculation of mean and standard deviation.

Images