Methods, apparatus, devices and storage media for searching the channel number of neural network architectures

Abstract

This application relates to the field of data processing technology, specifically to a method, apparatus, device, and storage medium for searching the channel number of a neural network architecture, aiming to improve the accuracy of searching the channel number of a neural network architecture based on a supernetwork. The method includes: constructing a supernetwork using multiple subnetworks through a depth-sharing strategy and a dimension-sharing strategy; sampling multiple network architectures from the supernetwork according to the sandwich method and a fair sampling strategy; training the multiple network architectures using a dynamic knowledge distillation method to obtain trained network architectures; inputting the trained network architectures and training data into the supernetwork to optimize the parameters of the supernetwork; repeating the above sampling, training, and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, obtaining a supernetwork with optimal parameters.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, device, and storage medium for searching the number of channels in a neural network architecture. Background Technology

[0002] Neural network channel number search (NNDS) is the process of finding the most suitable network architecture for a target task from multiple network architectures. It plays a crucial role in model compression and multi-objective neural network architecture search. Among these methods, NNDS based on hypernetworks are widely used in neural network architecture search systems due to their superior search efficiency. Existing NNDS methods primarily involve constructing a hypernetwork, using a differentiable search algorithm, and combining the outputs of different channel number choices through a weighted average. This allows for unified parameter updates across different channel number choices. Then, dynamic knowledge distillation is employed, first training the largest network and then using that network as a teacher to guide the hypernetwork's updates.

[0003] Existing methods for improving the efficiency of neural network channel number search do not actively address the issue of hypernetwork consistency, failing to accurately rank models within the hypernetwork and exhibiting low correlation with the actual model ranking. Furthermore, existing methods do not consider the impact of sharing strategies for different parameters within the hypernetwork and evaluation criteria on the search results. Additionally, the sharing strategies employed in existing methods are complex, requiring meticulous parameter tuning, resulting in poor performance in neural network channel number search. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for searching the number of channels in a neural network architecture, aiming to improve the accuracy of searching the number of channels in a neural network architecture based on hypernetworks.

[0005] A first aspect of this application provides a method for searching the number of channels in a neural network architecture, the method comprising:

[0006] A supernetwork is constructed by employing a depth-sharing strategy and a dimension-sharing strategy, using multiple subnetworks.

[0007] Based on the sandwich principle and fair sampling strategy, multiple network architectures are sampled from the supernetwork;

[0008] The multiple network architectures are trained using a dynamic knowledge distillation method to obtain the trained multiple network architectures.

[0009] The trained network architectures and training data are input into the supernetwork to optimize the parameters of the supernetwork.

[0010] Repeat the above sampling, training, and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, thus obtaining a supernetwork with optimal parameters.

[0011] Optionally, the method further includes:

[0012] Extract the target subnetwork from the trained hypernetwork;

[0013] The target subnetwork is evaluated using test data to obtain subnetwork evaluation results.

[0014] Optionally, while evaluating the performance of the target sub-network using test data to obtain the sub-network evaluation results, the method further includes:

[0015] The target subnetwork is modified using the test data to obtain the modified subnetwork.

[0016] Optionally, the method further includes:

[0017] Input the target task dataset into the hypernetwork;

[0018] The hypernetwork performs a channel number search based on the target task data to obtain the optimal network architecture corresponding to the target task data.

[0019] The optimal network architecture is trained using the target task data to obtain the optimal network architecture with parameters adjusted to the best.

[0020] The target task data is input into the optimal network architecture, the task corresponding to the target task data is executed, and the target task result is output.

[0021] Optionally, a supernetwork can be constructed using a depth-sharing strategy and a dimension-sharing strategy, through multiple subnetworks, including:

[0022] Place the multiple sub-networks in the same network space;

[0023] The parameters of the convolution operation in the same layer of the multiple sub-networks are set as shared parameters according to the output dimension;

[0024] Parameters at the same location in the multiple sub-networks are set as shared parameters.

[0025] Optionally, based on the sandwich principle and a fair sampling strategy, multiple network architectures are sampled from the supernetwork, including:

[0026] Based on the sandwich principle, the largest and smallest network architectures are selected from the hypernetwork.

[0027] The remaining network architectures are selected from the supernetwork based on a fair sampling strategy.

[0028] Optionally, the multiple network architectures are trained using a dynamic knowledge distillation method to obtain trained multiple network architectures, including:

[0029] The training data is input into the largest network architecture, and the largest network architecture is trained to obtain the trained largest network architecture.

[0030] The largest trained network architecture is used as the guiding architecture to guide the training of the remaining architectures, resulting in multiple trained network architectures.

[0031] A second aspect of this application provides a neural network architecture channel number search device, the device comprising:

[0032] The hypernetwork building module is used to construct a hypernetwork from multiple subnetworks using depth-sharing and dimension-sharing strategies.

[0033] An architecture sampling module is used to sample multiple network architectures from the hypernetwork based on the sandwich principle and a fair sampling strategy.

[0034] An architecture training module is used to train the multiple network architectures using a dynamic knowledge distillation method to obtain the trained multiple network architectures.

[0035] The supernetwork parameter optimization module is used to input the trained multiple network architectures and training data into the supernetwork and optimize the parameters of the supernetwork.

[0036] The iterative optimization module is used to repeatedly execute the above sampling, training and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, and obtain the supernetwork with the parameters adjusted to the optimal.

[0037] Optionally, the device further includes:

[0038] The subnetwork extraction module is used to extract the target subnetwork from the trained supernetwork;

[0039] The subnetwork evaluation module is used to evaluate the effectiveness of the target subnetwork using test data and obtain the subnetwork evaluation results.

[0040] Optionally, the device further includes:

[0041] The sub-network parameter correction module is used to correct the parameters of the target sub-network using the test data to obtain the parameter-corrected sub-network.

[0042] Optionally, the device further includes:

[0043] The data input module is used to input the target task data into the hypernetwork;

[0044] The channel number search module is used by the hypernetwork to perform a channel number search based on the target task data to obtain the optimal network architecture corresponding to the target task data.

[0045] The optimal network architecture training module is used to train the optimal network architecture using the target task data to obtain the optimal network architecture with parameters adjusted to the optimal level.

[0046] The task execution module is used to input the target task data into the optimal network architecture, execute the task corresponding to the target task data, and output the target task result.

[0047] Optionally, the hypernetwork building module includes:

[0048] The search space construction submodule is used to place the multiple sub-networks in the same network space;

[0049] The first parameter setting submodule is used to set the parameters of the convolution operation in the same layer of the multiple sub-networks as shared parameters according to the output dimension;

[0050] The second parameter setting submodule is used to set parameters at the same position in the multiple sub-networks as shared parameters.

[0051] Optionally, the architecture sampling module includes:

[0052] The first architecture sampling submodule is used to select the largest and smallest network architectures from the supernetwork according to the sandwich rule.

[0053] The second architecture sampling submodule is used to select the remaining network architecture from the supernetwork according to the fair sampling strategy.

[0054] Optionally, the architecture training module includes:

[0055] The first architecture training submodule is used to input the training data into the largest network architecture, train the largest network architecture, and obtain the trained largest network architecture.

[0056] The second architecture training submodule is used to use the largest trained network architecture as the guiding architecture to guide the training of the remaining architectures, thereby obtaining multiple trained network architectures.

[0057] A third aspect of this application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described in the first aspect of this application.

[0058] A fourth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in the first aspect of this application.

[0059] This application employs a neural network architecture channel number search method, utilizing depth-sharing and dimension-sharing strategies to construct a supernetwork through multiple subnetworks. Based on the sandwich principle and fair sampling, multiple network architectures are sampled from the supernetwork. These multiple neural network architectures and training data are input into the supernetwork, and dynamic knowledge distillation is used to optimize the supernetwork's parameters. The sampling, training, and parameter optimization steps are repeated until the supernetwork's parameters converge, resulting in a supernetwork with optimal parameters. This application utilizes depth-sharing and dimension-sharing strategies to construct a supernetwork and uses dynamic knowledge distillation to optimize its parameters. Most parameters in each subnetwork of the supernetwork are shared, simplifying training and ensuring the correlation between the selection of various channel number parameters. This guarantees the consistency of the supernetwork, meaning the ranking of models within the supernetwork matches the actual ranking of models, thus increasing the accuracy of neural network architecture channel number search. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of a neural network architecture channel number search method proposed in an embodiment of this application;

[0062] Figure 2 This is a schematic diagram of the training and testing framework for neural network channel number search proposed in an embodiment of this application;

[0063] Figure 3 This is a schematic diagram of a neural network architecture channel number search device proposed in an embodiment of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] When a neural network is required to perform a specific task, it is generally necessary to select an appropriate architecture to process the data and then perform the corresponding task.

[0066] refer to Figure 1 , Figure 1 This is a flowchart of a neural network architecture channel number search method proposed in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0067] S11: Employs a depth-sharing strategy and a dimension-sharing strategy to construct a supernetwork through multiple subnetworks.

[0068] In this embodiment, the supernetwork is a network composed of multiple neural network architectures in a common space, constructed according to certain rules. It is a parameter-sharing system, in which each neural network is a subnetwork of the supernetwork.

[0069] In this embodiment, the depth-sharing strategy uses the same parameters for all convolutional operations at the same layer in each sub-network of the supernetwork. The dimension-sharing strategy, on the other hand, sets the parameters of batch normalization operations with the same dimension at the same location across different sub-networks to be shared with the parameters of fully connected operations; the dimension is the number of channels.

[0070] For example, when performing image classification tasks, the ResNet20 image classification network is divided into three types of operations: convolution operations, batch normalization operations, and fully connected operations. For convolution operations, a depth-sharing strategy is adopted, which means that all convolution operations in the same layer of the network use the same parameters. For batch normalization operations and fully connected operations, a dimension-sharing strategy is adopted, which sets the parameters of the same dimension batch normalization operations at the same position in different sub-networks to shared parameters.

[0071] In this embodiment, parameter sharing between convolution and normalization operations and fully connected operations between different sub-networks is achieved through depth sharing and dimension sharing strategies. This realizes depth weight sharing, reduces the number of parameters, and increases the personalization between different dimension selections.

[0072] S12: Based on the sandwich principle and fair sampling strategy, multiple network architectures are sampled from the supernetwork.

[0073] In this embodiment, after the supernetwork is constructed, its parameters need to be optimized and adjusted. Targeted optimizations and adjustments can be made to the supernetwork for different objective tasks. Because the supernetwork contains a massive number of network architectures, training each architecture individually is difficult and time-consuming. Therefore, it is necessary to sample several network architectures from the supernetwork, train these architectures first, and then use these architectures to update the remaining architectures in the entire supernetwork.

[0074] In this embodiment, the sandwich method is first used to sample the largest and smallest network architectures from the supernetwork, and then a fair sampling strategy is used to select the remaining architectures.

[0075] For example, when you want to select a network architecture suitable for an image recognition task using a hypernetwork, during the hypernetwork training phase, you first select the largest and smallest image recognition network architectures from the hypernetwork, and then select a certain number of architectures from the remaining architectures. You can then use the image recognition task dataset for subsequent training.

[0076] S13: The multiple network architectures are trained using a dynamic knowledge distillation method to obtain the trained multiple network architectures.

[0077] In this embodiment, the dynamic knowledge distillation method involves first training the largest network architecture among multiple network architectures, and then using the trained largest network architecture to guide the optimization of the remaining architectures. Specific steps include:

[0078] S13-1: Input the training data into the largest network architecture, train the largest network architecture, and obtain the trained largest network architecture.

[0079] S13-2: Use the largest trained network architecture as the guiding architecture to guide the training of the remaining architectures, resulting in multiple trained network architectures.

[0080] In this embodiment, the training data uses random batch sampling. After obtaining multiple network architectures from the supernetwork, the largest network architecture can be trained first using the training data. After training the largest network architecture once, it is used as the guiding architecture to guide the training of the remaining architectures.

[0081] In this embodiment, the training data first needs to be correctly labeled. Then, the labeled training data is input into the largest network architecture among the multiple architectures obtained by sampling to train the architecture. After one round of training of the architecture, the largest architecture is used as the guiding architecture to guide the other architectures to conduct one round of training, so that multiple network architectures can be obtained after training.

[0082] For example, if a supernetwork is trained for an image recognition task, some images are first randomly sampled as training data. These images are preprocessed to label the objects to be recognized. The labeled image data is then input into the largest image recognition network architecture. Cross-entropy is used to train the largest image recognition network architecture. Then, the trained largest image recognition network architecture is used to train the remaining image recognition network architectures, resulting in multiple trained image recognition network architectures.

[0083] S14: Input the multiple network architectures and training data into the supernetwork and optimize the parameters of the supernetwork.

[0084] In this embodiment, the multiple network architectures and training data obtained from the previous sampling step are input into the supernetwork. These multiple network architectures have been pre-trained using the training data and can be used to guide the training of the remaining architectures in the supernetwork. This training data is used for the training of the remaining architectures in the supernetwork.

[0085] In this embodiment, since the convolution parameters of each layer in each subnetwork of the supernetwork are shared, and the parameters at the same position in different subnetworks are also shared, multiple pre-sampled network architectures can be directly input into the supernetwork after one round of training. The remaining network structures in the supernetwork can share parameters with the input network architecture, so a supernetwork with high consistency can be trained quickly.

[0086] For example, in an image recognition task, image data and multiple collected network architectures are input into a supernetwork to train the supernetwork, resulting in a trained supernetwork. The trained supernetwork can then be used to search for neural network architecture channels for image recognition tasks.

[0087] S15. Repeat the above sampling, training and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, and obtain the supernetwork with the parameters adjusted to the optimal.

[0088] In this embodiment, the parameters of the supernetwork have completed one round of parameter update through the above steps. After completing one round of parameter update, the sampling, training and parameter optimization steps of S12-S14 are executed cyclically to update the parameters of the supernetwork iteratively until the parameters of the entire supernetwork are updated to convergence, thus obtaining the supernetwork with the parameters adjusted to the optimal.

[0089] In this embodiment, a method for constructing and training a supernetwork for neural network channel number search is proposed. The supernetwork is constructed through a depth-sharing strategy and a dimension-sharing strategy. The network architecture in the supernetwork is sampled using the sandwich rule and the fair sampling principle. The parameters of the remaining architecture in the supernetwork are then updated using the sampled architecture, thereby improving the consistency of neural network channel number search.

[0090] In another embodiment of this application, the method further includes:

[0091] S21: Extract the target subnetwork from the trained supernetwork.

[0092] S22: Use test data to evaluate the effectiveness of the target sub-network and obtain the sub-network evaluation results.

[0093] In this embodiment, the trained supernetwork needs to be tested to evaluate the performance of each subnetwork. First, the target subnetwork is extracted from the trained supernetwork, and then the test data is input into the target subnetwork to evaluate its performance.

[0094] For example, in a supernetwork trained for an image recognition task, one of the image recognition networks can be extracted as a target subnetwork. Images from the test set are input into the target subnetwork, which then recognizes the images and outputs the recognition results. The performance of the subnetwork is scored based on the accuracy of the recognition results. Negative loss can be used to evaluate the network's performance, which can improve the accuracy of the supernetwork in ranking models.

[0095] In another embodiment of this application, while evaluating the performance of the target sub-network using test data to obtain the sub-network evaluation result, the method further includes:

[0096] S31: Use the test data to correct the parameters of the target sub-network to obtain the parameter-corrected sub-network.

[0097] In this embodiment, due to the architecture of the sub-network itself, the dynamic mean and dynamic variance values ​​in the batch normalization layer may become inappropriate when training the super-network. Therefore, while evaluating the sub-network using test set data, the dynamic mean and dynamic variance in the batch normalization layer are corrected by using the representation mean and variance of the test data itself.

[0098] In this embodiment, while evaluating the subnetwork, inappropriate parameters in the subnetwork are corrected by using the parameters of the current batch of data in the test set, which accelerates the evaluation process of the subnetwork while ensuring the recognition effect of the subnetwork.

[0099] In another embodiment of this application, the method further includes:

[0100] S41: Input the target task data into the hypernetwork.

[0101] In this embodiment, once the training and testing of the supernetwork are completed, the supernetwork can be used to search for the number of neural network architecture channels for the target task.

[0102] In this embodiment, the target task data is the task data that needs to be identified. For example, when the task is an image recognition task, the specific target task is to identify people in the image. Then the target task data is the image containing the task. The dataset containing the target task image can be input into the hypernet.

[0103] S42: The supernetwork performs a channel number search based on the target task data to obtain the optimal network architecture corresponding to the target task data.

[0104] In this embodiment, the hypernetwork searches for the number of channels in the neural network architecture based on the target task data. Different subnetworks in the hypernetwork perform differently on the target task dataset. Based on the performance of different subnetworks on the target task dataset, the number of channels in each layer of the neural network architecture is selected, thereby determining the neural network architecture for executing the target task.

[0105] For example, after inputting the image detection dataset into the hypernetwork, the hypernetwork selects the number of channels in each layer of the image recognition network based on the image extraction and recognition performance of each subnetwork, determines the structure of the image recognition network, and obtains the optimal network architecture for performing the image recognition task.

[0106] S43: Use the target task data to train the optimal network architecture to obtain the optimal network architecture with parameters adjusted to the best.

[0107] In this embodiment, after obtaining the optimal network architecture corresponding to the target task data, the target task dataset is re-input into the optimal network architecture, and the optimal network architecture is trained using the data in the target task dataset. When the parameters of the optimal network architecture are adjusted to the optimal, the optimal network architecture with the parameters adjusted to the optimal is output.

[0108] For example, in an image recognition task, the optimal network architecture of the image recognition network is determined. The image recognition task dataset is then re-input into the obtained optimal network architecture to train the network, thereby obtaining the optimal network architecture with the parameters adjusted to the best.

[0109] S44: Input the target task data into the optimal network architecture, execute the task corresponding to the target task data, and output the target task result.

[0110] In this embodiment, after the neural network architecture for performing the target task is determined by the hypernetwork and trained using the target task dataset, the hypernetwork outputs the optimal network architecture with the parameters adjusted to the best. Then, the target task data is input into the optimal network architecture, which can then perform the target task and obtain the processing result of the target task.

[0111] For example, in an image recognition task, the target task data is an image. The image is input into the image recognition network output by the hypernetic network, which performs the image recognition task and outputs the recognition result.

[0112] In this embodiment, after the supernetwork is trained, it can be used to perform tasks. When data is input into the supernetwork, the supernetwork can perform a neural network channel number search based on the data, select the network architecture most suitable for performing the task corresponding to the data, and then use the architecture to process the data input into the supernetwork to obtain the processing result. There is no need to manually verify multiple networks one by one. The network architecture most suitable for performing the task can be selected automatically, which increases the execution efficiency of the task.

[0113] In another embodiment of this application, a hypernetwork is constructed through multiple subnetworks using a depth-sharing strategy and a dimension-sharing strategy, including:

[0114] S51: Place the multiple sub-networks in the same network space.

[0115] In this embodiment, the network space is a common space that can accommodate multiple network architectures at the same time. Multiple network architectures are placed in this common space, and each network architecture belongs to a sub-network of that network architecture.

[0116] For example, multiple CNN networks or RNN networks with different architectures can be placed as subnetworks in the same network space.

[0117] S52: Set the parameters of the convolution operation in the same layer of the multiple sub-networks to shared parameters according to the output dimension.

[0118] In this embodiment, a depth-sharing strategy is adopted for convolutional operations in multiple sub-networks. All convolutional operations in the same layer are summarized according to the output dimension, and parameters with the same output dimension are set as shared parameters. Convolutional operations with smaller dimensions will be regarded as part of convolutional operations with larger dimensions, and convolutional operations with smaller dimensions can extract parameters of the corresponding dimension from convolutional operations with larger dimensions.

[0119] For example, in a sub-network with two layers, the first layer can have 2 or 4 channels, and the second layer can also have 2 or 4 channels. There are four possible combinations of channels: 2×2 (2 channels in both layers), 2×4 (2 channels in both layers), 4×2 (4 channels in both layers), and 4×4 (4 channels in both layers). If the network channel count is 2x2, the parameters used in the convolution operation between these two layers are a subset of the parameters used in the convolution operation between two layers with 2x4 channels. It's simply a matter of extracting the first two dimensions from the second dimension of the 2x4 convolution operation. Similarly, if the network channel count is 4x2, the parameters used in the convolution operation between these two layers are a subset of the parameters used in the convolution operation between two layers with 4x4 channels. Here, the dimension is equivalent to the number of channels.

[0120] In this embodiment, by using a depth-sharing strategy, all convolutional operations in the same layer use the same parameters, which greatly reduces the number of parameters required to optimize the network and lowers the difficulty of network optimization.

[0121] S53: Set the parameters at the same position in the multiple sub-networks as shared parameters.

[0122] In this embodiment, a dimension-sharing strategy is adopted for batch normalization and fully connected operations in multiple sub-networks. This means that the parameters of batch normalization and fully connected operations at the same position and dimension in different sub-networks are set as shared parameters. Specifically, if the number of channels in the same layer is the same in different sub-networks, the parameters between these two layers are shared. If the dimensions at the same position are different between two sub-networks, the parameters are not shared.

[0123] For example, if there are two subnetworks in a supernetwork, and the number of channels in the second layer of both subnetworks is 3, then the dimensions of the second layer in both subnetworks are the same. In this case, the parameters of the batch normalization operation and the fully connected operation of the second layer in both subnetworks are set to shared parameters.

[0124] In this embodiment, by setting the batch normalization operation and full connection operation of the same dimension at the same position between different sub-networks as shared operations, the number of parameters of the entire network is reduced, and the personalization between different dimension selections is increased, thereby improving the consistency of the supernetwork.

[0125] In another embodiment of this application, multiple network architectures are sampled from the hypernetwork based on the sandwich principle and a fair sampling strategy, including:

[0126] S61: Select the largest and smallest network architectures from the supernetwork according to the sandwich principle.

[0127] In this embodiment, the sandwich principle is to first select the largest and smallest network architectures, and then select the middle architecture.

[0128] For example, the largest network architecture in a hypernet is a 64×64×64 network (3 network layers, each with 64 channels), and the smallest network architecture in a hypernet is a 4×4×4 network (3 network layers, each with 4 channels). Therefore, when sampling, these two networks are sampled first.

[0129] S62: Select the remaining network architecture from the supernetwork according to the fair sampling strategy.

[0130] In this embodiment, the fair sampling strategy involves randomly selecting network architectures in the supernetwork, with each architecture having an equal probability of being selected.

[0131] For example, after sampling a 64×64×64 network and a 4×4×4 network, the remaining network architectures are randomly sampled, resulting in a 16×16×16 network (three layers in total, with 16 channels in the first, second, and third layers), a 32×32×16 network (three layers in total, with 32 channels in the first, second, and third layers), and a 32×16×8 network (three layers in total, with 32 channels in the first, second, and third layers).

[0132] In this embodiment, the largest and smallest architectures in the training space are first selected according to the sandwich method. Then, several remaining architectures are randomly selected according to a fair sampling strategy. These architectures are used to guide the training of the remaining architectures in the supernetwork, further improving the stability and consistency of the supernetwork during training.

[0133] In the above embodiments, a training and testing framework for hypernetworks for neural network channel number search is proposed. This framework constructs a hypernetwork using depth-sharing and dimension-sharing strategies, then samples the architecture using a combination of the sandwich method and a fair sampling strategy. Furthermore, dynamic knowledge distillation is used to update the parameters of the sampled architecture, thereby optimizing the hypernetwork parameters and obtaining a trained hypernetwork. The method in the above embodiments greatly improves the consistency of the hypernetwork, that is, the hypernetwork's ranking of the network architecture is consistent with the actual ranking of the network architecture, significantly improving the accuracy of neural network channel number search. Moreover, it is an automated method, eliminating the need for manual setting of complex hyperparameters and improving the efficiency of neural network channel number search.

[0134] In another embodiment of this application, the present application will be further described in conjunction with a schematic diagram of the training and testing framework for neural network channel number search proposed in this application.

[0135] As shown in question 2 Figure 2 This is a schematic diagram of the training and testing framework for neural network channel number search proposed in an embodiment of this application, as shown below. Figure 2 The process begins by constructing a supernetwork. Then, using the sandwich method and fair sampling principle, multiple architectures are sampled from the search space (supernetwork). Training data is obtained by randomly sampling from the training sample space. This training data is used to train these architectures for one round. The trained architectures and data are then input into the supernetwork to optimize its parameters. After this optimization, the sandwich method and fair sampling principle are used again to sample multiple network architectures from the previous training round. These newly sampled architectures are then trained again using the training data. After this training round, the newly sampled network architectures are input into the supernetwork for further optimization. This process is repeated until the supernetwork's parameters reach their optimal state, at which point training is complete. Finally, subnetworks are randomly extracted from the supernetwork for evaluation. In this training round, data is input into the sampled network architecture, a loss function is applied to obtain a loss value, and the network architecture adjusts its parameters accordingly. Figure 2 The dots of the same size on the right side represent training data, while the dots of different sizes on the left side represent multiple sampled network architectures.

[0136] Based on the same inventive concept, one embodiment of this application provides a neural network architecture channel number search device. (Reference) Figure 3 , Figure 3 This is a schematic diagram of a neural network architecture channel number search device 300 according to an embodiment of this application. Figure 3 As shown, the device includes:

[0137] Hypernetwork building module 301 is used to construct a hypernetwork from multiple subnetworks using a depth-sharing strategy and a dimension-sharing strategy.

[0138] The architecture sampling module 302 is used to sample multiple network architectures from the supernetwork according to the sandwich principle and the fair sampling strategy.

[0139] Architecture training module 303 is used to train the multiple network architectures using a dynamic knowledge distillation method to obtain the trained multiple network architectures.

[0140] The supernetwork parameter optimization module 304 is used to input the trained multiple network architectures and training data into the supernetwork, optimize the parameters of the supernetwork, and obtain a trained supernetwork.

[0141] The iterative optimization module 305 is used to repeatedly execute the above sampling, training and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, and obtain the supernetwork with the parameters adjusted to the optimal.

[0142] Optionally, the device further includes:

[0143] The subnetwork extraction module is used to extract the target subnetwork from the trained supernetwork;

[0144] The subnetwork evaluation module is used to evaluate the effectiveness of the target subnetwork using test data and obtain the subnetwork evaluation results.

[0145] Optionally, the device further includes:

[0146] The sub-network parameter correction module is used to correct the parameters of the target sub-network using the test data to obtain the parameter-corrected sub-network.

[0147] Optionally, the device further includes:

[0148] The data input module is used to input the target task data into the hypernetwork;

[0149] The channel number search module is used by the hypernetwork to perform a channel number search based on the target task data to obtain the optimal network architecture corresponding to the target task data.

[0150] The optimal network architecture training module is used to train the optimal network architecture using the target task data to obtain the optimal network architecture with parameters adjusted to the optimal level.

[0151] The task execution module is used to input the target task data into the optimal network architecture, execute the task corresponding to the target task data, and output the target task result.

[0152] Optionally, the hypernetwork building module includes:

[0153] The search space construction submodule is used to place the multiple sub-networks in the same network space;

[0154] The first parameter setting submodule is used to set the parameters of the convolution operation in the same layer of the multiple sub-networks as shared parameters according to the output dimension;

[0155] The second parameter setting submodule is used to set parameters at the same position in the multiple sub-networks as shared parameters.

[0156] Optionally, the architecture sampling module includes:

[0157] The first architecture sampling submodule is used to select the largest and smallest network architectures from the supernetwork according to the sandwich rule.

[0158] The second architecture sampling submodule is used to select the remaining network architecture from the supernetwork according to the fair sampling strategy.

[0159] Optionally, the architecture training module includes:

[0160] The first architecture training submodule is used to input the training data into the largest network architecture, train the largest network architecture, and obtain the trained largest network architecture.

[0161] The second architecture training submodule is used to use the largest trained network architecture as the guiding architecture to guide the training of the remaining architectures, thereby obtaining multiple trained network architectures.

[0162] Based on the same inventive concept, another embodiment of this application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the neural network architecture channel number search method as described in any of the above embodiments of this application.

[0163] Based on the same inventive concept, another embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the neural network architecture channel number search method described in any of the above embodiments of this application.

[0164] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0165] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0166] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0171] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0172] The method, apparatus, device, and storage medium for searching the number of channels in a neural network architecture provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for searching the number of channels in a neural network architecture, characterized in that, The method includes: A supernetwork is constructed using multiple subnetworks by employing a depth-sharing strategy and a dimension-sharing strategy. This includes: when performing image recognition tasks, dividing the ResNet20 image recognition network into three types of operations: convolution, batch normalization, and fully connected operations; employing the depth-sharing strategy for the convolution operations; and employing the dimension-sharing strategy for the batch normalization operations and fully connected operations, setting the parameters of the same dimension batch normalization operations at the same position across different subnetworks to shared parameters. Based on the sandwich principle and fair sampling strategy, multiple network architectures are sampled from the supernetwork, including: selecting the largest and smallest image recognition network architecture from the supernetwork, and then selecting multiple network architectures from the remaining architectures; Using a dynamic knowledge distillation method, multiple network architectures are trained on an image dataset from an image recognition task to obtain trained network architectures. This process includes: randomly sampling multiple image data sets from the image dataset of the image recognition task as training data; preprocessing the training data to label the objects to be recognized; inputting the labeled image data into the largest image recognition network architecture; and training the largest image recognition network architecture using cross-entropy. Finally, the trained largest image recognition network architecture is used to train the remaining image recognition network architectures (excluding the largest one) to obtain trained image recognition network architectures. The trained network architectures and training data are input into the super network, and the parameters of the super network are optimized, including: inputting the image dataset of the image recognition task and the trained network architectures into the super network, training the super network, and obtaining a trained super network. The trained super network is used to search for neural network architecture channels for the image recognition task. Repeat the above sampling, training, and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, thus obtaining a supernetwork with optimal parameters. Extracting the target subnetwork from the trained supernetwork includes: extracting one of the image recognition networks as the target subnetwork from the supernetwork trained for the image recognition task; The target subnetwork is evaluated using test data to obtain subnetwork evaluation results. This includes: inputting image data from the test set into the target subnetwork, the target subnetwork recognizing the image data from the test set and outputting recognition results, and scoring the performance of the target subnetwork based on the accuracy of the recognition results, so as to improve the accuracy of the supernetwork trained for the image recognition task in model ranking. The target subnetwork is modified using the test data to obtain the parameter-corrected subnetwork. The target task data is input into the hypernetwork; the target task data includes at least: image data to be identified; The hypernetwork performs a channel number search based on the target task data to obtain the optimal network architecture corresponding to the target task data, including: the hypernetwork selects the number of channels in each layer of the image recognition network based on the image extraction and recognition effect of each subnetwork, determines the structure of the image recognition network, and obtains the optimal network architecture for performing the image recognition task. Training the optimal network architecture using the target task data to obtain the optimal network architecture with parameters adjusted to the optimal level includes: re-inputting the image dataset of the image recognition task into the optimal network architecture of the image recognition network, training the optimal network architecture of the image recognition network, and then obtaining the optimal network architecture with parameters adjusted to the optimal level. The target task data is input into the optimal network architecture, the task corresponding to the target task data is executed, and the target task result is output, including: inputting an image into the optimal network architecture, executing an image recognition task, and outputting the image recognition result; This involves employing depth-sharing and dimension-sharing strategies to construct a supernetwork through multiple subnetworks, including: Place the multiple sub-networks in the same network space; The parameters of the convolution operation in the same layer of the multiple sub-networks are set as shared parameters according to the output dimension; Parameters at the same location in the multiple sub-networks are set as shared parameters.

2. The method according to claim 1, characterized in that, Based on the sandwich principle and fair sampling strategy, multiple network architectures are sampled from the supernetwork, including: Based on the sandwich principle, the largest and smallest network architectures are selected from the hypernetwork. The remaining network architectures are selected from the supernetwork based on a fair sampling strategy.

3. The method according to claim 1, characterized in that, The multiple network architectures are trained using a dynamic knowledge distillation method, resulting in trained network architectures, including: The training data is input into the largest network architecture, and the largest network architecture is trained to obtain the trained largest network architecture. The largest trained network architecture is used as the guiding architecture to guide the training of the remaining architectures, resulting in multiple trained network architectures.

4. A neural network architecture channel number search device, used to perform the steps in the method according to any one of claims 1-3, characterized in that, The device includes: The supernetwork construction module is used to construct a supernetwork through multiple subnetworks using a depth-sharing strategy and a dimension-sharing strategy. Specifically, when performing image recognition tasks, it divides the ResNet20 image recognition network into three types of operations: convolution operations, batch normalization operations, and fully connected operations. For the convolution operations, the depth-sharing strategy is used; for the batch normalization operations and the fully connected operations, the dimension-sharing strategy is used, setting the parameters of the same dimension batch normalization operations at the same position in different subnetworks to shared parameters. An architecture sampling module is used to sample multiple network architectures from the supernetwork according to the sandwich principle and a fair sampling strategy; specifically, it is used to select the largest and smallest image recognition network architecture from the supernetwork, and then select multiple network architectures from the remaining architectures. The architecture training module is used to train multiple network architectures using a dynamic knowledge distillation method on an image dataset from an image recognition task, resulting in trained network architectures. Specifically, it randomly samples multiple image data from the image dataset of the image recognition task as training data, preprocesses the training data, labels the objects to be recognized, inputs the labeled image data into the largest image recognition network architecture, and trains the largest image recognition network architecture using cross-entropy. Then, it uses the trained largest image recognition network architecture to train the remaining image recognition network architectures (excluding the largest one) among the multiple network architectures, resulting in trained image recognition network architectures. The hypernetwork parameter optimization module is used to input the trained multiple network architectures and training data into the hypernetwork and optimize the parameters of the hypernetwork; specifically, it is used to input the image dataset of the image recognition task and the trained multiple network architectures into the hypernetwork, train the hypernetwork, and obtain a trained hypernetwork, which is used to search for neural network architecture channels for the image recognition task. The iterative optimization module is used to repeatedly execute the above sampling, training and parameter optimization steps to iteratively optimize the parameters of the supernetwork until convergence, and obtain a supernetwork with the parameters adjusted to the optimal. The subnetwork extraction module is used to extract the target subnetwork from the trained supernetwork; specifically, it is used to extract one of the image recognition networks as the target subnetwork from the supernetwork trained for the image recognition task. The subnetwork evaluation module is used to evaluate the performance of the target subnetwork using test data and obtain the subnetwork evaluation result. Specifically, it is used to input image data from the test set into the target subnetwork, the target subnetwork recognizes the image data from the test set, outputs the recognition result, and scores the performance of the target subnetwork based on the accuracy of the recognition result, so as to improve the accuracy of the supernetwork trained for the image recognition task in model ranking. The sub-network parameter correction module is used to correct the parameters of the target sub-network using the test data to obtain the parameter-corrected sub-network. The data input module is used to input target task data into the hypernetwork; the target task data includes at least: image data to be identified; The channel number search module is used by the hypernetwork to search for the channel number based on the target task data to obtain the optimal network architecture corresponding to the target task data; specifically, the hypernetwork selects the channel number of each layer of the image recognition network based on the image extraction and recognition effect of each subnetwork, determines the structure of the image recognition network, and obtains the optimal network architecture for performing the image recognition task. The optimal network architecture training module is used to train the optimal network architecture using the target task data to obtain the optimal network architecture with parameters adjusted to the optimal level. Specifically, it is used to re-input the image dataset of the image recognition task into the optimal network architecture of the image recognition network, train the optimal network architecture of the image recognition network, and then obtain the optimal network architecture with parameters adjusted to the optimal level. The task execution module is used to input the target task data into the optimal network architecture, execute the task corresponding to the target task data, and output the target task result; specifically, it is used to input the image into the optimal network architecture, execute the image recognition task, and output the image recognition result. The hypernetwork construction module includes: The search space construction submodule is used to place the multiple sub-networks in the same network space; The first parameter setting submodule is used to set the parameters of the convolution operation in the same layer of the multiple sub-networks as shared parameters according to the output dimension; The second parameter setting submodule is used to set parameters at the same position in the multiple sub-networks as shared parameters.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 3.

Images