Neural Network Architecture Search Method, Apparatus, Computer Device, and Storage Medium

Abstract

The present application relates to a neural network structure search method, apparatus, computer equipment, and storage medium. The method includes: obtaining a feature vector of a neural network structure in a neural network search space; clustering the obtained neural network structure based on the feature vector to obtain the category to which each neural network structure belongs; selecting a neural network structure to be searched from the neural network structures of each category; constructing a target neural network search space based on the selected neural network structure to be searched; after training the neural network structure to be searched in the target neural network search space based on training data of a media task, searching for a target neural network structure for processing the media task in the trained target neural network search space in response to a network search request of the media task. The use of this method can improve the efficiency of network structure search.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a neural network structure search method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the design of neural network structures is shifting from manual design to automated machine design. Neural Architecture Search (NAS) can help developers automatically search for the optimal neural network structure.

[0003] In traditional technology, if the neural network search space is large, searching in the neural network search space often consumes a lot of computing resources and training time, and the final output search results are not accurate enough, resulting in low search efficiency of the neural network structure. Summary of the Invention

[0004] Based on this, it is necessary to provide a neural network structure search method, device, computer equipment and storage medium that can improve search efficiency in response to the above technical problems.

[0005] A neural network structure search method, the method comprising:

[0006] Obtain the feature vector of the neural network structure in the neural network search space;

[0007] Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs;

[0008] Selecting a neural network structure to be searched from the neural network structures of each category;

[0009] Constructing a target neural network search space based on the selected neural network structure to be searched;

[0010] After the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

[0011] A neural network structure search device, comprising:

[0012] A vector acquisition module is used to obtain the characteristic vector of the neural network structure in the neural network search space;

[0013] A clustering module, configured to cluster the obtained neural network structures based on the feature vectors to obtain a category to which each neural network structure belongs;

[0014] A structure selection module, configured to select a neural network structure to be searched from the neural network structures of each category;

[0015] A search space construction module, configured to construct a target neural network search space based on the selected neural network structure to be searched;

[0016] A structure search module is used to search for a target neural network structure for processing the media task in the trained target neural network search space in response to a network search request of the media task after the training of the neural network structure to be searched in the target neural network search space is completed based on the training data of the media task.

[0017] In one embodiment, the vector acquisition module is further configured to:

[0018] Obtain the receptive field and parameter quantity of each network layer in the neural network structure in the neural network space;

[0019] Determine a feature vector of the neural network structure based on the receptive field and parameter quantity of the neural network structure.

[0020] In one embodiment, the vector acquisition module is further configured to:

[0021] Determining a first eigenvector according to the receptive field of each network layer in the neural network structure;

[0022] Determining a second eigenvector based on a parameter value of each network layer in the neural network structure;

[0023] Based on the first eigenvector and the second eigenvector, a eigenvector of the neural network structure is constructed.

[0024] In one embodiment, the clustering module is further configured to:

[0025] Sampling from the neural network structure to obtain at least two neural network structure samples;

[0026] Taking the neural network structure sample as a cluster center, and determining a first distance between other neural network structures and the cluster center based on the feature vector;

[0027] Determine the category to which the other neural network structures belong based on the first distance to obtain an initial category;

[0028] The center of mass is determined in the neural network structure of the initial category, and a second distance between the neural network structure outside the center of mass and the center of mass is determined. Based on the second distance, the category to which the neural network structure outside the center of mass belongs is determined, and the category to which each of the neural network structures belongs is obtained.

[0029] In one embodiment, the structure selection module is further used to:

[0030] The neural network structures in each category are respectively formed into a supernet for training;

[0031] After the training is completed, the neural network structure in each category is tested based on the test set to obtain a first test result;

[0032] Among the neural network structures in each of the categories, a neural network structure to be searched is selected based on the first test result.

[0033] In one embodiment, the apparatus further comprises:

[0034] a model training module, configured to form a target supernet from the neural network structures to be searched in the target neural network space, and to train the target supernet based on the training data of the media task;

[0035] The structure search module is further used to:

[0036] In response to the network search request of the media task, testing the neural network structure to be searched in the target neural network search space according to the test data of the media task to obtain a second test result;

[0037] From the trained target neural network search space, search for a target neural network structure for processing the media task based on the second test result.

[0038] In one embodiment, the model training module is used to:

[0039] Each time, a neural network structure to be searched is randomly selected from the target supernet, and the selected neural network structure to be searched is trained based on the training data of the media task until the number of selections reaches a threshold or the target supernet converges, and the training is stopped.

[0040] In one embodiment, the apparatus further comprises:

[0041] An intensive training module is used to obtain a training data set corresponding to the media task; and perform intensive training on the target neural network structure based on the training data set;

[0042] The task processing module is used to process the media task according to the target neural network structure after intensive training.

[0043] In one embodiment, the task processing module is further configured to:

[0044] Acquire data to be processed corresponding to the media task; the data to be processed includes any one of image data, voice data and text data;

[0045] Inputting the data to be processed into the target neural network structure after the intensive training;

[0046] The data to be processed is processed layer by layer through each network layer of the target neural network structure after the intensive training to obtain a processing result.

[0047] In one embodiment, the media task includes any one of an image processing task, a speech processing task, and a text processing task, and the training data of the media task includes any one of image sample data, speech sample data, and text sample data.

[0048] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0049] Obtain the feature vector of the neural network structure in the neural network search space;

[0050] Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs;

[0051] Selecting a neural network structure to be searched from the neural network structures of each category;

[0052] Constructing a target neural network search space based on the selected neural network structure to be searched;

[0053] After the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0055] Obtain the feature vector of the neural network structure in the neural network search space;

[0056] Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs;

[0057] Selecting a neural network structure to be searched from the neural network structures of each category;

[0058] Constructing a target neural network search space based on the selected neural network structure to be searched;

[0059] After the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

[0060] A computer program, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the following steps:

[0061] Obtain the feature vector of the neural network structure in the neural network search space;

[0062] Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs;

[0063] Selecting a neural network structure to be searched from the neural network structures of each category;

[0064] Constructing a target neural network search space based on the selected neural network structure to be searched;

[0065] After the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

[0066] The above-mentioned neural network structure search method, device, computer equipment and storage medium obtain the characteristic vector of the neural network structure in the neural network search space, cluster the obtained neural network structures based on the characteristic vector, obtain the category to which each neural network structure belongs, and select the neural network structure to be searched from the neural network structures of each category, and construct the target neural network search space based on the selected neural network structure to be searched. It is possible to crop the original neural network search space to obtain the target neural network search space, and then when a network search request for a media task is received, the neural network structure to be searched in the target neural network search space can be directly trained based on the training data of the media task, and then the target neural network structure for processing the media task is searched in the trained target neural network search space, saving training time and computing resources used for searching, thereby improving the search efficiency of the neural network structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A diagram showing an application environment of a neural network structure search method according to an embodiment;

[0068] Figure 2 1 is a flow chart of a neural network structure search method according to an embodiment;

[0069] Figure 3 is a directed acyclic graph corresponding to a neural network search space in one embodiment;

[0070] Figure 4 A schematic diagram of a super network in one embodiment;

[0071] Figure 5 A schematic diagram of the structure of a MobileNetV1 block in one embodiment;

[0072] Figure 6A A schematic diagram of the structure of a LinearBottlenect block in one embodiment;

[0073] Figure 6B A schematic diagram of the structure of a LinearBottlenect block in one embodiment;

[0074] Figure 7 Schematic diagram of a flow chart of a feature vector acquisition step in one embodiment;

[0075] Figure 8 Schematic diagram of a clustering step in one embodiment;

[0076] Figure 9 A schematic flow chart of the steps for selecting a neural network structure to be searched in one embodiment;

[0077] Figure 10Schematic diagram of a process for searching a target neural network structure in one embodiment;

[0078] Figure 11 1 is a flow chart of a neural network structure search method according to another embodiment;

[0079] Figure 12 A schematic diagram of defining a super network in one embodiment;

[0080] Figure 13 is a structural block diagram of a neural network structure search device in one embodiment;

[0081] Figure 14 is a structural block diagram of a neural network structure search device in another embodiment;

[0082] Figure 15 is a diagram of the internal structure of a computer device in one embodiment;

[0083] Figure 16 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0085] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0086] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0087] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying, tracking, and measuring objects. This involves further processing the images, transforming them into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0088] Natural language processing (NLP) is a key area of research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0089] The neural network structure search method provided in the embodiment of the present application involves artificial intelligence machine learning and other technologies, which can be specifically applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The embodiment provided in the present application can be executed separately on the terminal 102 or the server 104, and can also be implemented through the interaction between the terminal 102 and the server 104. Taking the execution on the server 104 as an example, the server 104 obtains the characteristic vector of the neural network structure in the neural network search space; clusters the obtained neural network structure based on the characteristic vector to obtain the category to which each neural network structure belongs; selects the neural network structure to be searched from the neural network structures of each category; constructs the target neural network search space based on the selected neural network structure to be searched; after the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, searches for the target neural network structure for processing the media task in the trained target neural network search space.

[0090] Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices. The server 104 can be an independent physical server or a server cluster composed of multiple service nodes in the blockchain system. The service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP) protocol.

[0091] In addition, server 104 can also be a server cluster composed of multiple physical servers, and can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0092] In one embodiment, Figure 2 As shown, a neural network structure search method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the computer device (terminal or server) in the example:

[0093] S202, obtaining a feature vector of a neural network structure in a neural network search space.

[0094] Among them, the neural network search space is the space for searching the neural network structure, and the neural network search space defines the various neural network structures that can be characterized. The search space in the embodiment of the present application can be the space of a chain structure neural network, and the neural network structure contained therein can be a multi-layer sequence. For example, the neural network structure A can be an N-layer sequence, where the i-th layer receives its input from the i-1-th layer, and its output is used as the input of the i+1 layer. The eigenvector of the neural network structure is a vector used to characterize the structural characteristics of the neural network. It can be understood that for two neural network structures with similar structures, the eigenvectors corresponding to the two structures are also similar, that is, the distance between the eigenvectors corresponding to the two structures is small.

[0095] The neural network search space specifically defines the maximum number of layers in the neural network structure, the operations that can be performed by each layer, and the hyperparameters related to the operations. The operations that can be performed by each layer can be pooling, concatenation, skip connection, elemental addition, convolution, and some advanced convolutions such as depth-separable convolution or dilated convolution. The hyperparameters related to the operations can be the number of filters, the kernel size and the stride of the convolution layer, or simply the number of modules used for the fully connected network.

[0096] Specifically, all neural network structures in the neural network search space can be viewed as a directed acyclic graph (DAG). Each layer of the neural network structure is a node in the DAG, and the connection between layers is the edge in the DAG. Figure 3 The DAG calculation diagram of the neural network structure shown in the figure shows that when the connection relationship between different layers is determined, if each layer has m parameters, that is, there are m possible operations corresponding to each layer, and if the number of layers is N, then the neural network structure has m N possibilities, that is, the size of the corresponding neural network search space is m N ; When the connection relationship between different layers is not determined, if each node is numbered in sequence, then at the kth layer, at most k-1 different indexes can be sampled, resulting in 2 k-1 possible connections. If each layer has m parameters, that is, there are m possible operations corresponding to each layer. If the number of layers is N, then the neural network structure is possibilities, that is, the size of the corresponding neural network search space is

[0097] In one embodiment, if each node in the directed acyclic graph corresponding to all neural network structures in the neural network search space is fixed, that is, the neural network structure determines the connection relationship between different layers, then the resulting node-fixed directed acyclic graph corresponds to a hypernetwork (Hypernet or Supernet), and a subnetwork in the hypernetwork represents a neural network structure in the neural network search space. It should be noted that the hypernetwork in each embodiment of the present application can also be simply referred to as a supernet.

[0098] Figure 4 is a schematic diagram of a super network in an embodiment, Figure 4 On the left side is a chain neural network. Figure 4The right side of the middle is the hypernetwork corresponding to the chain neural network. The chain neural network consists of 3 layers. The parameters corresponding to each layer have 3 values, that is, each layer has 3 possibilities. Figure 4 The mesh network shown on the right side is the super network representing the chain neural network. Figure 4 The neural network structure corresponding to each path on the right side is a sub-network of the chain neural network, that is, the neural network structure corresponding to each path is the neural network structure in the search space corresponding to the chain neural network. For example, "B0T0-B1T1-B2T1" and "B0T0-B1T1-B2T2" are two different neural network structures. Figure 4 The hypernetwork of the chain neural network contains 3 3 A neural network structure, Figure 4 The size of the neural network search space of the chain neural network is 3 3 .

[0099] Taking the MobileNetV2 model as an example, the neural network search space corresponding to the MobileNetV2 model is explained. MobileNets is a lightweight neural network structure that can operate on mobile terminals such as mobile phones. The MobileNetV2 model consists of 21 blocks, and each block is numbered according to the connection sequence from input to output. The structure of the MobileNetV2 model is shown in the following table:

[0100] Table 1 Schematic diagram of mobilenetv2 model structure

[0101] serial number type 0 ConvBnReLU 1 MobileNetV1Block 2 LinearBottlenect .. .. 17 LinearBottlenect 18 ConvBnReLU 19 GlobalPooling 20 Fc

[0102] In the above table, ConvBnReLU corresponding to numbers 0 and 18 are convolution, batch normalization, and linear rectification function blocks, MobileNetV1Block corresponding to number 1 is the MobileNetV1 block, blocks corresponding to numbers 2 to 17 are all LinearBottlenect, which is a linear bottleneck structure block, GlobalPooling corresponding to number 19 is a global pooling block, and Fc corresponding to number 20 is a fully connected block.

[0103] The structure of MobileNetV1Block is as follows Figure 5As shown, it includes DepthwiseConv (depth-wise separable convolution layer) and PointwiseConv (point-by-point convolution layer). Since the MobileNetV1Block is only used once in the mobilenetv2 model, its parameters can be fixed. In other words, the MobileNetV1Block does not affect the size of the search space, and it can also be said that it does not participate in the search.

[0104] The structure of LinearBottlenect can be Figure 6A The structure shown, or Figure 6B The structures shown in the figure both include Expand (channel amplification layer), DepthwiseConv (depth-separable convolution layer) and Linear (linear layer). Expand is a sequential module (Sequential) of ConvBnReLU. The parameters corresponding to this layer include the channel amplification factor (expand_ratio). Its function is to use a 1x1 convolution kernel to expand the number of channels (channel) of the feature map. For example, when expand_ratio is 2, Expand will expand the number of channels of the feature map with an input channel number of 16 to 32; DepthwiseConv is a special convolution operator, and its corresponding parameters include the convolution kernel size (kernel_size) and the convolution step size (stride); Linear is a sequential module (Sequential) of ConvBn, and its parameters include the number of output channels (out_channel). Its function is to use a 1x1 convolution kernel to compress the number of channels (channel) of the feature map. For Figure 6A A LinearBottlenect shown in the figure, when its Expand input channel is consistent with the Linear output channel and the stride is not 2, will Figure 6A Adding skip connections to the structure shown in the figure yields Figure 6B The structure shown.

[0105] According to the above description, LinearBottlenect is the basic block structure of mobilenetv2. Each LinearBottlenect in mobilenetv2 can be characterized by parameters of five dimensions: in_channels, out_channel, stride, expand_ratio, and kernel_size. Among them, the in_channels of the current layer LinearBottlenect is the out_channel of the previous layer LinearBottlenect. Therefore, for each LinearBottlenect in mobilenetv2, the parameters of four dimensions: out_channel, stride, expand_ratio, and kernel_size can be used to characterize it. The specific values of the parameters in each dimension can be shown in Table 2 below:

[0106] Table 2. LinearBottlenect parameters

[0107] Parameter Dimension Value stride 1，2 expand_ratio 2，4，6，8 kernel_size 3，5 out_channel 4，6，8，10，12

[0108] In the table above, stride has two values of 1 and 4, expand_ratio has four values of 2, 4, 6 and 8, kernel_size has two values of 3 and 5, and out_channel has five values of 4, 6, 8, 10 and 12. For a LinearBottlenect, the corresponding specific network structure has 2×4×2×5=80 possibilities. For a chain neural network structure with 16 layers of LinearBottlenect, the subnetworks included in its super network are 80. 16 That is, the search space size corresponding to the 16-layer LinearBottlenect chain neural network structure is 80 16 .

[0109] In one embodiment, after obtaining a preset neural network search space, the computer device directly extracts each neural network structure from the neural network search space and calculates a feature vector of the corresponding neural network structure based on the parameters of each neural network structure.

[0110] In one embodiment, the server obtains a supernetwork corresponding to the neural network search space, determines the subnetworks corresponding to the supernetwork based on the supernetwork, and determines the obtained subnetworks as the neural network structure in the neural network search space, and then calculates the feature vectors of each neural network structure based on the obtained neural network structure.

[0111] S204: Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs.

[0112] A "class" is a collection of similarities, and clustering involves dividing a dataset into classes so that the data within each class is as similar as possible, while the data between classes is as different as possible. Clustering neural network structures involves grouping similar neural network structures into the same class.

[0113] Specifically, after obtaining the feature vectors of each neural network structure, the computer device performs similarity calculation on the neural network structure to obtain the similarity between each neural network structure, and determines the category to which each neural network structure belongs based on the similarity between each neural network structure.

[0114] In one embodiment, the computer device calculates the similarity between each neural network structure based on a preset clustering algorithm and the feature vectors of each neural network structure, and determines the category to which each neural network structure belongs based on the similarity between each neural network structure.

[0115] The clustering algorithm can be a partition-based clustering algorithm, specifically a k-means clustering algorithm, a k-medoids clustering algorithm, or a clarans clustering algorithm. The k-means clustering algorithm is a simple iterative clustering algorithm that uses distance as a similarity metric to discover K clusters in a given data set. The center of each cluster is obtained based on the mean of all values in the cluster, and the center of each cluster is described by a cluster center.

[0116] For example, if the neural network search space contains 50,000 neural network structures, the computer device can obtain the feature vectors of the 50,000 neural network structures respectively, and use the k-means clustering algorithm to cluster the 50,000 neural network structures based on the feature vectors of the 50,000 neural network structures to obtain 100 categories, each of which includes at least one neural network structure, and the 100 categories contain the 50,000 neural network structures in total.

[0117] S206, selecting the neural network structures to be searched from the neural network structures of each category.

[0118] The selection can be random or based on merit. The selection refers to sorting the performance of each neural network structure under a certain category, and selecting the neural network structure corresponding to the preset ranking threshold according to the sorting result.

[0119] In one embodiment, after determining the category to which each neural network structure belongs, the computer device uses a preset random algorithm to select a neural network structure to be searched from the neural network structures belonging to the category for each category.

[0120] Specifically, for each category, the computer device forms a corresponding super network based on the various neural network structures belonging to the same category, and for the multiple network layers contained in the super network, a random algorithm is used to determine the corresponding parameters of the layer layer by layer, thereby obtaining a randomly selected neural network structure to be searched.

[0121] It should be noted that when the number of neural network structures to be searched in the same category is more than two, each time a candidate network structure is randomly selected, the candidate network structure is deleted from the corresponding supernetwork to obtain the deleted supernetwork, and the next neural network structure to be searched is randomly selected based on the deleted supernetwork until a preset number of supernetworks are selected from the supernetwork.

[0122] refer to Figure 4 As shown in the hypernetwork diagram, assuming Figure 4 If the supernetwork in is a supernetwork corresponding to a certain category, the computer device randomly selects the parameters corresponding to the corresponding layers from layer 0 to layer 2. Specifically, for layer 0, one of the parameters "B0T0", "B0T1", and "B0T2" is randomly selected as the parameter of layer 0; for layer 1, one of the parameters "B1T0", "B1T1", and "B1T2" is randomly selected as the parameter of layer 1; for layer 2, one of the parameters "B2T0", "B2T1", and "B2T2" is randomly selected as the parameter of layer 2. If the parameter selected for layer 0 is "B0T1", the parameter selected for layer 1 is "B1T0", and the parameter selected for layer 2 is "B2T2", then the randomly selected neural network structure to be searched is "B0T1-B1T0-B2T2".

[0123] In one embodiment, after determining the category to which each neural network structure belongs, the computer device uses a preset optimal selection algorithm for each category to select the neural network structure to be searched from the neural network structures belonging to the category.

[0124] In one embodiment, after determining the category to which each neural network structure belongs, the computer device trains the neural network structures in each category respectively, determines the performance of the neural network structure based on the training results, and uses the neural network structure that meets the performance requirements as the neural network structure to be searched.

[0125] In one embodiment, after determining the category to which each neural network structure belongs, the computer device trains the neural network structures in each category respectively, determines the performance of each neural network structure based on the training results, and sorts each neural network structure based on the performance of each neural network structure, and selects the neural network structure to be searched from each neural network structure according to the sorting results.

[0126] Specifically, the computer device can train various neural network structures under the same category based on the training data of the media task, so as to select the neural network structure to be searched with better performance corresponding to the media task.

[0127] A media task is a task that can be processed by a neural network, and specifically can be an image processing task, a speech processing task, or a text processing task. An image processing task can be image classification, object detection, or image segmentation, a speech processing task can be speech synthesis, speech recognition, and a text processing task can be text recognition, text classification, or text analysis. The training data for a media task refers to training data that matches the category of the media task. For example, if the media task is an image processing task, the corresponding training data is image sample data; if the media task is a speech processing task, the corresponding training data is speech sample data; and if the media task is a text processing task, the corresponding training data is text sample data.

[0128] In one embodiment, for each category, the computer device forms a corresponding super network based on the various neural network structures belonging to the same category, and evaluates the performance of each sub-network contained in the super network to obtain the performance of each sub-network, and sorts the sub-networks based on the performance of each sub-network. According to the sorting results, the neural network structure to be searched is selected from each sub-network.

[0129] refer to Figure 4 As shown in the hypernetwork diagram, assuming Figure 4 The super network in is the super network corresponding to a certain category. The neural network structure corresponding to each path in the super network is the sub network of the category. Figure 4 The hypernetwork in the 3 The computer equipment evaluates the performance of each sub-network and sorts them out in descending order. 3 The sub-networks are sorted, and the sub-network ranked first is determined as the neural network structure to be searched. If the sub-network ranked first is "B0T1-B1T0-B2T2", the neural network structure to be searched selected from this category is "B0T1-B1T0-B2T2".

[0130] In one embodiment, after determining the categories to which each neural network structure belongs, the computer device may select a preset number of neural network structures to be searched for each category. The preset number corresponding to each category may be the same or different.

[0131] In one embodiment, the preset number can be determined based on the number of neural network structures included in each category. Specifically, after determining the category to which each neural network structure belongs, the computer device determines the number of neural network structures included in each category, and based on the proportion of this number in the total number of neural network structures in all categories, determines the preset number of neural network structures to be searched in each category.

[0132] For example, the neural network search space contains 2000 neural network structures, and the 2000 neural network structures are clustered into 3 categories, among which category 1 contains 400 neural network structures, category 2 contains 800 neural network structures, and category 3 contains 800 neural network structures. If a total of 50 neural network structures to be searched are to be selected in each category, then the preset number selected in category 1 is determined to be 400 / 2000×50=10, the preset number selected in category 2 is determined to be 800 / 2000×50=20, and the preset number selected in category 3 is determined to be 800 / 2000×50=20.

[0133] S208, constructing a target neural network search space based on the selected neural network structure to be searched.

[0134] Specifically, after the computer device selects the neural network structures to be searched from each category, it determines the search space corresponding to all the selected neural network structures to be searched as the target neural network search space.

[0135] For example, if the neural network search space contains 50,000 neural network structures, the computer device can obtain the feature vectors of the 50,000 neural network structures respectively, and use the k-means clustering algorithm to cluster the 50,000 neural network structures based on the feature vectors of the 50,000 neural network structures to obtain 100 categories, where each category includes at least one neural network structure, and the 100 categories contain the 50,000 neural network structures in total. Then, for the 100 categories, 5 neural network structures to be searched are selected from each category, so that a total of 500 neural network structures to be searched are selected, and the search space composed of the 500 neural network structures to be searched is determined as the target neural network search space.

[0136] In one embodiment, if the neural network structure to be searched is a neural network structure selected based on a media task, then the target neural network search space constructed based on the neural network structure to be searched corresponds to the media task. For example, if the neural network structure to be searched is selected based on an image processing task, then the target neural network search space constructed is a neural network search space of the image processing task category; if the neural network structure to be searched is selected based on a speech processing task, then the target neural network search space constructed is a neural network search space of the speech processing task category; if the neural network structure to be searched is selected based on a text processing task, then the target neural network search space constructed is a neural network search space of the text processing task category.

[0137] S210, after the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

[0138] Among them, media tasks are tasks that can be processed by neural networks, specifically image processing tasks, speech processing tasks or text processing tasks. Image processing tasks can be tasks such as image classification, target detection or image segmentation. Speech processing tasks can be tasks such as speech synthesis and speech recognition. Text processing tasks can be tasks such as text recognition, text classification, and text analysis.

[0139] It is understandable that the neural network structures used to process different media tasks may be different. Based on this, when searching for the target neural network structure from the target neural network structure, the type of media task can be specific and the target neural network structure that matches the type of media task can be selected so that the corresponding media task can be processed using the selected target neural network structure.

[0140] For example, for image classification tasks, the target neural network structure for image classification processing is searched from the target neural network space; for speech recognition tasks, the target neural network structure for speech recognition processing is searched from the target neural network space; for text recognition tasks, the target neural network structure for text recognition processing is searched from the target neural network space.

[0141] The training data of a media task refers to the training data that matches the category of the media task. For example, if the media task is an image processing task, the corresponding training data is image sample data; if the media task is a speech processing task, the corresponding training data is speech sample data; if the media task is a text processing task, the corresponding training data is text sample data.

[0142] Specifically, before searching for the target neural network structure from the target neural network search space, the training data corresponding to the media task can be used to train the neural network structure to be searched in the target neural network space to obtain a training result. When a network search request for a media task is received, the network search request for the media task is directly responded to, and the performance of each neural network structure to be searched in processing the media task is evaluated based on the training result to obtain the performance results of each neural network structure to be searched. Then, based on the performance results of each neural network structure to be searched, the target neural network structure for processing the media task is selected from each neural network structure to be searched.

[0143] In the above-mentioned neural network structure search method, by obtaining the characteristic vector of the neural network structure in the neural network search space, the obtained neural network structures are clustered based on the characteristic vector to obtain the category to which each neural network structure belongs, and the neural network structure to be searched is selected from the neural network structures of each category, and the target neural network search space is constructed based on the selected neural network structure to be searched. The original neural network search space can be cropped to obtain the target neural network search space, and then when a network search request for a media task is received, the neural network structure to be searched in the target neural network search space can be directly trained based on the training data of the media task, and then the target neural network structure for processing the media task is searched in the trained target neural network search space, which saves training time and computing resources used for search, thereby improving the search efficiency of the neural network structure.

[0144] In one embodiment, Figure 7 As shown, S202 specifically includes the following steps:

[0145] S702, obtaining the receptive field and parameter quantity of each network layer in the neural network structure in the neural network space.

[0146] The receptive field is the size of the area on the original image where pixels on the feature map output by each layer of the neural network are mapped. The parameter value is the value obtained by calculating the parameters corresponding to each layer of the neural network structure.

[0147] Specifically, after obtaining the neural network structure, the computer device determines the values of the parameters of each dimension corresponding to each layer of the neural network structure, and then calculates the receptive field corresponding to each layer based on the values of the parameters of the target dimension in the dimensions of each parameter. The values of the parameters of each dimension are calculated according to a preset operation method to obtain the parameter quantity corresponding to each layer. The parameters of each dimension include the convolution kernel size, convolution step size, number of input channels, number of output channels, etc.

[0148] In one embodiment, the computer determines the parameters of the target dimension as the convolution kernel size and the convolution step size. For any neural network structure, in order from input to output, the receptive field corresponding to the current layer is determined based on the convolution kernel size and the convolution step size corresponding to each layer of the network, and the receptive field corresponding to each layer of the network is obtained.

[0149] For example, for a certain neural network structure, in the order from input to output, the convolution kernel of the first layer is 3×3, and the convolution step is 1, then the receptive field corresponding to the first layer is 3×3; the convolution kernel of the second layer is 3×3, and the convolution step is 1, then the receptive field corresponding to the second layer is calculated by the following formula (1), and the receptive field corresponding to the second layer is 3×3, where formula (1) is as follows:

[0150] RF i =(RF i-1 -1)×stride i +ksize i (1)

[0151] Among them, RF i is the receptive field corresponding to the i-th layer network in the neural network structure, stride i ksize is the convolution step size corresponding to the i-th layer network in the neural network structure, i The convolution kernel size corresponding to the i-th layer network in the neural network structure.

[0152] In one embodiment, after obtaining the values of the parameters of each dimension, the computer device performs at least one of product and summation on the values of the parameters of each layer of the network for each layer of any neural network structure, obtains the operation result, and determines the operation result as the parameter quantity corresponding to the layer of the network.

[0153] For example, the product of the convolution kernel size, convolution step size, number of input channels, and number of output channels corresponding to each layer of the network is determined as the parameter corresponding to the layer of the network.

[0154] S704: Determine a feature vector of the neural network structure according to the receptive field and parameter quantity of the neural network structure.

[0155] Specifically, for any neural network structure, after determining the receptive fields and parameter quantities corresponding to each layer of the network, the computer device can vectorize the receptive fields and parameter quantities corresponding to each layer of the network to obtain a vector, and determine the obtained vector as the characteristic vector of the neural network structure.

[0156] In one embodiment, S704 specifically includes the following steps: determining a first eigenvector based on the receptive field of each network layer in the neural network structure; determining a second eigenvector based on the parameter amount of each network layer in the neural network structure; and constructing a eigenvector of the neural network structure based on the first eigenvector and the second eigenvector.

[0157] Specifically, for any neural network structure, after obtaining the receptive field corresponding to each layer of the network in the neural network structure, the computer device vectorizes the receptive field corresponding to each layer of the network, and determines the result of the vectorization processing as the first eigenvector; vectorizes the parameter quantity corresponding to each layer of the network, and determines the result of the vectorization processing as the second eigenvector; after obtaining the first eigenvector and the second eigenvector, the first eigenvector and the second eigenvector are spliced, and the spliced result is determined as the eigenvector of the corresponding neural network structure.

[0158] For example, for Figure 4 For any sub-network in the super-network shown, by vectorizing the receptive field of each layer of the sub-network, a 3-dimensional first eigenvector can be obtained, and by vectorizing the parameter quantity of each layer of the sub-network, a 3-dimensional second eigenvector can be obtained. The first eigenvector and the second eigenvector are concatenated to obtain a 6-dimensional eigenvector, which is the eigenvector corresponding to the sub-network.

[0159] In the above embodiment, the computer device obtains the receptive field and parameter amount of each network layer in the neural network structure in the neural network space, determines the characteristic vector of the neural network structure based on the receptive field and parameter amount of the neural network structure, and then clusters the neural network structures based on the characteristic vectors of each neural network structure. Based on the clustering results, neural networks with similar structures are deleted to obtain a target neural network search space, and then when searching in the target neural network search space, the training time and computing resources used for the search are reduced, thereby improving the search efficiency of the neural network structure.

[0160] In one embodiment, Figure 8 As shown, S204 specifically includes the following steps:

[0161] S802: Sampling from the neural network structure to obtain at least two neural network structure samples.

[0162] Specifically, the computer device can randomly select from various neural network structures and determine the selected neural network structure as a neural network structure sample.

[0163] In one embodiment, the computer device obtains a preset number of categories and selects a neural network structure sample that matches the preset category data from the neural network structure.

[0164] For example, there are 50,000 neural network structures in the neural network space, and the preset number of categories is 100. Then, 100 neural network structures are randomly selected from the 50,000 neural network structures as neural network structure samples.

[0165] S804: Use the neural network structure sample as a cluster center, and determine a first distance between other neural network structures and the cluster center based on the feature vector.

[0166] Among them, the first distance represents the similarity between other neural network structures and the neural network structure corresponding to the cluster center. It can be understood that the larger the first distance, the smaller the corresponding similarity, and the smaller the first distance, the greater the corresponding similarity.

[0167] Specifically, after determining the cluster center, the computer device calculates the first distance between any other neural network structure and each cluster center based on the feature vector of the neural network structure and the feature vector of each cluster center.

[0168] S806: Determine the category to which the other neural network structures belong based on the first distance to obtain an initial category.

[0169] Specifically, for any other neural network structure, after determining its first distance to each cluster center, the computer device determines the cluster center with the smallest first distance to the other neural network structure as the cluster center to which the other neural network structure belongs, and determines the classification corresponding to the cluster center as the classification to which the other neural network structure belongs, thereby obtaining the initial category to which each neural network structure belongs.

[0170] S808, determining the centroid in the neural network structure of the initial category, and determining a second distance between the neural network structure outside the centroid and the centroid, determining the category to which the neural network structure outside the centroid belongs based on the second distance, and obtaining the category to which each neural network structure belongs.

[0171] Among them, the centroid is the updated cluster center.

[0172] In one embodiment, for any initial category, the computer device determines a centroid vector based on the feature vectors of each neural network structure in the initial category. The neural network structure corresponding to the determined centroid vector is the centroid of the initial category. The neural network structure corresponding to the centroid vector has the smallest sum of distances with each neural network structure in the corresponding initial category.

[0173] In one embodiment, after determining the updated cluster center corresponding to each initial category, the computer device calculates the second distance between other neural network structures outside the updated cluster center and the updated cluster center, and determines the category to which the neural network structure outside the updated cluster center belongs based on the second distance, thereby obtaining the category to which each neural network structure belongs.

[0174] Specifically, after determining the updated cluster centers, the computer device calculates, for each of the other neural network structures, the second distances between the neural network structure and each of the updated cluster centers based on the feature vector of the neural network structure and the centroid vector of each of the updated cluster centers. For each of the other neural network structures, after determining the first distances to each of the cluster centers, the computer device determines the updated cluster center with the smallest second distance to the other neural network structure as the cluster center to which the other neural network structure belongs, and determines the category corresponding to the updated cluster center as the category to which the other neural network structure belongs, thereby obtaining the category to which each of the neural network structures belongs.

[0175] In one embodiment, after determining the category to which each neural network structure belongs, the computer device may determine the obtained category as the initial category and iteratively execute S808 until the maximum number of iterations is reached, or the distance between the determined center of mass and the corresponding initial category is less than a preset threshold, and then the category to which each neural network structure belongs determined when the iteration terminates is output as the result.

[0176] In the above embodiment, the computer device obtains at least two neural network structure samples by sampling from the neural network structure, uses the neural network structure samples as cluster centers, determines a first distance between other neural network structures and the cluster center based on the feature vector, determines the category to which the other neural network structures belong based on the first distance, obtains an initial category, determines the center of mass in the neural network structure of the initial category, and determines a second distance between the neural network structure outside the center of mass and the center of mass, determines the category to which the neural network structure outside the center of mass belongs based on the second distance, obtains the category to which each neural network structure belongs, and the neural network structures belonging to the same category are similar, so that based on the clustering results, neural networks with similar structures can be deleted to obtain a target neural network search space, and then when searching in the target neural network search space, the training time and computing resources used for searching are reduced, thereby improving the search efficiency of the neural network structure.

[0177] In one embodiment, Figure 9 As shown, S206 specifically includes the following steps:

[0178] S902, the neural network structures in each category are combined into a supernet for training.

[0179] Specifically, for any category, the computer device organizes all the neural network structures included in the category into a super network and trains the super network to obtain the shared weights of each sub-network in the super network, that is, to obtain the shared weights of each neural network structure in the super network.

[0180] In one embodiment, a computer device trains a supernetwork using a single-path training method. The specific process is as follows: a neural network structure is randomly selected from the supernetwork each time, and the selected neural network structure is trained until the number of selections reaches a threshold or the target supernetwork converges, and the training is stopped.

[0181] Specifically, the computer randomly samples a subnetwork from the hypernetwork for training each time. During the reverse parameter update, only the training parameters of that subnetwork are updated, without affecting other subnetworks that were not sampled. In the next iteration, the computer resamples, selecting another subnetwork for training. As the number of iterations increases, the entire hypernetwork is trained to convergence. When training a hypernetwork using a single-path training method, only the network activations need to be stored in memory for calculations, thus reducing the demand on computing resources during training and reducing the amount of computing resources used.

[0182] In one embodiment, a computer device can train supernetworks of various categories based on training data for media tasks. A media task is a task that can be processed by a neural network, and specifically can be an image processing task, a speech processing task, or a text processing task. An image processing task can be a task such as image classification, object detection, or image segmentation. A speech processing task can be a task such as speech synthesis or speech recognition. A text processing task can be a task such as text recognition, text classification, or text analysis. The training data for a media task refers to training data that matches the category of the media task. For example, if the media task is an image processing task, the corresponding training data is image sample data. If the media task is a speech processing task, the corresponding training data is speech sample data. If the media task is a text processing task, the corresponding training data is text sample data.

[0183] S904: After the training is completed, the neural network structure in each category is tested based on the test set to obtain a first test result.

[0184] The test set is a test data set used to verify the performance of the neural network structure, and specifically can be a test data set based on media tasks.

[0185] It can be understood that when the training data is image sample data, the corresponding test set is image test data; when the training data is voice sample data, the corresponding test set is voice test data; when the training data is text sample data, the corresponding test set is text test data.

[0186] In one embodiment, for any category, the computer device randomly selects a neural network structure in the category, and verifies the selected neural network structure on a test set using weights inherited from the super network to obtain a first test result of the selected neural network structure.

[0187] In one embodiment, for any category, the computer device verifies at least a portion of the neural network structures in the category on a test set using weights inherited from the supernetwork to obtain a first test result for the at least a portion of the neural network structures.

[0188] S906 , selecting a neural network structure to be searched from among the neural network structures in each category based on the first test result.

[0189] In one embodiment, for any category, the computer device randomly selects a neural network structure in the category, and after obtaining a first test result of the selected neural network structure, determines whether the first test result of the selected neural network structure meets the selection conditions. If so, the selected neural network structure is directly determined as the neural network structure to be searched; if not, the neural network structure to be searched is selected again in the category.

[0190] The first test result may be a performance parameter, for example, for an image classification task, the first test result may be an image classification accuracy rate. The selection condition may be a performance parameter threshold or a performance ranking threshold.

[0191] Specifically, when the selection condition is a performance parameter, the computer device determines that the selected neural network structure reaches the performance parameter threshold, then determines the selected neural network structure as the neural network structure to be searched; when the selection condition is a performance ranking threshold, the computer device randomly selects a preset number of neural network structures in the category, and based on the first test results of each selected neural network structure, sorts the selected neural network structures, and determines at least one neural network structure whose ranking meets the performance ranking threshold as the neural network structure to be searched.

[0192] In the above embodiment, the computer device trains by respectively organizing the neural network structures in each category into a supernet. After the training is completed, the neural network structures in each category are tested based on the test set to obtain a first test result. Among the neural network structures in each category, the neural network structure to be searched is selected based on the first test result, so that the neural network structure with better performance among similar structures can be retained and the neural network structure with poorer performance among similar structures can be deleted. Then, when searching in the target neural network search space, the target neural network structure with the best performance can be directly selected from the neural network structures with better performance in a shorter time, thereby improving the efficiency of the neural network structure search.

[0193] In one embodiment, the neural network structure search method further includes a process of training the neural network structure to be searched in the target neural network space, specifically comprising the following steps: forming a target supernet from the neural network structures to be searched in the target neural network space, and training the target supernet based on the training data of the media task. The target supernet is the target supernetwork.

[0194] Specifically, the computer device organizes all the neural network structures to be searched included in the target neural network space into a supernetwork to obtain the target supernetwork, and trains the target supernetwork to obtain the shared weights of each target subnetwork in the target supernetwork, that is, to obtain the shared weights of each neural network structure to be searched in the target supernetwork.

[0195] In one embodiment, a computer device trains a target supernetwork using a single-path training method. The specific process is as follows: a neural network structure to be searched is randomly selected from the target supernetwork each time, and the selected neural network structure to be searched is trained until the number of selections reaches a threshold or the target supernetwork converges, and the training is stopped.

[0196] Specifically, the computer randomly samples a target subnetwork from the target supernetwork for training each time. During the reverse parameter update, only the training parameters of that target subnetwork are updated, without affecting other target subnetworks that were not sampled. In the next iteration, the computer resamples, selecting another target subnetwork for training. As the number of iterations increases, the entire target supernetwork is trained to convergence. When training the target supernetwork using a single-path training method, only the network activations need to be stored in memory for computation, thus reducing the need for computing resources during training and reducing the amount of computing resources used.

[0197] In one embodiment, a computer device can train target supernetworks of various categories based on training data of media tasks. A media task is a task that can be processed by a neural network, and specifically can be an image processing task, a speech processing task, or a text processing task. An image processing task can be a task such as image classification, target detection, or image segmentation. A speech processing task can be a task such as speech synthesis or speech recognition. A text processing task can be a task such as text recognition, text classification, or text analysis. The training data of a media task refers to training data that matches the category of the media task. For example, if the media task is an image processing task, the corresponding training data is image sample data. If the media task is a speech processing task, the corresponding training data is speech sample data. If the media task is a text processing task, the corresponding training data is text sample data.

[0198] In the above embodiment, the computer device trains the target supernetwork corresponding to the target neural network search space by adopting a single-path training method, which can reduce the time spent on training the supernetwork during the neural network search, increase the training speed, and thus improve the efficiency of the neural network structure search.

[0199] In one embodiment, Figure 10 As shown, S210 specifically includes the following steps:

[0200] S1002, in response to a network search request of a media task, testing a neural network structure to be searched in a target neural network search space according to test data of the media task to obtain a second test result.

[0201] Among them, when the training data is image sample data, the corresponding test data is image test data; when the training data is voice sample data, the corresponding test data is voice test data; when the training data is text sample data, the corresponding test data is text test data.

[0202] In one embodiment, a computer device randomly selects a neural network structure to be searched in a target neural network search space, and verifies the selected neural network structure to be searched on test data using weights inherited from a target supernetwork to obtain a second test result of the selected neural network structure to be searched.

[0203] In one embodiment, the computer device verifies at least a portion of the neural network structures to be searched in the target neural network search space on the test data using the weights inherited from the parent super network to obtain a second test result for at least a portion of the neural network structures to be searched.

[0204] S1004: Searching for a target neural network structure for processing the media task based on the second test result from the trained target neural network search space.

[0205] In one embodiment, a computer device randomly selects a neural network structure to be searched in a target neural network search space, and after obtaining a second test result of the selected neural network structure, determines whether the second test result of the selected neural network structure to be searched meets a selection condition. If so, the selected neural network structure to be searched is directly determined as the target neural network structure; if not, the neural network structure to be searched is selected again in the target neural network search space.

[0206] The second test result may be a performance parameter, for example, for an image classification task, the second test result may be an image classification accuracy rate. The selection condition may be a performance parameter threshold or a performance ranking threshold.

[0207] Specifically, when the selection condition is a performance parameter, the computer device determines the selected neural network structure as the target neural network structure when it determines that the selected neural network structure reaches the performance parameter threshold; when the selection condition is a performance ranking threshold, the computer device randomly selects a preset number of neural network structures in the target neural network search space, and based on the second test results of each selected neural network structure to be searched, sorts the selected neural network structures to be searched, and determines at least one neural network structure to be searched whose ranking meets the performance ranking threshold as the target neural network structure.

[0208] In the above embodiment, the computer device tests the neural network structure to be searched in the target neural network search space based on the test data of the media task to obtain a second test result; from the trained target neural network search space, the target neural network structure for processing the media task is searched based on the second test result, so that the target neural network structure with the best performance can be selected from the various neural network structures to be searched in a shorter time, thereby improving the efficiency of the neural network structure search.

[0209] In one embodiment, the above-mentioned neural network structure search method also includes the following steps: obtaining a training data set corresponding to the media task; performing intensive training on the target neural network structure based on the training data set; and processing the media task according to the target neural network structure after intensive training.

[0210] Specifically, if the searched target neural network structure is one, then the target neural network structure is directly subjected to intensive training based on the training data set corresponding to the media task to obtain the target neural network structure after intensive training, and the media task is processed according to the target neural network after intensive training; if the searched target neural network structures are two or more, then each target neural network structure is trained separately based on the training data set corresponding to the media task to obtain each target neural network structure after intensive training, and each target neural network structure after intensive training is tested according to the test data of the media task to obtain a third test result, and then the optimal target neural network structure after intensive training is selected according to the third test result, and the media task is processed according to the selected target neural network structure after intensive training.

[0211] In the above embodiment, the computer device obtains a training data set corresponding to the media task and performs intensive training on the target neural network structure based on the training data set, thereby obtaining a target neural network structure after intensive training that better meets the requirements corresponding to the media task. Further, when processing the media task according to the target neural network structure after intensive training, better task processing effects can be achieved.

[0212] In one embodiment, the process of a computer device processing the media task according to the target neural network structure after intensive training includes the following steps: obtaining the data to be processed corresponding to the media task; inputting the data to be processed into the target neural network structure after intensive training; and processing the data to be processed layer by layer through each network layer of the target neural network structure after intensive training to obtain a processing result.

[0213] The data to be processed includes any one of image data, voice data and text data.

[0214] For example, if the target neural network structure obtained after enhanced training is a neural network structure for processing image classification tasks, then after the computer device obtains the image to be classified, it inputs the image to be classified into the target neural network structure after enhanced training, and performs feature extraction on the input image to be classified through the corresponding network layer of the target neural network structure after enhanced training to obtain image features, and analyzes the image features through the classification network layer and prediction network layer of the target neural network structure after training, thereby outputting the category of the target object in the image.

[0215] In the above embodiment, the computer device obtains the data to be processed corresponding to the media task, inputs the data to be processed into the target neural network structure after intensive training, and processes the data to be processed layer by layer through each network layer of the target neural network structure after intensive training to obtain the processing results, thereby improving the accuracy of data processing.

[0216] In one embodiment, Figure 11 As shown, a neural network structure search method is also provided, which is applied to Figure 1 The following steps are used as an example to illustrate the computer device (terminal or server) in the example:

[0217] S1102, obtaining the receptive field and parameter quantity of each network layer in the neural network structure in the neural network space.

[0218] S1104: Determine a first eigenvector based on the receptive field of each network layer in the neural network structure.

[0219] S1106, determining a second eigenvector based on the parameter quantity of each network layer in the neural network structure.

[0220] S1108: Construct a feature vector of a neural network structure based on the first feature vector and the second feature vector.

[0221] S1110, sampling from the neural network structure to obtain at least two neural network structure samples.

[0222] S1112: Use the neural network structure sample as a cluster center, and determine a first distance between other neural network structures and the cluster center based on the feature vector.

[0223] S1114: Determine the category to which the other neural network structures belong based on the first distance to obtain an initial category.

[0224] S1116, determining the centroid in the neural network structure of the initial category, and determining a second distance between the neural network structure outside the centroid and the centroid, determining the category to which the neural network structure outside the centroid belongs based on the second distance, and obtaining the category to which each neural network structure belongs.

[0225] S1118, respectively grouping the neural network structures in each category into a supernet for training.

[0226] S1120: After the training is completed, the neural network structure in each category is tested based on the test set to obtain a first test result.

[0227] S1122, among the neural network structures in each category, select a neural network structure to be searched based on the first test result.

[0228] S1124, constructing a target neural network search space based on the selected neural network structure to be searched.

[0229] S1126, in response to the network search request of the media task, the neural network structures to be searched in the target neural network space are organized into a target supernet, and the target supernet is trained based on the training data of the media task.

[0230] S1128, testing the neural network structure to be searched in the target neural network search space according to the test data of the media task to obtain a second test result.

[0231] S1130 , searching for a target neural network structure for processing the media task based on the second test result from the trained target neural network search space.

[0232] This application also provides an application scenario, which applies the above-mentioned neural network structure search method. Specifically, the application of the neural network structure search method in this application scenario is as follows:

[0233] The initial neural network search space is the neural network search space corresponding to the mobilenetv2 model shown in Table 1. Based on this neural network search space, the corresponding initial super network is defined. Specifically, Figure 12 An initial super-network is defined in the manner shown. For each initial sub-network in the initial super-network, the receptive field and parameter quantity of each block of the initial sub-network are calculated respectively, and the feature vector of each initial sub-network is determined based on the receptive field and parameter quantity of each block. Then, based on the feature vector of each initial sub-network, the initial sub-network is clustered using the K-means clustering algorithm to obtain the category to which each initial sub-network belongs. From the initial sub-networks contained in each category, a one-shot neural network architecture search method is used to select a preset number of sub-networks to be searched. A target neural network search space is constructed based on the selected sub-networks to be searched, and a target super-network corresponding to the target neural network search space is constructed. Then, based on the network search request of the image classification task, the target super-network is trained using image sample data, and based on the training results, a target neural network structure for image classification is searched in the target super-network, and the searched target neural network structure is used to process the image to be classified.

[0234] It should be understood that although the steps in the flowcharts 2, 7-11 are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Figure 2 、 7-At least part of the steps in 11 may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0235] In one embodiment, Figure 13 As shown, a neural network structure search device is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a vector acquisition module 1302, a clustering module 1304, a structure selection module 1306, a search space construction module 1308 and a structure search module 1310, wherein:

[0236] The vector acquisition module 1302 is used to obtain the feature vector of the neural network structure in the neural network search space.

[0237] The clustering module 1304 is used to cluster the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs.

[0238] The structure selection module 1306 is used to select the neural network structure to be searched from each category of neural network structures.

[0239] The search space construction module 1308 is used to construct a target neural network search space based on the selected neural network structure to be searched.

[0240] The structure search module 1310 is used to search for the target neural network structure for processing the media task in the trained target neural network search space in response to the network search request of the media task after the training of the neural network structure to be searched in the target neural network search space is completed based on the training data of the media task.

[0241] The above-mentioned neural network structure search device obtains the characteristic vectors of the neural network structure in the neural network search space, clusters the obtained neural network structures based on the characteristic vectors, obtains the category to which each neural network structure belongs, and selects the neural network structure to be searched from the neural network structures of each category, and constructs the target neural network search space based on the selected neural network structure to be searched. It can realize the cropping of the original neural network search space to obtain the target neural network search space, and then when receiving the network search request of the media task, it can directly train the neural network structure to be searched in the target neural network search space based on the training data of the media task, and then search for the target neural network structure for processing the media task in the trained target neural network search space, saving training time and computing resources used for search, thereby improving the search efficiency of the neural network structure.

[0242] In one embodiment, the vector acquisition module 1302 is further used to: obtain the receptive field and parameter value of each network layer in the neural network structure in the neural network space; and determine the characteristic vector of the neural network structure based on the receptive field and parameter value of the neural network structure.

[0243] In one embodiment, the vector acquisition module 1302 is further used to: determine a first eigenvector based on the receptive field of each network layer in the neural network structure; determine a second eigenvector based on the parameter amount of each network layer in the neural network structure; and construct a eigenvector of the neural network structure based on the first eigenvector and the second eigenvector.

[0244] In one embodiment, the clustering module 1304 is further used to: sample from the neural network structure to obtain at least two neural network structure samples; use the neural network structure sample as the cluster center, and determine a first distance between other neural network structures and the cluster center based on the feature vector; determine the category to which the other neural network structures belong based on the first distance to obtain an initial category; determine the center of mass in the neural network structure of the initial category, and determine a second distance between the neural network structures outside the center of mass and the center of mass, determine the category to which the neural network structures outside the center of mass belong based on the second distance, and obtain the category to which each neural network structure belongs.

[0245] In one embodiment, the structure selection module 1306 is further used to: respectively organize the neural network structures in each category into a supernet for training; after the training is completed, test the neural network structures in each category based on the test set to obtain a first test result; and select the neural network structure to be searched among the neural network structures in each category based on the first test result.

[0246] In one embodiment, Figure 14 As shown, the device also includes:

[0247] A model training module 1312 is configured to form a target supernet from the neural network structures to be searched in the target neural network space, and to train the target supernet based on the training data of the media task;

[0248] The structure search module 1310 is also used to: respond to the network search request of the media task, test the neural network structure to be searched in the target neural network search space according to the test data of the media task, and obtain a second test result; and search for the target neural network structure for processing the media task from the trained target neural network search space based on the second test result.

[0249] In one embodiment, the model training module 1312 is further configured to randomly select a neural network structure to be searched from the target supernet each time, and train the selected neural network structure to be searched based on the training data of the media task until the number of selections reaches a threshold or the target supernet converges, and the training is stopped.

[0250] In one embodiment, Figure 14 As shown, the device also includes:

[0251] The reinforcement training module 1314 is used to obtain a training data set corresponding to the media task; and perform reinforcement training on the target neural network structure based on the training data set;

[0252] The task processing module 1316 is used to process media tasks according to the target neural network structure after intensive training.

[0253] In one embodiment, the task processing module 1316 is also used to: obtain the data to be processed corresponding to the media task; the data to be processed includes any one of image data, voice data and text data; input the data to be processed into the target neural network structure after intensive training; and process the data to be processed layer by layer through each network layer of the target neural network structure after intensive training to obtain a processing result.

[0254] In one embodiment, the media task includes any one of an image processing task, a speech processing task, and a text processing task, and the training data of the media task includes any one of image sample data, speech sample data, and text sample data.

[0255] The specific definition of the neural network structure search device can be found in the definition of the neural network structure search method above, and will not be repeated here. The various modules in the above-mentioned neural network structure search device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.

[0256] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 15 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store neural network structure data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a neural network structure search method is implemented.

[0257] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 16 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a neural network structure search method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0258] Those skilled in the art will understand that Figure 15 or Figure 16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0259] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0260] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0261] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0262] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0263] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0264] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A neural network structure search method, characterized in that: The method comprises: Obtain the feature vector of the neural network structure in the neural network search space; Clustering the obtained neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs; Selecting a neural network structure to be searched from the neural network structures of each category; Constructing a target neural network search space based on the selected neural network structure to be searched; After the training of the neural network structure to be searched in the target neural network search space based on the training data of the media task is completed, in response to the network search request of the media task, the target neural network structure for processing the media task is searched in the trained target neural network search space.

2. The method according to claim 1, characterized in that The step of obtaining a feature vector of a neural network structure in a neural network search space includes: Obtain the receptive field and parameter quantity of each network layer in the neural network structure in the neural network space; Determine a feature vector of the neural network structure based on the receptive field and parameter quantity of the neural network structure.

3. The method according to claim 2, characterized in that The step of calculating the characteristic vector of the neural network structure according to the receptive field and parameter quantity of the neural network structure includes: Determining a first eigenvector according to the receptive field of each network layer in the neural network structure; Determining a second eigenvector based on a parameter value of each network layer in the neural network structure; Based on the first eigenvector and the second eigenvector, a eigenvector of the neural network structure is constructed.

4. The method according to claim 1, wherein Clustering the acquired neural network structures based on the feature vectors to obtain the category to which each neural network structure belongs includes: Sampling from the neural network structure to obtain at least two neural network structure samples; Taking the neural network structure sample as a cluster center, and determining a first distance between other neural network structures and the cluster center based on the feature vector; Determine the category to which the other neural network structures belong based on the first distance to obtain an initial category; The center of mass is determined in the neural network structure of the initial category, and a second distance between the neural network structure outside the center of mass and the center of mass is determined. Based on the second distance, the category to which the neural network structure outside the center of mass belongs is determined, and the category to which each of the neural network structures belongs is obtained.

5. The method according to claim 1, wherein The step of selecting a neural network structure to be searched from the neural network structures of each category includes: The neural network structures in each category are respectively formed into a supernet for training; After the training is completed, the neural network structure in each category is tested based on the test set to obtain a first test result; Among the neural network structures in each of the categories, a neural network structure to be searched is selected based on the first test result.

6. The method according to claim 1, characterized in that The method further comprises: The neural network structures to be searched in the target neural network space are combined into a target supernet, and the target supernet is trained based on the training data of the media task; The step of searching, in response to the network search request for the media task, a target neural network structure for processing the media task in the trained target neural network search space comprises: In response to the network search request of the media task, testing the neural network structure to be searched in the target neural network search space according to the test data of the media task to obtain a second test result; From the trained target neural network search space, search for a target neural network structure for processing the media task based on the second test result.

7. The method according to claim 6, characterized in that The training of the target supernet based on the training data of the media task includes: Each time, a neural network structure to be searched is randomly selected from the target supernet, and the selected neural network structure to be searched is trained based on the training data of the media task until the number of selections reaches a threshold or the target supernet converges, and the training is stopped.

8. The method according to any one of claims 1 to 7, characterized in that After searching the trained target neural network search space for a target neural network structure for processing the media task, the method further includes: Obtaining a training data set corresponding to the media task; Performing intensive training on the target neural network structure based on the training data set; The media task is processed according to the target neural network structure after reinforcement training.

9. The method according to claim 8, characterized in that Processing the media task according to the target neural network structure after intensive training includes: Acquire data to be processed corresponding to the media task; the data to be processed includes any one of image data, voice data and text data; Inputting the data to be processed into the target neural network structure after the intensive training; The data to be processed is processed layer by layer through each network layer of the target neural network structure after the intensive training to obtain a processing result.

10. The method according to any one of claims 1 to 7, characterized in that The media task includes any one of an image processing task, a speech processing task, and a text processing task, and the training data of the media task includes any one of image sample data, speech sample data, and text sample data.

11. A neural network structure search device, characterized in that: The device comprises: A vector acquisition module is used to obtain the characteristic vector of the neural network structure in the neural network search space; A clustering module, configured to cluster the obtained neural network structures based on the feature vectors to obtain a category to which each neural network structure belongs; A structure selection module, configured to select a neural network structure to be searched from the neural network structures of each category; A search space construction module, configured to construct a target neural network search space based on the selected neural network structure to be searched; A structure search module is used to search for a target neural network structure for processing the media task in the trained target neural network search space in response to a network search request of the media task after the training of the neural network structure to be searched in the target neural network search space is completed based on the training data of the media task.

12. The device according to claim 11, characterized in that The vector acquisition module is further used for: Obtain the receptive field and parameter quantity of each network layer in the neural network structure in the neural network space; Determine a feature vector of the neural network structure based on the receptive field and parameter quantity of the neural network structure.

13. The device according to claim 12, characterized in that The vector acquisition module is further used for: Determining a first eigenvector according to the receptive field of each network layer in the neural network structure; Determining a second eigenvector based on a parameter value of each network layer in the neural network structure; Based on the first eigenvector and the second eigenvector, a eigenvector of the neural network structure is constructed.

14. The device according to claim 11, characterized in that The clustering module is also used to: Sampling from the neural network structure to obtain at least two neural network structure samples; Taking the neural network structure sample as a cluster center, and determining a first distance between other neural network structures and the cluster center based on the feature vector; Determine the category to which the other neural network structures belong based on the first distance to obtain an initial category; The center of mass is determined in the neural network structure of the initial category, and a second distance between the neural network structure outside the center of mass and the center of mass is determined. Based on the second distance, the category to which the neural network structure outside the center of mass belongs is determined, and the category to which each of the neural network structures belongs is obtained.

15. The device according to claim 11, characterized in that The structure selection module is also used for: The neural network structures in each category are respectively formed into a supernet for training; After the training is completed, the neural network structure in each category is tested based on the test set to obtain a first test result; Among the neural network structures in each of the categories, a neural network structure to be searched is selected based on the first test result.

16. The device according to claim 11, characterized in that The device further comprises: a model training module, configured to form a target supernet from the neural network structures to be searched in the target neural network space, and to train the target supernet based on the training data of the media task; The structure search module is also used to: In response to the network search request of the media task, testing the neural network structure to be searched in the target neural network search space according to the test data of the media task to obtain a second test result; From the trained target neural network search space, search for a target neural network structure for processing the media task based on the second test result.

17. The device according to claim 16, characterized in that The model training module is also used to: Each time, a neural network structure to be searched is randomly selected from the target supernet, and the selected neural network structure to be searched is trained based on the training data of the media task until the number of selections reaches a threshold or the target supernet converges, and the training is stopped.

18. The device according to any one of claims 11 to 17, characterized in that After searching the trained target neural network search space for a target neural network structure for processing the media task, the apparatus further includes: An intensive training module is used to obtain a training data set corresponding to the media task; and perform intensive training on the target neural network structure based on the training data set; The task processing module is used to process the media task according to the target neural network structure after intensive training.

19. The device according to claim 18, characterized in that The task processing module is also used for: Acquire data to be processed corresponding to the media task; the data to be processed includes any one of image data, voice data and text data; Inputting the data to be processed into the target neural network structure after the intensive training; The data to be processed is processed layer by layer through each network layer of the target neural network structure after the intensive training to obtain a processing result.

20. The device according to any one of claims 11 to 17, characterized in that The media task includes any one of an image processing task, a speech processing task, and a text processing task, and the training data of the media task includes any one of image sample data, speech sample data, and text sample data.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

23. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images