A neural network architecture search method, device, storage medium and program product

Abstract

The present disclosure relates to a neural network architecture search method, device, storage medium and program product. The neural network architecture search method comprises: determining an image processing task, and determining a network architecture search space and a candidate operation search space for image processing of the image processing task; creating a tree structure based on the network architecture search space and the candidate operation search space, and decomposing the network architecture search space and the candidate operation search space based on the tree structure to obtain a neural network architecture of the image processing task. The present disclosure can reduce the search complexity of the neural network architecture.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and particularly relates to a neural network architecture search method, device, storage medium and program product. BACKGROUND

[0002] In recent years, deep neural networks have made breakthrough progress in computer vision and natural language processing tasks, thus attracting more and more artificial intelligence (AI) researchers to design model structures with better performance. Among them, automated neural architecture search (NAS) can search the model structure with the best performance under a given amount of computing power.

[0003] Currently, in the processing process of a data processing model, neural network architecture search can be performed through NAS technology. In order to improve the model structure search efficiency of the neural network, one-shot NAS and gradient-based NAS methods are proposed in related technologies. One-shot NAS uses a weight sharing strategy, which is similar to training a super network containing all candidate architectures. Each candidate architecture directly inherits the weight from the super network and does not need further training, which can improve the neural network architecture search efficiency, but will cause architecture evaluation degradation and damage search accuracy. In the gradient-based NAS, architecture parameters are introduced, which are trained jointly with network weights through gradient descent, so that NAS can be trained end-to-end, but the architecture parameters cannot truly reflect the strength of each candidate operation. Therefore, the current NAS method for neural network architecture search of a data processing model has high search complexity. SUMMARY

[0004] To overcome the problems in related technologies, the present disclosure provides a neural network architecture search method, device, storage medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a neural network architecture search method is provided, comprising:

[0006] determining an image processing task, and determining a network architecture search space and a candidate operation search space for image processing of the image processing task; creating a tree structure based on the network architecture search space and the candidate operation search space, and decomposing the network architecture search space and the candidate operation search space based on the tree structure to obtain a neural network architecture of the image processing task.

[0007] In an implementation, the creating a tree structure based on the network architecture search space and the candidate operation search space, and decomposing the network architecture search space and the candidate operation search space based on the tree structure comprises:

[0008] The first tree structure is created based on the network architecture search space, a root node of the first tree structure is a hypernet structure with an edge number being a number of layers of the network architecture search space and each edge corresponding to a path number of the network architecture search space, a depth of the first tree structure is the number of layers, and a leaf node of the first tree structure is a single-path structure; a target single path is selected based on the first tree structure; a candidate operation search space corresponding to the target single path is decomposed into a second tree structure, a root node of the second tree structure corresponds to all candidate operations in the candidate operation search space, child nodes at a same level correspond to mutually exclusive operation groups, and a leaf node of the second tree structure is a single candidate operation; and a target operation corresponding to the target single path is selected based on the second tree structure.

[0009] In an implementation, the creating a first tree structure based on the network architecture search space comprises:

[0010] Each architecture layer in the network architecture search space is mapped into a node, and the nodes are connected through directed edges according to input-output relationships between the architecture layers in the network architecture search space, to obtain a tree structure with a depth being the number of layers; wherein a starting node of the directed edge corresponds to an input architecture in the network architecture search space, a terminal node corresponds to an output architecture in the network architecture search space, and a number of directed edges between the nodes is a path number of the network architecture search space; the directed edges corresponding to the nodes mapped by each architecture layer in the network architecture search space are sequentially split in an order from small to large in depth, so that each node corresponds to a number of child nodes of the directed edges, until leaf nodes corresponding to single-path structures are reached.

[0011] In an implementation, the selecting a target single path based on the first tree structure comprises:

[0012] In the first tree structure, a root node with a smallest depth is expanded to a first depth; each node at the first depth is scored, and a node with a highest score is selected as a root node; the above process is repeatedly performed until a leaf node with a highest score is determined, and a single path corresponding to the leaf node with the highest score is selected as a target single path.

[0013] In an implementation, the decomposing a candidate operation search space corresponding to the target single path into a second tree structure comprises:

[0014] Map all candidate operations in the candidate operation search space corresponding to the target single path to root nodes; branch all candidate operations corresponding to the root nodes into two mutually exclusive operation groups; map the candidate operations in the mutually exclusive operation groups to root nodes respectively, and repeatedly perform the step of branching all candidate operations corresponding to the root nodes into two mutually exclusive operation groups until a leaf node of a single candidate operation is obtained.

[0015] In an implementation form, the selecting the target operation corresponding to the target single path based on the second tree structure comprises:

[0016] In the second tree structure, expand from the root node of the minimum depth to a second depth; score each node at the second depth, and select the node with the highest score as the root node; repeatedly perform the above process until the leaf node with the highest score is determined, and the single operation corresponding to the leaf node with the highest score is selected as the target operation corresponding to the target single path.

[0017] In an implementation form, the scoring each node at the first depth comprises:

[0018] For each node included in the first depth, determine the subnets corresponding to all single paths mapped by the node respectively, and sum the scores of the subnets corresponding to all single paths to obtain the score of the node.

[0019] In an implementation form, the scoring each node at the second depth comprises:

[0020] For each node included in the second depth, determine the operation group mapped by the node respectively, and score the operation group as a whole to obtain the score of the node.

[0021] According to a second aspect of the embodiments of the present disclosure, a neural network architecture search device is provided, comprising:

[0022] A determining unit is configured to determine an image processing task, and determine a network architecture search space and a candidate operation search space for image processing of the image processing task; a searching unit is configured to create a tree structure based on the network architecture search space and the candidate operation search space, and decompose the network architecture search space and the candidate operation search space based on the tree structure to obtain a neural network architecture of the image processing task.

[0023] In an implementation form, the searching unit creates a tree structure based on the network architecture search space and the candidate operation search space in the following manner, and decomposes the network architecture search space and the candidate operation search space based on the tree structure:

[0024] Based on the network architecture search space, a first tree structure is created, a root node of the first tree structure is a super network structure with an edge number being a number of layers corresponding to the network architecture search space, each edge corresponds to a path number of the network architecture search space, a depth of the first tree structure is the number of layers, and a leaf node of the first tree structure is a single path structure; based on the first tree structure, a target single path is selected; a candidate operation search space corresponding to the target single path is decomposed into a second tree structure, a root node of the second tree structure corresponds to all candidate operations in the candidate operation search space, child nodes at a same level correspond to mutually exclusive operation groups, and a leaf node of the second tree structure is a single candidate operation; and based on the second tree structure, a target operation corresponding to the target single path is selected.

[0025] In an implementation, the searching unit creates the first tree structure based on the network architecture search space in the following manner:

[0026] Each architecture layer in the network architecture search space is mapped into a node, and the nodes are connected through directed edges according to input-output relationships between the architecture layers in the network architecture search space, so as to obtain a tree structure with a depth being the number of layers; wherein a starting node of the directed edge corresponds to an input architecture in the network architecture search space, a terminal node corresponds to an output architecture in the network architecture search space, and a number of directed edges between the nodes is a path number of the network architecture search space; the directed edges corresponding to the nodes mapped by each architecture layer in the network architecture search space are split in an order from small to large depth, so that each node corresponds to a number of child nodes of the directed edges, until leaf nodes of a single path structure are reached.

[0027] In an implementation, the searching unit selects the target single path based on the first tree structure in the following manner:

[0028] In the first tree structure, a root node with a smallest depth is expanded to a first depth; each node at the first depth is scored, and a node with a highest score is selected as a root node; the above process is repeatedly executed until a leaf node with a highest score is determined, and a single path corresponding to the leaf node with the highest score is selected as the target single path.

[0029] In an implementation, the searching unit decomposes the candidate operation search space corresponding to the target single path into the second tree structure in the following manner:

[0030] Map all candidate operations in the candidate operation search space corresponding to the target single path to root nodes; branch all candidate operations corresponding to the root nodes into two mutually exclusive operation groups; map the candidate operations in the mutually exclusive operation groups to root nodes respectively, and repeatedly perform the step of branching all candidate operations corresponding to the root nodes into two mutually exclusive operation groups until a leaf node of a single candidate operation is obtained.

[0031] In an implementation form, the searching unit selects the target operation corresponding to the target single path based on the second tree structure in the following manner:

[0032] In the second tree structure, expand from the root node of the minimum depth to the second depth; score each node at the second depth, and select the node with the highest score as the root node; repeatedly perform the above process until the leaf node with the highest score is determined, and the single operation corresponding to the leaf node with the highest score is selected as the target operation corresponding to the target single path.

[0033] In an implementation form, the searching unit scores each node at the first depth in the following manner:

[0034] For each node included at the first depth, determine the subnets corresponding to all single paths mapped by the node respectively, and sum the scores of the subnets corresponding to all single paths to obtain the score of the node.

[0035] In an implementation form, the searching unit scores each node at the second depth in the following manner:

[0036] For each node included at the second depth, determine the operation group mapped by the node respectively, score the operation group as a whole to obtain the score of the node.

[0037] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the method in the first aspect or any one of the implementation forms of the first aspect.

[0038] According to a fourth aspect of embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program / instruction, and the computer program / instruction is executed by a processor to implement the method in the first aspect or any one of the implementation forms of the first aspect.

[0039] According to a fifth aspect of embodiments of the present disclosure, a computer program product is provided, and the computer program product includes a computer program, and the computer program is used to implement the method in the first aspect or any one of the implementation forms of the first aspect when executed by a processor.

[0040] The technical scheme provided by the embodiments of the present disclosure can include the following beneficial effects: the network architecture search space and the candidate operation search space for image processing of an image processing task are created into a tree structure, and the network architecture search space and the candidate operation search space are decomposed based on the tree structure to obtain a neural network architecture of the image processing task, and neural network architecture search is performed. Since the tree structure has a depth, the depth of search on the tree structure can be determined based on the precision of neural network architecture search, and neural network architecture search is performed based on the depth, thereby realizing partial search of the network architecture search space and the candidate operation search space and reducing search complexity.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0043] Figure 1 is a flowchart of a neural network architecture search method according to an exemplary embodiment.

[0044] Figure 2 is a flowchart of creating a first tree structure based on a network architecture search space according to an exemplary embodiment.

[0045] Figure 3 shows a first tree structure diagram shown in an exemplary embodiment of the present disclosure.

[0046] Figure 4 d a is a process diagram for network architecture search space decomposition when d

[0047] Figure 5 d a is a process diagram for network architecture search space decomposition when d

[0048] Figure 6 is a flowchart of creating a second tree structure based on each candidate operation in a candidate operation search space according to an exemplary embodiment.

[0049] Figure 7 shows a second tree structure diagram shown in an exemplary embodiment of the present disclosure.

[0050] Figure 8A process diagram of searching a neural network architecture in combination with a first tree structure and a second tree structure is shown in an example embodiment of the present disclosure.

[0051] Figure 9 A block diagram of a neural network architecture searching device is shown according to an example embodiment.

[0052] Figure 10 A block diagram of an electronic device for neural network architecture searching is shown according to an example embodiment. DETAILED DESCRIPTION

[0053] The example embodiments will be described in detail herein with reference to the attached drawings. The following description is with reference to the drawings, in which like numerals represent like elements or functions throughout the several figures. The implementation described in the following example embodiments is not meant to represent all implementations consistent with the present disclosure.

[0054] In the drawings, like reference numerals in the different drawings denote like or similar elements among the several figures. The described embodiments are only a part of the embodiments consistent with the present disclosure, not all of the embodiments. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present disclosure, and should not be understood as a limitation of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative labor are within the scope of protection of the present disclosure. The embodiments of the present disclosure are described in detail below with reference to the drawings.

[0055] In recent years, important progress has been made in the research of computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence. Artificial intelligence (AI) is a new science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks and many other technology categories. Computer vision, as an important branch of artificial intelligence, is specifically about letting machines recognize the world. Computer vision technology usually includes face recognition, live detection, fingerprint recognition and anti-forgery verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, behavior recognition, three-dimensional reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone images, cloud services, smart home, wearable devices, unmanned vehicles, autonomous driving, intelligent medical care, face payment, face unlocking, fingerprint unlocking, face and certificate verification, smart screen, smart TV, camera, mobile Internet, network live broadcast, beauty, makeup, medical cosmetology, intelligent temperature measurement and other fields.

[0056] Among them, neural network architecture search (NAS) for data processing has attracted more and more interest in academia and industry due to its ability to find high-performance neural network architectures with minimal human intervention. Among them, the data processing involved in the embodiments of the present disclosure can be the data processing process involved in the fields to which the above-mentioned artificial intelligence is applied. Further, neural network architecture search mainly automatically explores the search space through reinforcement learning or evolutionary algorithm. However, when exploring the search space, a large number of architectures need to be trained to achieve better accuracy, which consumes a lot of resources, which will limit the application of neural network architecture search in users with limited computing resources. Recent one-shot network architecture search (one-shot NAS) improves the efficiency of NAS. One-shot NAS uses a weight sharing strategy, which is similar to training a super network containing all candidate architectures. Each candidate architecture directly inherits the weight from the super network and does not need further training. In the one-shot NAS method, gradient-based NAS (such as DARTS) introduces architecture parameters. The architecture parameters are trained jointly with the network weights through gradient descent, so that NAS can be trained end-to-end. Although the efficiency of the one-shot NAS algorithm is very high, it will cause the architecture evaluation to degenerate and damage the search accuracy. The gradient-based NAS method has another disadvantage, that is, the architecture parameters cannot truly reflect the strength of each candidate operation. Therefore, how to reduce the processing process of NAS and reduce the search complexity is a problem to be solved.

[0057] Therefore, the embodiments of the present disclosure provide a neural network architecture search method for data processing, create a tree structure based on a network architecture search space and a candidate operation space corresponding to a neural network architecture, and perform neural network architecture search based on the tree structure to reduce search complexity.

[0058] Figure 1 is a flowchart of a neural network architecture search method according to an exemplary embodiment, as shown in Figure 1 The neural network architecture search method includes the following steps.

[0059] In step S11, an image processing task is determined, and a network architecture search space and a candidate operation search space for image processing of the image processing task are determined.

[0060] In step S12, a tree structure is created based on the network architecture search space and the candidate operation search space, and the network architecture search space and the candidate operation search space are decomposed based on the tree structure to obtain a neural network architecture for the image processing task.

[0061] The embodiment of the present disclosure provides a neural network architecture search method for data processing. A tree structure is created based on a network architecture search space and a candidate operation space for image processing of an image processing task, and the network architecture search space and the candidate operation search space are decomposed based on the tree structure to realize neural network architecture search based on the tree structure. Since the tree structure has a depth, the depth of search on the tree structure can be determined based on the accuracy of neural network architecture search, and neural network architecture search is performed based on the depth, thereby realizing partial search of the network architecture search space and the candidate operation space and reducing search complexity.

[0062] In an embodiment, a tree structure of a matching search space architecture can be constructed based on the network architecture search space in the embodiment of the present disclosure, which is referred to as a first tree structure. The first tree structure can be understood as an architecture tree created based on network architecture layers of the network architecture search space, and the architecture tree can reflect each network architecture layer involved in the network architecture search space. Each node in the first tree structure represents an architecture layer in the network architecture search space. The first tree structure and the architecture tree are sometimes used alternately in the following embodiments of the present disclosure, and a person skilled in the art should understand the consistency of their meanings.

[0063] In an example embodiment of the present disclosure, the architecture layers included in the network architecture search space and the input-output relationship between the architecture layers can be determined, and the number of paths of the network architecture search space can be determined. Based on the number of architecture layers and the number of paths, the first tree structure is created.

[0064] Figure 2 FIG. 1 is a flowchart illustrating a process of creating a first tree structure based on a network architecture search space according to an example embodiment. Referring to FIG. 1, the process of creating the first tree structure based on the network architecture search space includes the following steps. Figure 2 As shown in FIG. 1, the process of creating the first tree structure based on the network architecture search space includes the following steps.

[0065] In step S21, each architecture layer in the network architecture search space is mapped to a node.

[0066] In step S22, the nodes are connected by directed edges according to the input-output relationship between the architecture layers in the network architecture search space, and a tree structure with a depth of the number of architecture layers is obtained.

[0067] The starting node of the directed edge corresponds to the input architecture in the network architecture search space, the terminal node corresponds to the output architecture in the network architecture search space, and the number of directed edges between the nodes is the number of paths of the network architecture search space.

[0068] In step S23, the directed edges corresponding to the nodes mapped by each architecture layer in the network architecture search space are sequentially split in order of depth from small to large, so that each node corresponds to a number of directed edges. The number of directed edges is obtained by each node, until the leaf node corresponding to the single path structure is reached.

[0069] In the embodiments of the present disclosure, by decomposing each architecture layer in the network architecture search space into a first tree structure, a tree structure matching the network architecture search space can be obtained, and then the search space can be searched based on the first tree structure.

[0070] In the embodiments of the present disclosure, the root node of the first tree structure created based on the network architecture search space is a hypernetwork structure with edge number corresponding to the layer number of the network architecture search space, and each edge corresponds to the path number of the network architecture search space. The depth of the first tree structure is the layer number, and the leaf node of the first tree structure is a single path structure.

[0071] In an example, the embodiments of the present disclosure take the architecture layer (layer) number of the network architecture search space as 3 and the path (path) as 2 as an example to exemplarily illustrate the process of constructing the first architecture tree. Wherein, L represents the architecture layer number, and M represents the path. Therefore, the network structure to be searched in the embodiments of the present disclosure can be understood as a search space with L=3 and M=2. Each architecture layer in the network architecture layer is mapped to a node, and it is assumed that the mapped nodes are represented by 0, 1 and 2. Wherein, node 0 is an input architecture, node 1 is an output architecture of node 0, and node 2 is an output architecture of node 0 and node 1. The node 0, node 1 and node 2 are connected by a directed edge, and a hypernetwork architecture containing edges and nodes can be obtained. Figure 3 A first tree structure diagram shown in an example embodiment of the present disclosure is shown. Referring to Figure 3 As shown, the network architecture to be searched has L=3 edges, and each edge has M=2 candidate operations. Therefore, the root node of the architecture tree is a hypernetwork architecture with L edges, and each edge has M paths, wherein each path represents a different operation. In other words, the nodes in the architecture tree can be connected by directed edges corresponding to the number of candidate operations or the number of paths. In the embodiments of the present disclosure, the number of candidate operations is M=2, and the node 0, node 1 and node 2 are connected by 2 directed edges.

[0072] Further, the depth of the architecture tree in the embodiments of the present disclosure is L=3. For each node at depth i, the architecture tree splits the M paths in the i-th edge of the architecture represented by these nodes, and obtains M child nodes at each node. The architecture tree repeats this branching operation for each node until the leaf node is reached. Referring to Figure 3 As shown, each leaf node represents a single path structure, and different leaf nodes correspond to different single path structures. The union of the leaf nodes is a set containing all candidate architectures.

[0073] In the embodiments of the present disclosure, after the first tree structure is created based on the network architecture search space, the selection of the target single path can be performed based on the first tree structure, so as to realize the architecture search of the search space.

[0074] In the embodiments of the present disclosure, when the target single path is selected, the depth of search in the first tree structure can be determined based on the search precision, which can also be referred to as an expansion depth, and is referred to as a first depth hereinafter. The first depth can be understood as a kind of hyperparameter, and is denoted by d a , where the subscript a represents the architecture.

[0075] In the embodiments of the present disclosure, d a may be a pre-set value. The value of d a may be selected between 1 and L, and the minimum value is 1 and the maximum value is L. It can be understood that the smaller the value of d a , the smaller the search complexity, and the larger the value of d a , the larger the search complexity.

[0076] In an implementation manner, based on the first tree structure, when the target single path is selected, the first depth is determined, and in the first tree structure, the root node with the minimum depth is expanded to the first depth. Each node at the first depth is scored, and the node with the highest score is selected as the root node. The above process is repeatedly performed until the leaf node with the highest score is determined, and the single path corresponding to the leaf node with the highest score is selected as the target single path.

[0077] Continuing from the example shown in the above Figure 3 , the root node with the depth of 0 is expanded to all its successors until the depth of d da , and after the expansion, at most M da leaf nodes are obtained. The subnets defined by the M da leaf nodes are scored, and the node with the highest score is selected as the root node in the next step. The above process is defined as a decision step in the BFS, and the decision step is repeatedly performed until the single path architecture is determined. d a represents how many network layers are branched at each decision step.

[0078] In the embodiments of the present disclosure, the above search process of the network architecture search space can be understood as a process of completing the architecture search in a breadth-first search (BFS) manner. The BFS needs a hyperparameter d a . d a represents how many network layers are branched at each decision step. The BFS starts from the root node with the depth of 0, expands all its successors until the depth of d a , and after the expansion, at most M daThere are leaf nodes. BFS scores the subnets defined by these leaf nodes and selects the node with the highest score as the root node for the next step.

[0079] Figure 4 and Figure 5 The text shows d respectively. a For 1 and d a This is a schematic diagram illustrating the process of network architecture search space decomposition at time 3. (See also...) Figure 4 As shown, d a =1, for Figure 3 In the process of dividing nodes with a depth of 1, edges are split and scored. The edge between node 0 and node 1 is split, and the resulting edges are scored. The edge with the highest score is selected and retained. The same process is repeated for the edges between node 0 and node 2. Finally, the edge between node 1 and node 2 is split, and the resulting edges are scored. The edge with the highest score is selected and retained. After this difference process, the following is formed: Figure 4 The diagram shows the single-path structure from node 0 to node 1 and node 2, as well as the single-path structure from node 1 to node 2, which is the final target single path.

[0080] See Figure 5 As shown, d a If the depth is 3, then the nodes with a depth of 3 are split and scored. In this embodiment, d... a Since 3 corresponds to all leaf nodes, we score all leaf nodes and select the edge with the highest score to keep, resulting in the following: Figure 5 The target single-path structure is shown.

[0081] In this embodiment of the disclosure, when selecting a target single path based on a first tree structure and a first depth, the first tree structure is decomposed into layers according to the first depth, and edge decomposition is performed on the layers to be decomposed. For each node included in the decomposed layer, the subnets corresponding to all single paths mapped to the node are determined, and the sum of the scores of all subnets corresponding to single paths is used as the node's score. That is, the scores of all subnets containing individual edges are determined, and the edge corresponding to the subnet with the highest score is retained. The node corresponding to the edge of the subnet with the highest score is used as the root node for the next step, which is equivalent to omitting the search for other nodes except the node with the highest score, thus reducing the search complexity. Continuing from the above... Figure 3 and Figure 4 The example shown would require 2 steps if using the traditional method. 3 =Search for 8 architectures, but using Figure 4 The example shown only requires searching for two architectures, reducing search complexity.

[0082] In yet another embodiment of the disclosure, a tree structure corresponding to the candidate operation search space can be created based on the candidate operation search space, which is referred to as a second tree structure below. The second tree structure is used to hierarchically group the candidate operations in the candidate operation search space for decomposition of the candidate operation search space. The second tree structure can also be understood as a binary operation tree. In the following embodiments of the disclosure, the second tree structure and the binary operation tree are sometimes used alternately, and those skilled in the art should understand the consistency of their meanings.

[0083] In an example embodiment of the disclosure, a binary operation tree can be constructed based on the mapping of mutually exclusive operation groups for each candidate operation in the candidate operation search space.

[0084] Figure 6 FIG. 3 is a flow diagram illustrating a process of creating a second tree structure based on each candidate operation in a candidate operation search space according to an example embodiment. Referring to FIG. 3, the implementation process of creating the second tree structure includes the following steps. Figure 6 As shown in FIG. 3, the implementation process of creating the second tree structure includes the following steps.

[0085] In step S31, all candidate operations in the candidate operation search space are mapped as root nodes.

[0086] In step S32, all candidate operations corresponding to the root nodes are branched into two mutually exclusive operation groups.

[0087] In step S33, the candidate operations in the mutually exclusive operation groups are mapped as root nodes, respectively, and the step of branching all candidate operations corresponding to the root nodes into two mutually exclusive operation groups is repeated until leaf nodes of single candidate operations are obtained.

[0088] In the embodiments of the disclosure, the candidate operations in the candidate operation search space can be divided into mutually exclusive operation groups, i.e., there is no intersection between the two operation groups. Each operation group can include one or more operations. Each operation group is mapped as a node, i.e., different nodes map different operation groups. In other words, each node in the binary operation tree is an operation group, and an operation group is composed of one or more different operations. The root node of the binary operation tree represents the entire candidate operation search space and contains all operations in the candidate operation search space. The binary operation tree branches from the root node into two child nodes representing two mutually exclusive operation groups. This branching is repeated until the leaf nodes are reached. Each leaf node represents a single operation. The union of the leaf nodes is the entire operation search space.

[0089] Figure 7 FIG. 4 shows a schematic diagram of the second tree structure according to an example embodiment of the disclosure. Figure 7The candidate operation search space includes 5 different operation operations as an example. The root node corresponding to the depth 0 includes all 5 different operation operations in the candidate operation search space. The node corresponding to the depth 1 is mapped into two operation groups with and without operation. The node corresponding to the depth 2 is mapped into two operation groups between the convolution operation and the topology operation for the 4 operation operations in the operation group with operation. The node corresponding to the depth 3 is mapped into 1*1 convolution operation and 3*3 convolution operation for the 2 convolution operations in the convolution operation group. The 2 topology operations in the topology operation group are mapped into residual operation and maximum pooling operation. That is, the leaf node corresponding to the single operation is obtained at the node with the depth of 3. The union of the leaf node corresponding to the single operation at the depth of 1 and the leaf node corresponding to the single operation at the depth of 3 is the entire 5 different operation operations in the candidate operation space.

[0090] In the embodiment of the present disclosure, the candidate operation in the candidate operation search space is decomposed into a second tree structure, and a tree structure matching the candidate operation search space is obtained, and then the selection of the candidate operation is performed based on the second tree structure.

[0091] In an example, the candidate operation search space in the embodiment of the present disclosure can be a candidate operation search space corresponding to a target single path after the architecture search space is searched based on the architecture tree.

[0092] In an implementation, based on the second tree structure, when the candidate operation is selected, the second tree structure can be expanded from the root node with the minimum depth to the second depth. Each node at the second depth is scored, and the node with the highest score is selected as the root node. The above process is repeatedly performed until the leaf node with the highest score is determined, and the single operation corresponding to the leaf node with the highest score is selected as the target operation corresponding to the target single path.

[0093] The process of searching the candidate operation in the candidate operation search space in the embodiment of the present disclosure can also be understood as a process of performing BFS on a binary operation tree. The process of performing BFS on the binary operation tree also needs to define the expansion depth, that is, the definition of the second depth, which is represented by d o below. The BFS starts from the root node with the depth of 0, and all the successor nodes with the depth of d o are expanded. After the expansion, at most 2 do leaf nodes are obtained. These leaf nodes represent the current candidate operation group. The nodes mapping these different operation groups are scored by the BFS, and the node with the highest score is selected as the root node in the next stage. The above process is defined as a search stage, and the search stage is repeatedly performed until the leaf node is selected, that is, the single operation is selected.

[0094] In the embodiments of the present disclosure, when scoring the nodes of different operation groups at a certain depth, the depth corresponding to each node is determined. For each node, the operation group to which the node is mapped is determined, the operation group as a whole is scored to obtain a score value, which is taken as the score value of the node. In other words, the i th edge contains multiple operations, and the output of the layer will be the sum of the outputs of all operations of the layer. For example, Figure 1 Among the 4 operation groups with operations at the depth 1 in the above example, the operation group composed of 4 different operation groups needs to be scored as a whole, and the score value corresponding to the operation group is obtained as the score value of the node.

[0095] In the embodiments of the present disclosure, when searching for a target single operation based on the second depth and the second tree structure, the M operations of a certain edge can be divided into two mutually exclusive operation groups to construct a binary operation tree, each operation group is scored, and the operation group with the highest score value is selected as the operation group reserved in the current layer, and the next stage of mutually exclusive operation group division is performed, and the target single operation is obtained by repeating the execution. The second depth can be a pre-set value. The value of the second depth can be selected between 1 and M, the minimum value is 1, and the maximum value is M. It can be understood that the smaller the value of the second depth in the embodiments of the present disclosure, the smaller the search complexity, and the larger the value of the second depth, the greater the search complexity. Typically, the value of the second depth is 1.

[0096] In the embodiments of the present disclosure, the selection of the target single operation based on the binary operation tree can reduce the operation space of the search. For example, referring to the tree structure shown in Figure 7 If the traditional method is used, the 5 operations need to be searched respectively. If the BFS method provided in the embodiments of the present disclosure is used, the second depth is set to 1, and the two nodes at the depth 1 need to be searched, which reduces the search complexity.

[0097] In the embodiments of the present disclosure, when performing neural network architecture search, the first tree structure and the second tree structure described above can be combined for search. That is, based on the network architecture search space, a first tree structure is created, the root node of the first tree structure is a super network structure with the number of edges being the number of layers of the network architecture search space and each edge corresponding to the number of paths of the network architecture search space, the depth of the first tree structure is the number of layers, and the leaf node of the first tree structure is a single path structure. Based on the first tree structure, a target single path is selected. The candidate operation search space corresponding to the target single path is decomposed into a second tree structure, the root node of the second tree structure corresponds to all candidate operations in the candidate operation search space, the child nodes at the same level correspond to mutually exclusive operation groups, and the leaf node of the second tree structure is a single candidate operation. Based on the second tree structure, a target operation corresponding to the target single path is selected.

[0098] The neural network architecture search method provided by the embodiments of the present disclosure can be understood as a tree-based network architecture search method, which can be referred to as TNAS, the English full name of which is NAS with trees. TNAS introduces an architecture tree and a binary operation tree to respectively decompose the network architecture layer and operation search space. By combining the two trees, the network architecture search space is iteratively branched into two mutually exclusive subspaces. TNAS performs a double-layer breadth-first search (BFS) based on the architecture tree and the binary operation tree. By adjusting the expansion depth of BFS, TNAS explicitly controls the number of network architectures that need to be explored (referred to as exploration size), and can exponentially reduce the exploration size. It is a novel, flexible, accurate and efficient NAS new paradigm.

[0099] Figure 8 A process diagram for searching a neural network architecture in combination with a first tree structure and a second tree structure is shown in an example embodiment of the present disclosure. Among them, Figure 8 M = 5, L = 6, d o = 1, d a = 2 are described. The tree-based network architecture search method (TNAS). Given a search space containing M candidate operations and L edges, TNAS constructs a binary operation tree and an architecture tree. TNAS starts from a super-net architecture with M paths and L edges, and performs a double-layer breadth-first search on the binary operation tree and the architecture tree. Among them, the double-layer breadth-first search on the binary operation tree can be understood as the operation process of the outer loop. In the outer loop, TNAS defaults to expand the depth d o to perform BFS. Each decision stage of the outer loop branches each operation group of the selected layer into two sub-operation groups, and the two sub-operation groups define the operation search space of the inner loop. The outer loop repeats the decision stage until each edge of the architecture reaches the leaf node of the binary operation tree, that is, all edges select a single operation. The process of performing a double-layer breadth-first search on the architecture tree can be understood as the inner loop process. In the inner loop, TNAS performs a BFS with an expansion depth of d a on the architecture tree. Each decision step of the inner loop selects d a edges to branch, obtains at most 2 da subnets, scores each subnet, and then selects the one with the highest score. The selected subnet will be used to replace the super-net and become the starting point (root node) for the next step. The inner loop repeats the above decision step until it selects a leaf node of the architecture tree, that is, all layers of the architecture have determined their operation groups at the current decision stage.

[0100] It can be understood that the neural network architecture search method provided by the embodiment of the present disclosure can be used to create a tree structure for a given image processing task and a series of candidate operations (such as a convolution operation with a convolution kernel of 1, a convolution operation with a convolution kernel of 3, a maximum pooling operation, etc.) of a deep learning network architecture for image processing, to decompose the network architecture search space and the candidate operation search space; based on the network architecture search space and the candidate operation search space, and based on the created tree structure, the neural network architecture is searched to obtain the optimal network architecture for performing the image processing task.

[0101] Based on the same concept, the embodiment of the present disclosure also provides a neural network architecture search device.

[0102] It can be understood that the neural network architecture search device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the implementation of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solution of the embodiment of the present disclosure.

[0103] Figure 9 is a block diagram of a neural network architecture search device according to an exemplary embodiment. Referring to Figure 9 The device 100 includes a determination unit 101 and a search unit 102.

[0104] The determination unit 101 is configured to determine an image processing task and determine a network architecture search space and a candidate operation search space for image processing of the image processing task. The search unit 102 is configured to create a tree structure based on the network architecture search space and the candidate operation search space, and decompose the network architecture search space and the candidate operation search space based on the tree structure to obtain a neural network architecture for the image processing task.

[0105] In an embodiment, the search unit 102 creates a tree structure based on the network architecture search space and the candidate operation search space, and decomposes the network architecture search space and the candidate operation search space based on the tree structure to obtain a neural network architecture for the image processing task in the following manner:

[0106] Based on the network architecture search space, a first tree structure is created, a root node of the first tree structure is a super network structure with an edge number being a layer number corresponding to the network architecture search space, each edge corresponds to a path number of the network architecture search space, a depth of the first tree structure is the layer number, and a leaf node of the first tree structure is a single path structure. Based on the first tree structure, a target single path is selected. A candidate operation search space corresponding to the target single path is decomposed into a second tree structure, a root node of the second tree structure corresponds to all candidate operations in the candidate operation search space, child nodes at the same level correspond to mutually exclusive operation groups, and a leaf node of the second tree structure is a single candidate operation. Based on the second tree structure, a target operation corresponding to the target single path is selected.

[0107] In an implementation, the searching unit 102 creates a first tree structure based on the network architecture search space in the following manner:

[0108] Each architecture layer in the network architecture search space is mapped into a node, and the nodes are connected through directed edges according to the input-output relationship between the architecture layers in the network architecture search space, to obtain a tree structure with a depth of the layer number. The starting node of the directed edge corresponds to an input architecture in the network architecture search space, the terminal node corresponds to an output architecture in the network architecture search space, and the number of directed edges between the nodes is the path number of the network architecture search space. The directed edges corresponding to the nodes mapped by each architecture layer in the network architecture search space are sequentially split in the order of increasing depth, so that each node corresponds to a number of child nodes of the directed edges, until the leaf nodes corresponding to the single path structures are obtained.

[0109] In an implementation, the searching unit 102 selects a target single path based on the first tree structure in the following manner:

[0110] In the first tree structure, the root node with the smallest depth is expanded to a first depth. Each node at the first depth is scored, and the node with the highest score is selected as the root node. The above process is repeatedly performed until the leaf node with the highest score is determined, and the single path corresponding to the leaf node with the highest score is selected as the target single path.

[0111] In an implementation, the searching unit 102 decomposes a candidate operation search space corresponding to the target single path into a second tree structure in the following manner:

[0112] All candidate operations in the candidate operation search space corresponding to the target single path are mapped into a root node. All candidate operations corresponding to the root node are branched into two mutually exclusive operation groups. The candidate operations in the mutually exclusive operation groups are respectively mapped into root nodes, and the step of branching all candidate operations corresponding to the root node into two mutually exclusive operation groups is repeatedly performed until the leaf nodes of the single candidate operations are obtained.

[0113] In one implementation, the searching unit 102 selects the target operation corresponding to the target single path based on the second tree structure in the following manner:

[0114] In the second tree structure, the root node with the minimum depth is expanded to the second depth. Each node at the second depth is scored, and the node with the highest score is selected as the root node. The above process is repeated until the leaf node with the highest score is determined, and the single operation corresponding to the leaf node with the highest score is selected as the target operation corresponding to the target single path.

[0115] In one implementation, the searching unit 102 scores each node at the target depth in the following manner, where the target depth includes the first depth or the second depth:

[0116] If the target depth is the first depth, for each node included in the target depth, the subnets corresponding to all single paths mapped by the node are determined, and the sum of the scores of the subnets corresponding to all single paths is taken as the score of the node. If the target depth is the second depth, for each node included in the target depth, the operation group mapped by the node is determined, and the score obtained by scoring the operation group as a whole is taken as the score of the node.

[0117] As to the apparatus in the above embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0118] Figure 10 is a block diagram of an electronic device 200 for neural network architecture search according to an example embodiment.

[0119] As shown in Figure 10 One embodiment of the present disclosure provides an electronic device 200. The electronic device 200 includes a memory 201, a processor 202, and an input / output (I / O) interface 203. The memory 201 is configured to store instructions. The processor 202 is configured to invoke the instructions stored in the memory 201 to perform the semantic segmentation model training method of the embodiments of the present disclosure. The processor 202 is connected to the memory 201 and the I / O interface 203, for example, through a bus system and / or other forms of connection mechanism (not shown). The memory 201 can be used to store programs and data, including the programs related to the semantic segmentation model training method in the embodiments of the present disclosure. The processor 202 performs various functional applications and data processing of the electronic device 200 by running the programs stored in the memory 201.

[0120] The processor 202 in the embodiments of the present disclosure can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic array (PLA), and can be a combination of one or more of a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities.

[0121] The memory 201 in the embodiments of the present disclosure can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, a random access memory (RAM), a cache, and / or the like. The non-volatile memory can include, for example, a read only memory (ROM), a flash memory, a hard disk drive (HDD), a solid state drive (SSD), and / or the like.

[0122] In the embodiments of the present disclosure, the I / O interface 203 can be used to receive input instructions (such as digital or character information, and generate key signal inputs related to user settings and function control of the electronic device 200, etc.), and can also output various information to the outside (such as images or sounds, etc.). In the embodiments of the present disclosure, the I / O interface 203 can include one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel, etc.

[0123] In some embodiments, the present disclosure provides a computer readable storage medium storing computer executable instructions, which when executed by a processor, perform any of the methods described above.

[0124] In some embodiments, the present disclosure provides a computer program product comprising a computer program, which when executed by a processor, performs any of the methods described above.

[0125] Although the operations in the diagrams are described in a particular, sequential order, this should not be understood as a requirement that these operations be performed in the described order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous.

[0126] The methods and apparatus of the present disclosure can be implemented using standard programming techniques, with rules-based logic or other logic to implement various method steps. It should be noted that the words "component" and "module," as used herein and in the claims, are intended to encompass implementations using one or more lines of software code, and / or hardware implementations, and / or devices that receive inputs and produce outputs.

[0127] Any of the steps, operations, or procedures described herein can be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or procedures described.

[0128] The foregoing description of implementations of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to be limited to the precise form disclosed, and various modifications and variations are possible in light of the above teachings or can be acquired from practice of the present disclosure. One skilled in the art will recognize the same can be employed other than the specific embodiments described and utilized in the foregoing description without departing from the spirit and essential characteristics of the present disclosure.

[0129] With respect to the devices in the above-described embodiments, the specific manner in which the various modules perform operations has been described in detail in the embodiments related to the methods, and will not be described in detail here.

[0130] It can be understood that "multiple" in the present disclosure means two or more, and other quantifiers are similar. The association relationship of "and / or" describing the associated objects means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship. The singular form "a", "said" and "the" are also intended to include the plural form, unless the context clearly indicates otherwise.

[0131] It will be further understood that the terms "first", "second", etc. are used to describe various information but should not be construed as requiring these pieces of information to be in a particular order. The terms "first", "second", etc. are merely used to differentiate one piece of information from another piece of information of the same type, and do not indicate a particular order or level of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, without departing from the scope of the present disclosure.

[0132] It will be further understood that, unless otherwise specified, "connected" includes both direct and indirect connections.

[0133] It will be further understood that, although the operations of the embodiments of the present disclosure are described in a particular, sequential order for convenient presentation, unless otherwise specified, any sequence of operations can be performed to achieve the desired results. Multi-tasking and parallel processing can also be advantageous in some environments.

[0134] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure including those variations contained within the scope of the present disclosure, as well as those adaptations resulting from the exercise of the common general knowledge of those skilled in the art to which the present disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0135] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A neural network architecture search method, characterized by, The method comprises the following steps: determining an image processing task, and determining a network architecture search space and a candidate operation search space for image processing of the image processing task; based on the network architecture search space and the candidate operation search space, creating a tree structure, and based on the tree structure, decomposing the network architecture search space and the candidate operation search space to obtain a neural network architecture of the image processing task; wherein, based on the network architecture search space and the candidate operation search space, creating a tree structure, and based on the tree structure, decomposing the network architecture search space and the candidate operation search space, comprises: based on the network architecture search space, creating a first tree structure, the root node of the first tree structure is a hypernet structure with the number of edges being the number of layers corresponding to the network architecture search space, each edge corresponds to the number of paths of the network architecture search space, the depth of the first tree structure is the number of layers, and the leaf node of the first tree structure is a single path structure; based on the first tree structure, selecting a target single path; wherein, when selecting a target single path, the depth of searching in the first tree structure is determined based on the search accuracy; decomposing the candidate operation search space corresponding to the target single path into a second tree structure, the root node of the second tree structure corresponds to all candidate operations in the candidate operation search space, the child nodes at the same level correspond to mutually exclusive operation groups, and the leaf node of the second tree structure is a single candidate operation; based on the second tree structure, selecting a target operation corresponding to the target single path.

2. The method of claim 1, wherein, The method comprises the following steps: based on the network architecture search space, creating a first tree structure, comprises: mapping each architecture layer in the network architecture search space to a node, and connecting the nodes through directed edges according to the input-output relationship between the architecture layers in the network architecture search space to obtain a tree structure with a depth of the number of layers; wherein, the starting node of the directed edge corresponds to the input architecture in the network architecture search space, the terminal node corresponds to the output architecture in the network architecture search space, and the number of directed edges between nodes is the number of paths of the network architecture search space; 3. The method according to claim 1 or 2, characterized in that, according to the order of depth from small to large, the directed edges corresponding to the nodes mapped by each architecture layer in the network architecture search space are split in turn, so that each node corresponds to a number of sub-nodes of the directed edges, until the leaf node of the corresponding single path structure is reached. The method comprises the following steps: based on the first tree structure, selecting a target single path, comprises: in the first tree structure, expanding from the root node with the smallest depth to the first depth; 4. The method of claim 1, wherein, scoring each node at the first depth, and selecting the node with the highest score as the root node; repeat the above process until the leaf node with the highest score is determined, and the single path corresponding to the leaf node with the highest score is selected as the target single path. The method comprises the following steps: decomposing the candidate operation search space corresponding to the target single path into a second tree structure, comprises: mapping all candidate operations in the candidate operation search space corresponding to the target single path to a root node; branching all candidate operations corresponding to the root node into two mutually exclusive operation groups; The candidate operations in the mutually exclusive operation group are respectively mapped as root nodes, and the step of branching all candidate operations corresponding to the root nodes into two mutually exclusive operation groups is repeatedly performed until a leaf node of a single candidate operation is obtained.

5. The method according to claim 1 or 4, characterized in that, The target operation corresponding to the target single path is selected based on the second tree structure, including: In the second tree structure, starting from the root node with the minimum depth and expanding to the second depth; Scoring each node at the second depth and selecting the node with the highest score as the root node; The above process is repeatedly performed until the leaf node with the highest score is determined, and the single operation corresponding to the leaf node with the highest score is selected as the target operation corresponding to the target single path.

6. The method of claim 3, wherein, The scoring of each node at the first depth includes: For each node included in the first depth, determine the subnets corresponding to all single paths mapped by the node, and sum the scores of the subnets corresponding to all single paths as the score of the node.

7. The method of claim 5, wherein, The scoring of each node at the second depth includes: For each node included in the second depth, determine the operation group mapped by the node, and score the operation group as a whole to obtain the score of the node.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the method of any one of claims 1-7.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the method of any one of claims 1-7.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the method of any one of claims 1-7.

Images