Neural architecture search method, neural architecture search device and readable recording medium

Abstract

The invention provides a neural architecture search method, a neural architecture search device and a readable recording medium for a neural architecture search (NAS) pipeline. The present disclosure provides methods, systems, and apparatus for a neural architecture search (NAS) pipeline for optimized neural architecture search (NAS). The method comprises the following steps: acquiring a first search space comprising a plurality of candidate layers for a neural network architecture; performing non-training NAS in the first search space to obtain a first architecture set; obtaining a second search space based on the first set of architectures; performing gradient-based search in a second search space to obtain a second architecture set; using the second architecture set as an initial sample to perform a sampling method search; and obtaining an output architecture as an output of the sampling method search.

Description

Technical Field

[0001] The systems and methods of exemplary embodiments of the present disclosure are directed to providing a pipeline for evaluating a machine learning model. Background Art

[0002] In the related art, a neural network can be generally characterized by two main parameters: (1) the architecture of the neural network and (2) the weights applied to the inputs transmitted between neurons. Typically, the architecture is designed manually, i.e., by hand, by the user, and the weights are optimized by training the network using a training set and a neural network algorithm. Therefore, in order to optimize the performance of the neural network, especially since the neural network is generally static after being deployed for use, the architectural design is an important consideration. Figure 1 is a diagram showing an example of a neural network architecture of the related technology.

[0003] Reference Figure 1 , the neural network architecture of the related technology is defined by the connection, the type of layer (or action), the number of layers, and the number of channels in each layer. The layer types may include, for example, convolution (in the case of a convolutional neural network (CNN)), activation (e.g., ReLU (Rectified Linear Unit)), pooling, fully connected, batch normalization, random inactivation (dropout), etc. The network architecture design is reflected by the combination of layers corresponding to at least some of these layer types.

[0004] In the related art, there is NAS (Neural Architecture Search) as a technique for automatically designing the architecture of a neural network. The related art methods for performing NAS may include zero-shot NAS, hypernetwork-based NAS, and simple iterative search.

[0005] In related techniques, training-free (zero-shot) NAS predicts network performance without training the network parameters. This method is fast, but sometimes does not have good accuracy. For example, this method can be done by evaluating metrics or architectures in the search space without actually training.

[0006] Figure 2 is a diagram of hypernetwork-based (gradient-based) NAS representing the associated technique. Figure 3 An example of a superblock in a hypernetwork-based NAS is shown below. Figure 4 An example of the flow of the NAS method based on the hypernetwork is shown as a related technology.

[0007] Hypernetwork-based (gradient-based) NAS can use a hypernetwork. A hypernetwork is a network that includes all candidate architectures. Figure 2 , the hypernetwork-based search searches for neural network architectures through a process of training multiple connected superblocks at once, each superblock corresponds to a layer (and / or a set of layers) and includes many options for candidate layers (and / or a set of layers). For example, option 1 can be 3×3CNN, option 2 can be 5×5CNN, option 3 can be max pooling or residual block, etc. In the hypernetwork-based search, the best layer is searched and selected in each superblock. Figure 3 and Figure 4 As shown, each candidate has a θ value (or architecture parameter), which is updated through the training process. That is, if a candidate is judged to significantly contribute to the reduction of loss, the θ value of the candidate (indicating the importance of the candidate layer) is increased through the error back propagation method, and vice versa. After the architecture parameter θ is updated in one iteration of the super network-based search, the weight parameters are updated using training images (e.g., images of cars). This process is repeated until a certain degree of network stability is reached (i.e., a certain degree of delay (inference time) or loss is reached) or until a specified number of iterations are completed. When the training is completed, the best layer (i.e., the layer with the highest θ value) is selected for each super block to generate the final (optimal) network architecture.

[0008] Figure 5 This is a functional block diagram of the sampling method of the correlation technique. Figure 5 , the sampling method is an iterative process by which the controller generates sample architecture candidates (e.g., from a search space including a set of layers). The sample architecture is optimized and evaluated through training to obtain metrics (e.g., accuracy, latency, model size, etc.). Next, the controller generates the next sample architecture candidate based on the evaluation result of the previous sample using methods such as reinforcement learning, Bayesian optimization, and evolutionary algorithms. The process is repeated until the target performance is achieved (e.g., until the target accuracy, target latency, target model size, etc. are achieved).

[0009] Figure 5 The sampling method exemplified in can lead to the most correct / optimal architecture, but it is very time-consuming to perform across a large number of candidate locations due to the many training steps involved.

[0010] When NAS is performed manually (as is common in related art systems), higher quality architectures can sometimes be obtained reliably, but it is very time consuming and burdensome to the user. Therefore, it is desirable to automate NAS.

[0011] However, the above-mentioned conventional methods for automating NAS in the related art may be time-consuming, and sometimes the optimal network architecture may not be obtained. In particular, when the search space is large, excessive resources and time are consumed for completion. On the other hand, when the search space is too small, the possibility of becoming a suboptimal architecture with low performance is very high. Therefore, a method for NAS that can find a network with the best performance in the shortest time is needed. Summary of the invention

[0012] According to an embodiment, a method, system, and apparatus for performing an optimized neural architecture search (NAS) pipeline are provided. Specifically, the apparatus and method of the exemplary embodiment can sequentially perform untrained NAS, hypernetwork / gradient-based search, and sampling search to reduce the number of architectures that need to be trained at the end of the sampling search while ensuring the high quality of the architecture provided at the end of the sampling search. Therefore, an overall high-quality / optimal architecture can be found in the shortest possible time.

[0013] According to an embodiment, a method for performing a neural architecture search (NAS) may be provided. The method includes: obtaining a first search space including a plurality of candidate layers for a neural network architecture; performing untrained NAS in the first search space to obtain a first architecture set; obtaining a second search space based on the first architecture set; performing a gradient-based search in the second search space to obtain a second architecture set; performing a sampling search using the second architecture set as an initial sample; and obtaining an output architecture as an output of the sampling search.

[0014] According to some embodiments, the first search space may be obtained based on a set of architecture parameters. The second search space may be obtained based on a subspace including more than one architecture within the first set of architectures.

[0015] According to several embodiments, the sampling search may be performed iteratively. The sampling search may include an evolutionary search algorithm. The number of iterations of the sampling search may be based on a specified threshold. The untrained NAS may be performed based on more than one metric.

[0016] Additional aspects will be described in part in the description that follows and may be apparent in part from the description, or may be achieved by practicing the presented embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Hereinafter, features, advantages, and significance of exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings, in which like reference numerals represent like elements.

[0018] Figure 1 is a diagram showing an example of a neural network architecture of the related technology.

[0019] Figure 2 It is a functional block diagram of a hypernetwork-based (gradient-based) search method of the association technique.

[0020] Figure 3 This is a diagram showing an example of a super block of a hypernetwork-based search method of a related technology.

[0021] Figure 4 It is a flowchart of a hypernetwork-based search method of association technology.

[0022] Figure 5 This is a functional block diagram showing the sampling method of the related art.

[0023] Figure 6 is a diagram of an exemplary neural architecture search (NAS) pipeline representing one or more embodiments.

[0024] Figure 7 is a flow chart of an exemplary method for performing an optimized NAS process in accordance with one or more embodiments. DETAILED DESCRIPTION

[0025] The following detailed description of exemplary embodiments refers to the accompanying drawings. The foregoing disclosure provides examples and illustrations, but is not intended to be exhaustive or to limit the implementation to the strict form disclosed. Modified forms and deformed forms can be considered in reference to the above disclosure, or can be obtained according to the implementation of the implementation. Moreover, one or more features or constituent elements of an embodiment can be incorporated into another embodiment (or one or more features of another embodiment) or combined with another embodiment (or one or more features of another embodiment). In addition, it is understood that in the flowcharts and descriptions of actions provided below, one or more actions can be omitted, one or more actions can be added, one or more actions can be performed (at least partially) simultaneously, and the order of one or more actions can be reversed.

[0026] It is obvious that the systems and / or methods described in this specification can be implemented by various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the actions and behaviors of the systems and / or methods are recorded in this specification without mentioning specific software codes. Therefore, it is understood that software and hardware can be designed in a manner to implement the systems and / or methods based on the records of this specification.

[0027] Specific combinations of features are described in the claims and / or disclosed in this specification, but these combinations are not intended to limit the disclosure of conceivable implementations. In fact, many of these features can be combined in schemes that are not specifically described in the claims and / or not specifically disclosed in this specification. Each of the dependent claims listed below may be directly dependent on only one claim, but the disclosure of conceivable implementations includes the combination of each dependent claim with all other claims in the claims.

[0028] As long as it is not clearly stated, the elements, actions or commands used in this specification should not be interpreted as being important or necessary. In addition, when used in this specification, the articles "a" and "an" mean including more than one project and can be used as words that can be interchanged with "more than one". In the case of only meaning one project, the term "one" or similar language is used. In addition, when used in this specification, the terms "has", "have", "having", "include", "including" etc. mean to be open terms. Moreover, unless otherwise expressly provided, a statement such as "based on..." means to refer to "at least partially based on...". Moreover, "at least one of [A] and [B]" or "at least one of [A] or [B]" etc. should be understood to include only A, only B or both A and B.

[0029] References throughout this specification to "one embodiment," "an embodiment," "a non-limiting exemplary embodiment," or similar language refer to a particular feature, structure, or characteristic described in connection with the illustrated embodiment being included in at least one embodiment of the present solution. Thus, phrases such as "in one embodiment," "in an embodiment," "in a non-limiting exemplary embodiment," and similar language throughout this specification may all refer to the same embodiment, but do not necessarily.

[0030] Furthermore, the features, advantages, and characteristics described in the present disclosure may be combined in any preferred manner in one or more embodiments. In contrast to the description of this specification, those skilled in the art will recognize that the present disclosure may be implemented without one or more of the specific features or advantages of a specific embodiment. In other cases, additional features and advantages may be recognized in a specific embodiment, but they are not present in all embodiments of the present disclosure.

[0031] The exemplary embodiments of the present disclosure provide a method and system for sequentially performing untrained NAS, hypernetwork / gradient-based search, and sampling search to reduce the number of architectures that need to be trained at the end of the sampling search, and on the other hand, to ensure the high quality of the architecture provided at the end of the sampling search. Therefore, an overall high-quality / optimal architecture can be found in the shortest possible time.

[0032] Figure 6 600. Figure 6 , a baseline model and / or target latency for a desired architecture may be provided (e.g., as input by a user) to the NAS pipeline 600. In particular, this is useful for providing an initial search space, since an initial search space can be generated based on the baseline model, and the target latency can be used to filter out architectures that are inconsistent with the target latency.

[0033] Reference Figure 6 In action S601, an initial first search space may be obtained (e.g., constructed). The initial first search space may include a plurality of candidate layers for a neural network architecture. According to an embodiment, the initial first search space may be provided by a user, or may be obtained based on a set of architecture parameters.

[0034] Reference Figure 6 , in action S602, the initial first search space may be narrowed to obtain a second search space. According to an embodiment, the narrowing may be performed by first performing a training-free NAS on the first search space to obtain a first set of architectures. It should be understood that a variety of methods may be used in a training-free NAS as long as no training is involved. For example, a good architecture may be evaluated based on a simple metric. Thus, a first set of architectures may be obtained. Training is not included in action S602, so the completion speed should be relatively fast.

[0035] Afterwards, the first architecture set can be used to obtain the second search space. It can also be considered that the second search space is a "reduced" or "shrinked" search space compared to the first search space. According to an embodiment, a subspace that includes and / or surrounds each architecture in the first architecture set can be used to determine the second search space. For example, in the case where the first architecture set includes architecture 1, architecture 2, and architecture 3, subspace 1, subspace 2, and subspace 3 can correspond to the search space (subspace) surrounding architecture 1, architecture 2, and architecture 3, respectively. Therefore, in one example, the second search space can be the union of subspace 1, subspace 2, and subspace 3.

[0036] Refer again Figure 6 In action S603, a second (reduced) search space can be used to perform a hypernetwork-based NAS (gradient-based search). According to this method, a second set of architectures based on the second search space is obtained. The gradient-based search is accompanied by only one training, and the second search space is a search space that has been reduced, so it should still be relatively fast.

[0037] Refer again Figure 6 In action S604, each architecture in the second architecture set can be used as an initial sample to perform a sampling method (full training search). According to an embodiment, this can be an iterative full training search. Compared with the first architecture set, the second architecture set has been relatively limited, so even if full training is performed, the time is still relatively short. On the contrary, since full training is performed, the accuracy can be considered to be optimal / high. Therefore, the NAS pipeline 600 can bring about the result that the best architecture is obtained.

[0038] According to one embodiment, the sampling method search can be an evolutionary search algorithm. In addition, the number of iterations can be based on a specified threshold. Specifically, the evolutionary search is performed iteratively, and in addition, the evolutionary search can include using a performance metric or a fitness / validation score to obtain an optimized architecture based on a second search space. According to one embodiment, in each iteration of the evolutionary search, the entire search space (e.g., any conceivable layer, layer type, number of channels, number of layers) is used to mutate the candidate architecture (e.g., change a 5×5 convolutional layer to a 3×3 convolutional layer), and the candidate architecture is trained and evaluated (e.g., with several rounds (epochs)). Then, the architecture with the lowest score in the candidate pool is replaced with a new (mutated) architecture that is determined to function better. It is repeated for multiple iterations until the iteration is performed for a specified number of times, or until a specified threshold (e.g., a specified optimal performance) is reached, and the optimized architecture in the second search space is output. Nevertheless, it should be noted that it should be understood that several conceivable benchmarks can be used for thresholds, which are thresholds for selecting optimized architectures. The simplest approach is to select the architecture with the highest score, but in the case of considering multiple metrics (e.g., accuracy and latency (execution speed)), an exemplary process will output all architectures in the Pareto-front set. Another conceivable approach is to use crossover in the evolutionary search process. Specifically, a low-scoring architecture can be replaced by an architecture that is a crossover of two architectures that have a higher score than the low-scoring architecture.

[0039] The above-mentioned embodiment utilizes evolutionary search, but it is understood that one or more other embodiments are not limited thereto. In other embodiments, other sampling methods or algorithms (eg, reinforcement learning) may be utilized.

[0040] Figure 7 A flow chart showing an exemplary method 700 for performing an optimized NAS process. Figure 7 , in action S710, a first search space having candidate layers for a neural network architecture is obtained. This can be similar to the above-mentioned Figure 6 Action S601 described above is similar.

[0041] In action S720, a non-trained NAS may be performed on the first search space to obtain a first architecture set. Figure 6 Portions of action S602 described above are similar.

[0042] In action S730, a second (reduced) search space may be obtained based on the first set of architectures. According to some embodiments, this may be based on a subspace that includes and / or surrounds each architecture in the first set of architectures to determine the second search space. This may be similar to the above-mentioned method for Figure 6 Portions of action S602 described above are similar.

[0043] In action S740, a hypernetwork-based (gradient-based) search may be performed in the second search space obtained in action S730 to output a second architecture set. Figure 6 Parts of the described action S603 are similar.

[0044] In action S750, the second set of frames obtained from action S740 may be used as an initial sample to perform a sampling method. According to one embodiment, the sampling method may be an evolutionary search algorithm. The evolutionary search algorithm may be repeated over several iterations, and the number of iterations may be based on a specified threshold. This may be similar to the above-mentioned method for Figure 6 Action S604 described above is similar.

[0045] In action S760, the best architecture as a result of the sampling method performed in action S750 is output. It should be understood that this embodiment describes outputting one best architecture, but some embodiments may also output multiple best architectures.

[0046] In view of the above, exemplary embodiments of the present disclosure provide a method and system for optimizing neural architecture search (NAS) by narrowing the search space after roughly setting the initial search space in a manner that substantially speeds up the search, and by combining the narrowed search space with the hypernetwork NAS and iterative sampling method search, an architecture with the best performance can be obtained. Therefore, by combining these search methods in sequence, the best performance architecture can be found with substantially shorter search time and computation time.

[0047] It should be understood that the specific order or hierarchy of the functional blocks in the process / flowchart disclosed in this specification is an example of an example of a method. It should be understood that the specific order or hierarchy of the functional blocks in the process / flowchart can be rearranged based on design preferences. Moreover, several functional blocks can be combined or can be omitted. The attached method technical solution presents the elements of various functional blocks as an example of a sequence, and is not intended to be limited to the specific order or hierarchy presented.

[0048] Several embodiments may be associated with systems, methods, and / or computer-readable media at any level of detail of the technology that can be imagined. Moreover, one or more of the above-mentioned constituent elements described above may be implemented as commands that can be executed by at least one processor stored on a computer-readable medium (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or multiple media) having computer-readable program commands for causing a processor to perform actions.

[0049] A computer-readable storage medium may be a tangible device that can hold and store commands for use by a command execution device. The computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any preferred combination thereof, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media also includes: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disc read-only memories (CD-ROM), digital versatile discs (DVD), memory sticks, floppy disks, mechanically encoded punched cards, and other devices or raised structures in grooves with recorded commands, and any appropriate combination thereof. When used in this specification, a computer-readable storage medium itself should not be interpreted as a temporary signal such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0050] The computer-readable program commands recorded in this specification can be downloaded from a computer-readable storage medium to a respective computing / processing device, or can be downloaded to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program commands from the network and transmits the computer-readable program commands to be stored in a computer-readable storage medium in the respective computing / processing device.

[0051] The computer readable program code / commands for performing actions may be any of an assembly command, an instruction set architecture (ISA) command, a machine command, a machine-dependent command, a microcode, a firmware command, a state setting data, a setting data of an integrated circuit, or a source code or an object code, the source code or the object code being written in any combination of one or more programming languages ​​including object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program commands may be executed entirely on the user's computer as a stand-alone software package, or may be executed partially on the user's computer, may be executed partially on the user's computer and partially on a remote computer, or may be executed entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, for example, in order to perform a scheme or action, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) uses state information of computer-readable program commands to personalize the electronic circuit, thereby executing the computer-readable program commands.

[0052] These computer-readable program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to generate a machine so that commands executed by the processor of the computer or other programmable data processing device create a means for implementing the functions / actions specified in one or more functional blocks of the flowchart and / or block diagram. These computer-readable program commands can also be stored in a computer-readable storage medium, wherein the computer-readable storage medium can instruct a computer, a programmable data processing device, and / or other devices to function in a specific way, and therefore, the computer-readable storage medium storing the commands has a product, which includes commands for implementing the scheme of the functions / actions specified in one or more functional blocks of the flowchart and / or block diagram.

[0053] In addition, computer-readable program commands can be loaded into a computer, other programmable data processing apparatus, or other device, and a series of action steps can be performed on the computer, other programmable apparatus, or other device to generate a computer-implemented process. Therefore, the commands executed on the computer, other programmable apparatus, or other device will implement the functions / actions specified in one or more functional boxes in the flowchart and / or block diagram.

[0054] The flowchart and block diagram in the figure show the architecture, function and action of the possible implementation of the system, method and computer-readable medium of various embodiments. In this regard, each function block in the flowchart or block diagram can represent a module, segment or part of a command including one or more executable commands for implementing the specified logical function (which can be multiple). The method, computer system and computer-readable medium can also include additional function blocks, fewer function blocks, function blocks different from it or function blocks configured differently from the function blocks shown in the figure. In several alternative implementations, the functions recorded in the function blocks can also be performed in an order different from the order recorded in the figure. For example, two function blocks shown in succession can actually be executed simultaneously or substantially simultaneously, or the function blocks can sometimes be executed in the opposite order according to the functions included. It should also be noted that the combination of each function block of the block diagram and / or flow chart and the function blocks of the block diagram and / or flow chart can be implemented by a system based on special hardware, wherein the system performs the specified function or action, or performs a combination of special hardware and computer commands.

[0055] It is apparent that the systems and / or methods described in this specification may be implemented by various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the actions and behaviors of the systems and / or methods are described in this specification without reference to specific software codes. It is understood that software and hardware can be designed to implement the systems and / or methods based on the description of this specification.

Claims

1. A neural architecture search method is a method for performing neural architecture search (NAS), wherein: The method comprises: Obtaining a first search space comprising a plurality of candidate layers for a neural network architecture; Performing untrained NAS in the first search space to obtain a first architecture set; Acquire a second search space based on the first architecture set; performing a gradient-based search in the second search space to obtain a second set of architectures; Using the second framework set as an initial sample to perform a sampling search; and An output architecture is obtained as an output of the sampling method search.

2. The neural architecture search method according to claim 1, wherein: The first search space is obtained based on a set of architecture parameters.

3. The neural architecture search method according to claim 1 or 2, wherein: The second search space is obtained based on a subspace including one or more architectures in the first set of architectures.

4. The neural architecture search method according to any one of claims 1 to 3, wherein: The sampling search is performed iteratively.

5. The neural architecture search method according to claim 4, wherein: The sampling search includes an evolutionary search algorithm.

6. The neural architecture search method according to claim 5, wherein: The number of iterations of the sampling method search is based on a specified threshold.

7. The neural architecture search method according to any one of claims 1 to 6, wherein: The untrained NAS is performed based on more than one metric.

8. A neural architecture search device is a device for performing neural architecture search (NAS), the device comprising: at least one memory storing computer executable instructions; and at least one processor, The processor is configured to execute the computer executable instructions to perform the following processing: Obtaining a first search space comprising a plurality of candidate layers for a neural network architecture; Performing untrained NAS in the first search space to obtain a first architecture set; Acquire a second search space based on the first architecture set; performing a gradient-based search in the second search space to obtain a second set of architectures; Using the second architecture set as an initial sample to perform a sampling search; as well as An output architecture is obtained as an output of the sampling method search.

9. The neural architecture search device according to claim 8, wherein: The first search space is obtained based on a set of architecture parameters.

10. The neural architecture search device according to claim 8 or 9, wherein: The second search space is obtained based on a subspace including one or more architectures in the first set of architectures.

11. The neural architecture search device according to any one of claims 8 to 10, wherein: The sampling search is performed iteratively.

12. The neural architecture search device according to claim 11, wherein: The sampling search includes an evolutionary search algorithm.

13. The neural architecture search device according to claim 12, wherein: The number of iterations of the sampling method search is based on a specified threshold.

14. The neural architecture search device according to any one of claims 8 to 13, wherein: The untrained NAS is performed based on more than one metric.

15. A non-transitory computer-readable recording medium having recorded thereon commands executable by at least one processor, the commands causing the at least one processor to perform a neural architecture search method, the neural architecture search method comprising: Obtaining a second search space based on the first architecture set; performing a gradient-based search in the second search space to obtain a second set of architectures; Using the second architecture set as an initial sample to perform a sampling search; as well as An output architecture is obtained as an output of the sampling method search.

16. The non-transitory computer-readable recording medium according to claim 15, wherein: A first search space is obtained based on a set of architectural parameters.

17. The non-transitory computer-readable recording medium according to claim 15 or 16, wherein: The second search space is obtained based on a subspace including one or more architectures in the first set of architectures.

18. The non-transitory computer-readable recording medium according to any one of claims 15 to 17, wherein: The sampling search is performed iteratively.

19. The non-transitory computer-readable recording medium according to claim 18, wherein: The sampling search includes an evolutionary search algorithm.

20. The non-transitory computer-readable recording medium according to claim 19, wherein The number of iterations of the sampling method search is based on a specified threshold.

Images