Neural network construction device and method, information processing device and recording medium

Abstract

The neural network construction device (10) comprises: an acquisition unit (11) for acquiring resource information related to the computing resources of the embedded device and performance constraints related to the processing performance of the embedded device; a setting unit (12) for setting the scale constraints of the neural network based on the resource information; a generation unit (13) for generating a model of the neural network based on the scale constraints; and a judgment unit (14) for judging whether the generated model satisfies the performance constraints and outputting data based on the judgment result.

Description

Technical Field

[0001] The present invention relates to information processing technology for constructing neural networks. Background Art

[0002] As a technology that can more efficiently design a neural network suitable for processing by multiple hardware, an information processing device and an information processing method are disclosed, which include an acquisition unit for acquiring constraints related to multiple hardware devices and a determination unit for determining whether the neural network satisfies the constraints (see Patent Document 1).

[0003] Prior Art Literature

[0004] Patent Literature

[0005] [Patent Document 1] International Publication No. 2017 / 187798 Summary of the invention

[0006] Problems to be solved by the invention

[0007] In the technology described in Patent Document 1, whether each neural network as a candidate for the best neural network satisfies the above-mentioned constraints is determined. That is, the number of trial and error based on design and determination is huge and time-consuming before the best neural network is obtained.

[0008] Therefore, the present disclosure provides a neural network construction device that helps improve the efficiency of obtaining the best neural network by narrowing down the candidates of the neural network. In addition, the present disclosure provides a neural network construction method and program used in the neural network construction device.

[0009] Means for solving problems

[0010] A neural network construction device involved in one embodiment of the present invention for solving the above-mentioned problems comprises: an acquisition unit, which acquires a first condition and a second condition, wherein the first condition is a condition used to determine a candidate hyperparameter, the candidate hyperparameter is a candidate for a hyperparameter of a neural network to be constructed, and the second condition is a condition related to the performance that the model of the neural network should have; a setting unit, which uses the first condition to determine the candidate hyperparameter; a generation unit, which uses the candidate hyperparameter to generate a model of the neural network; and a judgment unit, which judges whether the generated model satisfies the second condition and outputs data based on the result of the judgment.

[0011] In addition, a neural network construction method involved in one embodiment of the present invention is a neural network construction method executed by an operation processing device in a neural network construction device having an operation processing device and a storage device, obtaining resource information related to the computing resources possessed by an embedded device and performance constraints related to the processing performance possessed by the embedded device, setting a scale constraint of the neural network based on the resource information, generating a model of the neural network based on the scale constraint, judging whether the generated model satisfies the performance constraint, and outputting data based on the result of the judgment.

[0012] In addition, a program involved in one embodiment of the present invention is a program executed by an operation processing unit in a neural network construction device having an operation processing unit and a storage device, and the program is executed by the operation processing unit so that the neural network construction device performs the following operations: obtaining resource information related to computing resources possessed by an embedded device and performance constraints related to processing performance possessed by the embedded device, setting a scale constraint of the neural network based on the resource information, generating a model of the neural network based on the scale constraint, judging whether the generated model satisfies the performance constraint, and outputting data based on the result of the judgment.

[0013] Furthermore, in order to facilitate understanding of the present disclosure, the terms are explained as follows.

[0014] Python: A general-purpose programming language. Widely used in the field of machine learning.

[0015] Model: A mathematical formula or function that makes a desired prediction or judgment based on given data.

[0016] Neural Network: A model based on a network of artificial neurons (also called nodes) that imitates the structure of nerve cells and neural circuits in the human brain.

[0017] Weight: One of the parameters of the model, representing the strength of the coupling between neurons. Also called coupling load.

[0018] Bias: One of the parameters of the model that adjusts the output obtained according to the input value and weight to the neuron.

[0019] Here, the concept of a neural network including the relationship between neurons, weights, and biases is represented using drawings. Figure 1 It is a diagram used to illustrate the concept of neural network. Figure 1 The neural network illustrated in FIG. 1 is composed of a plurality of layers each including a plurality of neurons represented by white circles.

[0020] The leftmost layer is the input layer of the neural network, and input values ​​are set for each neuron in this layer. The lines connecting neurons between layers represent weights. The input values ​​of each neuron are multiplied by the weights and then input to the neurons in the right layer. The rightmost layer is the output layer of the neural network, and the values ​​of each neuron in this layer are the results of the prediction or judgment made by the neural network. In addition, the bias is Figure 1 In the figure, the shaded circles represent input values ​​respectively input from the neurons in the layer on the left, as described above.

[0021] Fully coupled neural network: It is a hierarchical neural network with a structure in which the neurons in each layer are coupled with all the neurons in the next layer. Figure 1 The neural network is a fully coupled neural network.

[0022] Learning: Repeatedly adjust weights and biases to make the prediction / judgment output corresponding to the input data close to correct.

[0023] Learning data: Data used for learning the model of the generated neural network. Image data or numerical data, etc. are prepared according to the target problem.

[0024] Inference model: The model that has been learned is called an inference model. The accuracy of prediction / judgment is evaluated by using this inference model.

[0025] Reasoning: Assign unknown data that has not been used in learning to the reasoning model to obtain prediction / judgment results.

[0026] Hyperparameters: Among the parameters of the model, these are not parameters that are determined by learning like weights, but rather parameters that need to be determined in advance before learning, such as the number of neurons or the depth (number of layers) of the network. The configuration of the model is determined by setting hyperparameters.

[0027] Evaluation completed model: A model in which unknown data not used in learning is assigned to the inference model and the accuracy is evaluated.

[0028] Effects of the Invention

[0029] The neural network construction device provided in the present disclosure narrows down the candidates of neural networks that meet various conditions, which helps to improve the efficiency of obtaining the best neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a diagram used to illustrate the concept of neural network.

[0031] Figure 2 This is a block diagram showing an example of the functional configuration of a neural network building device according to an embodiment.

[0032] Figure 3 This is a block diagram showing an example of a hardware configuration used to implement a neural network building device according to an embodiment.

[0033] Figure 4 This is a diagram for explaining the concept of distribution of hyperparameters used in building a neural network.

[0034] Figure 5 This is a flowchart showing an example of a processing procedure of a neural network building method executed by a neural network building device according to an embodiment.

[0035] Fig. 6A This is a diagram for explaining an overview of a hyperparameter search method using Bayesian optimization.

[0036] Figure 6B This is a diagram for explaining an overview of a hyperparameter search method using Bayesian optimization.

[0037] Figure 6C This is a diagram for explaining an overview of a hyperparameter search method using Bayesian optimization.

[0038] Figure 7 This is a diagram showing an example of the configuration of a fully coupled neural network.

[0039] Figure 8 This is a diagram showing an example of the configuration of a convolutional neural network.

[0040] Fig. 9 It is a graph showing an example of the frequency characteristics of a low-pass filter.

[0041] Fig.10 This is a flowchart showing an example of a processing procedure of a neural network building method executed by a neural network building device according to an embodiment.

[0042] Fig.11 This is a flowchart showing an example of the first part of another example of the processing procedure of the neural network building method executed by the neural network building device according to the embodiment.

[0043] Fig.12 This is a flowchart showing an example of the latter part of another example of the processing procedure of the neural network building method executed by the neural network building device according to the embodiment. DETAILED DESCRIPTION

[0044] (Insights, etc., which are the basis of the present invention)

[0045] As described above, in the prior art, it takes a long time of trial and error to satisfy hardware constraints and obtain a neural network with higher accuracy.

[0046] On the other hand, neural networks are also being introduced into so-called embedded devices (sometimes called embedded devices or embedded systems, which are not specifically distinguished below and are recorded as embedded devices) installed in electrical products or automobiles in pursuit of high functionality. Furthermore, in the current situation where IoT (Internet of Things) is constantly developing, embedded devices are being installed not only in electrical products but also in a variety of things in order to give them additional functions including communication.

[0047] Such embedded devices are subject to hardware constraints due to the size, purpose, usage, price, etc. of the object to be mounted. However, in the above-mentioned prior art, various neural networks that work in various embedded products used in various objects cannot be developed quickly and at low cost.

[0048] The above-mentioned hardware constraints are just one example, and there may be other constraints determined by various factors. In the above-mentioned prior art, a lot of trial and error is required before obtaining a neural network that satisfies such constraints.

[0049] In view of such problems, the present inventors have conceived a technology for more quickly acquiring candidates for neural networks that show higher accuracy while satisfying hardware constraints imposed during the design and development process of embedded devices and the like.

[0050] The neural network construction device involved in this technology comprises: an acquisition unit, which acquires a first condition and a second condition, wherein the first condition is a condition used to determine a candidate hyperparameter, the candidate hyperparameter is a candidate for a hyperparameter of a neural network to be constructed, and the second condition is a condition related to the performance that the model of the neural network should have; a setting unit, which uses the first condition to determine the candidate hyperparameter; a generation unit, which uses the candidate hyperparameter to generate a model of the neural network; and a judgment unit, which judges whether the generated model satisfies the second condition and outputs data based on the result of the judgment.

[0051] Thus, by selecting from the candidates that have been narrowed down by excluding objects with structures that are unlikely to satisfy the conditions, the optimal neural network can be efficiently obtained.

[0052] For example, the setting unit may calculate at least one of an upper limit and a lower limit of the candidate hyper-parameter using the first condition, and determine one or more candidate hyper-parameters based on at least one of the calculated upper limit and lower limit.

[0053] Thus, by selecting from the candidates that have been narrowed down by excluding objects with structures that are unlikely to meet the desired scale or performance, the optimal neural network can be efficiently obtained.

[0054] In addition, for example, the first condition may include a resource condition related to the computing resources possessed by the embedded device, and the setting unit may calculate an upper limit of the candidate hyperparameter based on the resource condition, and determine at least a part of the hyperparameters below the upper limit as the candidate hyperparameters.

[0055] In the neural network construction device, the scale of the generated neural network model is accommodated within the range that can be installed in the embedded device based on the specified hardware specifications. Therefore, there is no need to repeat trial and error based on design and judgment as in the previous method, and any model once generated is used as the object of judgment on whether the second condition is satisfied, which is less wasteful. Then, the model that satisfies the second condition is the object of evaluation of accuracy after further learning. In other words, it is possible to efficiently obtain candidates for models that can be installed in the above-mentioned specified embedded device and are the object of evaluation of accuracy without repeating the trial and error process starting from design as in the past. By adopting another expression, it is possible to suppress the overhead before obtaining the optimal neural network model for the embedded device intended to be used.

[0056] In addition, for example, the resource condition may include information about the memory size of the embedded device, and the setting unit may calculate the upper limit of the hyperparameters of the neural network accommodated by the memory size as the upper limit of the candidate hyperparameters, and determine at least a part of the hyperparameters below the upper limit as the candidate hyperparameters.

[0057] Thus, the embedded device to be used and the factors that have a great influence on whether the neural network can be installed in the embedded device are considered in advance. Therefore, the generated model is a model that can be installed in the embedded device, so it is possible to suppress the unnecessary execution of the subsequent judgment related to the second condition and the prediction accuracy evaluation processing.

[0058] In addition, for example, the first condition may include information on at least one of the size of input data to the neural network and the size of output data from the neural network, and the setting unit may calculate the upper limit of the candidate hyperparameter based on at least one of the size of the input data and the size of the output data included in the first condition, and determine at least a part of the hyperparameters below the calculated upper limit as the one or more candidate hyperparameters. More specifically, the size of the input data may be the dimension of the input data, the size of the output data may be the dimension of the output data, and the one or more candidate hyperparameters may include at least one number of layers and nodes of the neural network, respectively. In addition, the first condition may also include information indicating that the neural network is a convolutional neural network. In addition, in this case, the input data may be image data, the size of the input data may be the number of pixels of the image data, the size of the output data may be the number of classes into which the image data is classified, and the one or more candidate hyperparameters may be at least one of the number of layers, the size of the kernel, the depth of the kernel, the size of the feature map, the window size of the pooling layer, the padding amount, and the step amount of the convolutional neural network. Alternatively, the first condition may include an accuracy target for inference based on the neural network model, and the setting unit may calculate a lower limit of the candidate hyperparameters using the accuracy target, and determine at least a portion of the calculated hyperparameters above the lower limit as one or more candidate hyperparameters.

[0059] Thus, the candidates for the best neural network can be effectively narrowed down to neural networks having a configuration that satisfies conditions determined according to the problem to be solved.

[0060] In addition, for example, the second condition may include a time condition related to a benchmark required time for inference processing of a model using a neural network, the generating unit calculates the required time for inference processing using the generated model based on the resource condition, and the judging unit judges whether the generated model satisfies the second condition by comparing the calculated required time with the benchmark required time.

[0061] Thus, even if it is a model that satisfies the constraints of scale, the model that does not meet the performance requirements corresponding to the purpose is screened out in advance, and the models that are evaluated for accuracy after further learning can be reduced. For example, the resource condition may also include information on the operation frequency of the operation processing device of the embedded device, and the generation unit obtains the number of execution cycles of the part corresponding to the inference processing of the generated model, and calculates the required time using the number of execution cycles and the operation frequency. Thus, the model that cannot perform the specified processing within the required processing time is excluded from the object of accuracy evaluation. Therefore, it is possible to suppress the unnecessary execution of the subsequent prediction accuracy evaluation processing. In addition, more specifically, the generation unit may generate a first source code based on a language related to the operation processing device for the part corresponding to the inference processing of the model, and use the intermediate code obtained by compiling the first source code to obtain the number of execution cycles. In addition, for example, the neural network construction device may also include a learning unit and an output unit, the acquisition unit may also acquire learning data of the neural network, the judgment unit may output data representing a model judged to satisfy the second condition among the models generated by the generation unit, the learning unit may use the learning data to perform learning of the model represented by the data output by the judgment unit, and the output unit may output at least a portion of the model after learning.

[0062] By using such learning to determine parameters such as weights, it is possible to obtain a candidate for a neural network model that satisfies scale and performance constraints and is installed in a predetermined embedded device.

[0063] In addition, for example, the learning unit may further execute prediction accuracy evaluation of the learned model, and generate data related to the executed prediction accuracy evaluation.

[0064] Thus, it is possible to utilize information indicating the best candidate in terms of accuracy among the candidates for the model to be installed. In addition, more specifically, the learning unit may also generate a second source code based on a language related to the operation processing device for a portion corresponding to the inference processing of the learned model, and use the second source code to perform the prediction accuracy evaluation.

[0065] In addition, for example, the data related to the prediction accuracy evaluation may be data of a list of evaluation completed models representing models for which the prediction accuracy evaluation has been performed, and the generation unit, the judgment unit or the learning unit excludes from the processing object a model generated by using a plurality of hyperparameters in the same combination as any one of the models represented by the list of evaluation completed models.

[0066] This avoids processes such as generating models using the same combination of hyperparameters, and enables more efficient acquisition of candidates for neural network models.

[0067] In addition, for example, the output unit may output the output model in the form of source code based on a language related to the arithmetic processing device. In addition, for example, the output unit may output the output model in the form of a hardware description language.

[0068] In addition, for example, the judgment unit may stop the generation of the neural network model by the generation unit when the result of the prediction accuracy evaluation being executed satisfies a prescribed condition. More specifically, the acquisition unit may acquire an accuracy target representing a prescribed level of accuracy of the neural network model, the prescribed condition being a situation in which the result of the prediction accuracy evaluation does not meet the accuracy target in a prescribed number of models in a continuous generation order.

[0069] In the neural network construction device involved in this technology, although all combinations of hyperparameters that satisfy the scale constraint can be used to generate candidate models, it is sometimes foreseeable that when a certain degree of exploration is performed, the possibility of obtaining a more suitable model is not high even if further exploration is performed. In such a case, by stopping further generation of models, the reduction in cost-effectiveness for obtaining a more suitable model can be suppressed.

[0070] In addition, an information processing device according to one embodiment of the present invention includes a processing unit and a storage unit, wherein the storage unit stores a model generated by any one of the above-mentioned neural network construction devices, and the processing unit reads the model from the storage unit and executes it.

[0071] The information processing device obtained in this way can suppress the cost of design and development and have the desired accuracy.

[0072] In addition, for example, a neural network construction method involved in one embodiment of the present invention is a neural network construction method executed by an operation processing device in a neural network construction device including an operation processing device and a storage device, obtaining resource information related to computing resources possessed by an embedded device and performance constraints related to processing performance possessed by the embedded device, setting a scale constraint of the neural network based on the resource information, generating a model of the neural network based on the scale constraint, judging whether the generated model satisfies the performance constraint, and outputting data based on the result of the judgment.

[0073] Thus, by selecting from the candidates that have been narrowed down by excluding objects with structures that are unlikely to satisfy the conditions, the optimal neural network can be efficiently obtained.

[0074] In addition, for example, one embodiment of the present invention involves a program that is executed by a processing unit in a neural network construction device that includes a processing unit and a storage device, and the program is executed by the processing unit so that the neural network construction device: obtains resource information related to computing resources possessed by an embedded device and performance constraints related to processing performance possessed by the embedded device, sets a scale constraint of the neural network based on the resource information, generates a model of the neural network based on the scale constraint, determines whether the generated model satisfies the performance constraint, and outputs data based on the result of the determination.

[0075] Thus, by selecting from the candidates that have been narrowed down by excluding objects with structures that are unlikely to satisfy the conditions, the optimal neural network can be efficiently obtained.

[0076] Furthermore, these overall or specific aspects may be implemented through systems, integrated circuits, or computer-readable recording media such as CD-ROMs, or through any combination of devices, systems, methods, integrated circuits, computer programs, or recording media.

[0077] Hereinafter, the neural network construction device involved in the embodiment will be described with reference to the accompanying drawings. The embodiment in the present disclosure represents a specific example of the present invention, and the numerical values, constituent elements, configuration and connection methods of the constituent elements, and steps (processes) and the order of the steps represented are examples and do not limit the present invention. In addition, among the constituent elements in the embodiment, the constituent elements not included in the independent claims are constituent elements that can be added arbitrarily. In addition, each figure is a schematic diagram and is not necessarily a strict illustration.

[0078] (Implementation Method)

[0079] [constitute]

[0080] A plurality of embodiments are described below. First, the configuration of a neural network building device common to these embodiments is described.

[0081] Figure 2 1 is a block diagram showing an example of the functional configuration of the neural network building device 10 .

[0082] The neural network construction device 10 includes an acquisition unit 11 , a setting unit 12 , a generation unit 13 , a determination unit 14 , a learning unit 19 , and an output unit 15 .

[0083] The acquisition unit 11 acquires the condition information given to the neural network construction device 10 and the learning data used for learning the model of the generated neural network.

[0084] One of the conditions represented by the condition information is a condition used in determining the candidates for the hyperparameters of the neural network to be constructed (hereinafter also referred to as the first condition). The condition information also represents a condition related to the performance that the model of the neural network to be constructed should have (hereinafter also referred to as the second condition). The first condition and the second condition are described together in the detailed description of each embodiment.

[0085] The learning data is data used for learning the model of the neural network.

[0086] The acquisition unit 11 accepts condition information and learning data as user input, or reads and acquires condition information and learning data from a location accessed according to user operations or instructions of a prescribed program, or acquires condition information and learning data through calculations and other processing based on the information thus acquired.

[0087] The setting unit 12 determines candidate hyperparameters that are candidates for hyperparameters of the neural network to be constructed based on the first condition. This condition will be described later with examples.

[0088] The generation unit 13 generates a model of the neural network using the candidate hyperparameters determined by the setting unit 12 .

[0089] The determination unit 14 determines whether the neural network model generated by the generation unit 13 satisfies the second condition, and outputs data based on the result of the determination. For example, the determination unit 14 outputs list data indicating the models determined to satisfy the second condition.

[0090] The learning unit 19 performs learning of the model generated by the generating unit 13 using the learning data. The model to be learned is selected from the models shown in the list data output by the judging unit 14, for example. In addition, the learning unit 19 evaluates the prediction accuracy of the learned model, that is, the inference model, and outputs data related to the prediction accuracy evaluation. For example, the learning unit 19 outputs data indicating the results of the prediction accuracy evaluation of each inference model.

[0091] The output unit 15 outputs at least a part of the inference model. For example, referring to the results of the above-mentioned prediction accuracy evaluation represented by the data output by the learning unit 19, the data of the inference model that satisfies the prescribed conditions, such as the best results, is output. The user can obtain the inference model output from the output unit 15 as an inference model that satisfies the conditions represented by the condition information given to the neural network building device 10.

[0092] The neural network construction device 10 including these functional components is implemented by, for example, a personal computer, a server computer, or cloud computing (hereinafter, referred to as the computer 1 without distinguishing between them). Figure 3This is a block diagram for explaining an example of the hardware configuration of the computer 1 that realizes the neural network building device 10 .

[0093] The computer 1 includes an input device 2 , a processing device 3 , an output device 4 , a storage device 5 , and a communication device 6 , which are connected to each other via a bus 7 so as to be able to communicate with each other.

[0094] The input device 2 is, for example, a keyboard, a pointing device such as a mouse, or a touch screen, and receives instructions or data input from a user.

[0095] The arithmetic processing device 3 is various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor), which reads and executes a prescribed program stored in the storage device 5 to process information, and also controls each device as a hardware component to realize the above-mentioned functional components.

[0096] The output device 4 is a display device such as a monitor, and prompts the user to input data by displaying characters and graphics on the screen, or presents the progress status or results of the processing performed by the arithmetic processing device 3 .

[0097] The storage device 5 is a storage medium such as a RAM or a ROM, and stores the above-mentioned program, data referred to during execution of the program, and generated intermediate data and final data in a volatile or nonvolatile manner.

[0098] For example, when the computer 1 is implemented by cloud computing, the communication device 6 is a device including an input / output port for transmitting and receiving data between a plurality of computers, such as a network interface card.

[0099] In the neural network construction device 10 having such a hardware configuration, the above-mentioned functional components are realized by processing information or controlling each device by the operation processing device 3 executing the prescribed software. Using the information (data) acquired by the acquisition unit 11, the setting unit 12, the generation unit 13, the judgment unit 14, and the learning unit 19 perform a series of processes, and the learning model of the neural network suitable for the desired purpose is output from the output unit 15. The order of a series of processes before the output of the learning model of the neural network (hereinafter also referred to as the construction of the neural network) is described in the detailed description of each embodiment.

[0100] Next, a description will be given of how to obtain an optimal neural network model by using the condition information (first condition and second condition) mentioned in the description of the above-mentioned configuration in the neural network building device 10 .

[0101] [Conditions in the construction of neural networks]

[0102] Conventionally, in order to obtain the best neural network for a certain application, it is determined whether each candidate neural network satisfies the required conditions. Therefore, the number of trial and error iterations before obtaining the best neural network is huge and takes a long time.

[0103] The conditions used in constructing the neural network in the present invention can also be said to be constraints imposed on the constructed neural network.

[0104] The first condition is the constraints related to the structure (scale) of the neural network. For example, the neural network installed in the embedded device is executed by limited resources and hardware, and its execution environment is far more stringent than the environment for building the neural network. However, in the previous neural network construction method, a neural network of a scale that is not suitable for execution in such an embedded device is also generated and included in the objects of the above-mentioned determination.

[0105] Therefore, in the present invention, the upper limit as a constraint related to the scale of the neural network is calculated and set in advance based on the hardware-related information such as the frequency of the CPU and the size of the memory (ROM / RAM) in the execution environment of the neural network, and the neural network is generated. As a result, the time required for the generation and determination of a neural network that exceeds the upper limit can be saved. In addition, as another constraint related to the scale of the neural network, the minimum amount of calculation required to solve the problem to be solved using the neural network to be constructed, that is, the lower limit, can be calculated. By generating the neural network after setting the lower limit, the time required for the generation and determination of a neural network that does not meet the lower limit can be saved.

[0106] In addition, the information related to the hardware of the embedded device mentioned above and the required computing amount corresponding to the problem are examples of objects that can be used in the calculation of constraints related to the scale of the neural network. The constraints related to the scale of the neural network can also be calculated based on other indicators.

[0107] In addition, the second condition used in the construction of the neural network in the present invention is a constraint related to the performance of the neural network. The constraint is set with respect to the required accuracy or processing time, etc. As information based on the constraint, for example, information about the execution environment of the neural network (hardware information such as CPU frequency and memory size) can be used. For example, by using this information, the processing time required for the generated neural network to process the problem is calculated, and only the neural network whose processing time satisfies the constraint is used for learning using the learning data. In other words, the time required for learning a neural network with a long processing time can be saved.

[0108] In this way, by generating a neural network that satisfies the first condition, which is a constraint related to the scale of the neural network to be generated, and setting only the neural network that satisfies the second condition, which is a constraint related to the performance of the generated neural network, as the object of learning processing, it is possible to achieve the effect of reducing the time required to obtain the optimal neural network.

[0109] The difference between the conventional method and the method of the present invention using the above-mentioned constraints until the optimal neural network is obtained is described using the accompanying drawings. Figure 4 This is a diagram for explaining the concept of distribution of hyperparameters used in building a neural network.

[0110] In order to generate a neural network model, it is necessary to set hyperparameters such as the number of neurons and the number of layers. The structure of the generated neural network is determined by the values ​​of these hyperparameters, and the resources required for execution or the time required to process the problem are greatly affected by the structure. Figure 4 There are infinitely many values ​​for the hyperparameters represented by crosses in . In addition, for the sake of illustration, Figure 4 The rectangle in the figure shows the possible range of hyperparameters in this case, but the actual range is infinite. In other words, it will take more time to explore the optimal configuration of a neural network with infinite hyperparameters one by one.

[0111] In the present invention, for example, the range of hyperparameters to be generated is limited by using a constraint related to the scale as the upper limit and a constraint determined according to the problem as the lower limit. Figure 4 In the example, a neural network is generated using limited hyperparameters (candidate hyperparameters described later) in the shaded range. In addition, neural networks that do not satisfy performance-related constraints are excluded from the learning object. This can reduce the time required to obtain a neural network with the best configuration.

[0112] In addition, for the sake of convenience, the above description describes the hyperparameter as one type, but in fact, there may be multiple types, such as two types related to the number of neurons and the number of layers included in the neural network, and the candidate hyperparameters and hyperparameters in the description of the above and following embodiments should be appropriately interpreted as a combination of multiple types of hyperparameters.

[0113] Here, use Figure 5 The flowchart shown in the figure explains an example of a processing procedure of building a neural network executed in the neural network building device 10 having the above-mentioned configuration.

[0114] First, the acquisition unit 11 acquires condition information (first condition, second condition) and learning data used in neural network construction (S501). The acquisition unit 11 acquires the condition information by calculating the condition information using information prepared by the user based on the purpose of the neural network to be constructed and input to the neural network construction device 10, for example. Alternatively, the acquisition unit 11 may acquire information input to the neural network construction device 10 after the user has performed the calculation as the condition information. The learning data is also prepared by the user based on the purpose of the neural network to be constructed and input to the neural network construction device 10, or is placed on a server accessible to the neural network construction device 10.

[0115] Next, the setting unit 12 determines a candidate hyper-parameter using the condition information (S502). The candidate hyper-parameter may be determined by, for example, setting a range in which the value can be taken.

[0116] Next, the generator 13 generates a list of candidate hyperparameters determined in step S502 (hereinafter also referred to simply as a candidate list) (S503).

[0117] Next, the generator 13 searches for the best candidate hyperparameter from the candidate list, and generates a neural network model using the searched candidate hyperparameter (S504). This search can use, for example, a Bayesian optimization method. In this method, it is assumed that the distribution of the prediction accuracy of the neural network model is based on a normal distribution, and the hyperparameter is searched from the candidate list using a posterior distribution calculated based on the distribution of the prediction accuracy.

[0118] Fig. 6A , Figure 6B and Figure 6CIt is a diagram for explaining the overview of the exploration method using Bayesian optimization of hyperparameters. The curves shown in each figure show the correspondence between the value of the hyperparameter and the assumed prediction accuracy of the model generated using the hyperparameter. Each hyperparameter included in the candidate list is located somewhere on the horizontal axis of the curve graph area. The thick solid curve on the curve graph represents the expected value of the prediction accuracy obtained by Bayesian optimization for each hyperparameter. In addition, the dotted curve represents the ideal value that should be obtained as an evaluation point for each hyperparameter. In addition, each black dot and white circle represents an evaluation point for the evaluation of the prediction accuracy of one hyperparameter performed by the learning unit 19 described later. The shaded area will be described later. Fig. 6A , Figure 6B and Figure 6C They represent the three stages of the chronological order in this method.

[0119] In the initial stage of this exploration, there are no evaluation points or very few evaluation points, so there are many unevaluated neural network models, that is, many unevaluated hyperparameters. Therefore, the uncertainty of the expected value of the prediction accuracy is large. The shaded area in each figure represents the range of prediction accuracy with a certain probability or more for each hyperparameter obtained as a posterior distribution. Fig. 6A In the figure, since it is still in the initial stage, the shadow area is relatively large.

[0120] In the next stage, the hyperparameter with large uncertainty is selected to generate a model and evaluate its prediction accuracy. The prediction accuracy evaluation point (white circle) is newly obtained, and the prediction accuracy distribution is updated based on the normal distribution. Then the uncertainty is further updated, and the model is generated by the hyperparameter with large uncertainty after the update and evaluated. By repeating this process, the uncertainty corresponding to the overall hyperparameter becomes smaller. This is achieved by comparing Fig. 6A , Figure 6B and Figure 6C The size of the shaded area can also be known. In this way, while reducing uncertainty, explore hyperparameters with higher prediction accuracy. In addition, if the uncertainty decreases to a certain extent as the exploration proceeds, the exploration is concentrated near the hyperparameters with high prediction accuracy that have been evaluated.

[0121] In addition, in such a method, a search method that takes into account the suitability corresponding to the constraints indicated by the condition information may be used.

[0122] Next, the determination unit 14 checks whether the neural network search has been completed once using all the candidate hyperparameters in the candidate list (S505). If not, the process proceeds to step S506, and if completed, the process transfers to step S510 described later.

[0123] If the answer is no in step S505, the determination unit 14 checks whether the model generated in step S504 is a model for which the prediction accuracy evaluation has been completed (S506). This check is performed based on the evaluation completed model list generated by the learning unit 19 described later. If the evaluation has not been completed, the process proceeds to step S507, and if the evaluation has been completed, the process proceeds to step S510 described later.

[0124] Next, the learning unit 19 executes learning of the unevaluated model using the learning data acquired in step S501 ( S507 ).

[0125] Next, the learning unit 19 evaluates the prediction accuracy of the learned model (inference model) (S508), and adds the evaluated inference model to the evaluated model list (S509). In step S506, the judgment unit 14 uses the evaluated model list representing the model after learning by the learning unit 19 and the evaluation of the prediction accuracy. In addition, the evaluated inference model is stored in the storage device 5 as the inference model output from the neural network construction device 10.

[0126] Finally, the output unit 15 outputs the evaluated inference model saved in the storage device 5 in step S509 (S510). However, the output object is not limited to this, and may also be the inference model with the highest prediction accuracy and all inference models that meet the second condition. In addition, for example, when there is no inference model that meets the second condition, the output unit 15 may also output a warning. In addition, the output here refers to, for example, displaying on the output device 4 such as a display, or writing to a specified storage location outside the storage device 5 or the neural network construction device 10.

[0127] At this point, the processing of the neural network construction method executed in the neural network construction device 10 is completed. Note that the above-mentioned processing procedure is an example, and various modifications are possible.

[0128] For example, if the answer is “Yes” in step S505 or “Yes” in step S506 , the process ends after the output of step S510 , but the order before the end is not limited to this.

[0129] For example, in step S506, it is determined whether the result of the accuracy evaluation meets the specified conditions, and the output of step S510 is performed in accordance with the result of the determination. As an example of a specified condition, it may be a situation where the result of the prediction accuracy evaluation of a specified number of models in a continuous generation order does not reach the accuracy target, or a situation where the change in the result of the prediction accuracy evaluation of a specified number of models in a continuous generation order does not increase by more than a specified size. This corresponds to the following situation: it can be expected that at the moment when a certain degree of exploration is carried out, even if further exploration is carried out, the possibility of obtaining a more suitable model is not high. In such a case, by stopping further model generation and stopping exploration, the time for obtaining a model suitable for the desired purpose can be shortened, thereby suppressing the reduction in cost performance. As another example, it may be that the number of models that meet a certain accuracy target reaches a specified value.

[0130] In addition, in the judgment of step S505, it is also possible to judge whether the exploration using a predetermined number or ratio or more is completed instead of whether the exploration using all hyperparameters is completed. Alternatively, when the exploration using Bayesian optimization is carried out to a certain extent and the uncertainty is reduced to a certain extent, the vicinity of the hyperparameter with low prediction accuracy that has been evaluated is excluded from the objects of exploration and then the judgment of step S505 is performed.

[0131] In addition, in step S509 or S510, the results of the prediction accuracy evaluation may also be saved or output. The results may be saved, for example, in a part of the evaluation completed model list or in another list. Alternatively, the evaluation completed model list or the other list may also include information equivalent to whether the accuracy of each inference model reaches the target or the achievement rate of each inference model.

[0132] In addition, instead of using the evaluation completed model to perform confirmation in step S506, confirmation may be performed based on (a combination of) hyperparameters that have been extracted from the candidate list or the individual list.

[0133] In addition, more detailed examples of constraints and the like are also given in the description of each embodiment below.

[0134] [Implementation Method 1]

[0135] So far, several types of examples have been given about the conditions (constraints) in the construction of a neural network. In each embodiment described below, these types of constraints are described using specific examples. As embodiment 1, constraints determined according to the problem to be solved using a neural network are described.

[0136] <Example of upper limit determined according to the problem>

[0137] When using Figure 7 When the fully coupled neural network of the configuration example is used for classification or regression reasoning, the model is designed to abbreviate the input data. Therefore, the upper limit of hyperparameters such as the number of layers and the number of nodes in the intermediate layer can be determined based on the input dimension and the output dimension. That is, the upper limit of the number of nodes in each intermediate layer is the number obtained by subtracting 1 from the number of nodes in the previous layer. In addition, the upper limit of the number of intermediate layers is the following number: the number of intermediate layers whose number of nodes is successively reduced by 1 from the intermediate layer containing one less node than the input layer, until the intermediate layer containing one more node than the output layer can be configured.

[0138] In addition, when using convolutional neural networks for classification or regression, such as Figure 8 As shown in the configuration example, the model is designed so that the feature image (also called feature map) after convolution or pooling is smaller than the size of the input to each convolution layer (numbers such as "30×30" in the figure). Therefore, the upper limit of the number of intermediate layers is determined within the range that can maintain the size of the convolutional feature image.

[0139] <Example of lower limit determined according to the problem>

[0140] When using a convolutional neural network to restore an image (such as removing noise), by assigning frequency characteristics to the components that you want to cut off (or the components that you want to pass) through the neural network, you can determine the lower limit of hyperparameters such as the number of intermediate layers of the neural network to be generated or the kernel size of each layer. The setting of this lower limit is explained using a specific example.

[0141] When a noise removal filter that "cuts off noise above a cutoff frequency f by g% or more (hereinafter referred to as condition X)" is generated as a neural network, the lower limit of the hyperparameter is determined in the following procedure.

[0142] (Step 1) Find a single low-pass filter that satisfies condition X.

[0143] Here, a low-pass filter means a filter that allows components with frequencies lower than the cutoff frequency to pass through with little attenuation, while cutting off components with frequencies higher than the cutoff frequency. Although a pure low-pass filter cannot simply filter out noise and cut it off, this sequence is performed in order to serve as a benchmark for evaluating the upper limit of the desired noise cutting performance.

[0144] The frequency characteristic of the low-pass filter |O / I| is shown in the following equation 1, which is calculated based on the filter kernel size n, frequency ω, and kernel coefficient k. i (0≤i≤n-1) is found.

[0145] [Number 1]

[0146]

[0147] Here, if the kernel coefficient ki is assumed to be Gaussian distributed (so-called Gaussian filter), then when the kernel size n=3, the frequency characteristic |O / I| is as shown in the frequency characteristic of the low-pass filter. Fig. 9 As shown by the solid line curve in the curve graph, it becomes a cosine curve with an amplitude of 0 at the Nyquist frequency fN (that is, the component of the Nyquist frequency is 100% cut off). This low-pass filter with a frequency characteristic of 50% cut off at 0.5fN satisfies the condition X when f=0.5fN, g=40%, but does not satisfy the condition X when f=0.5fN, g=60%. In addition, the frequency characteristic |O / I| of the low-pass filter with a kernel size of n=5 is as follows Fig. 9 The low-pass filter cut off at 75% at 0.5fN also satisfies condition X when f=0.5fN and g=60%.

[0148] In this way, by assuming the distribution of kernel coefficients, the lower limit of the kernel size n of a single low-pass filter that satisfies the condition X can be determined.

[0149] (Order 2) Decompose a single low-pass filter into a convolutional neural network.

[0150] Consider constructing the single low-pass filter obtained in procedure 1 by connecting multiple filters in series. For example, as shown in equation 2, a Gaussian filter with a kernel size of n=5 can be constructed by connecting two stages of Gaussian filters with a kernel size of n=3.

[0151] [Number 2]

[0152]

[0153] Similarly, as shown in the following equation 3, a filter with a kernel size of n can be constructed by connecting m stages of filters with a kernel size of n′.

[0154] [Number 3]

[0155] n=1+(n′-1)×m…(Formula 3)

[0156] Here, m corresponds to the number of intermediate layers (convolutional layers) of the convolutional neural network, and by changing it to follow the increase or decrease of the kernel size n′, a frequency characteristic equivalent to a filter with a kernel size of n is achieved.

[0157] In this way, by determining the lower limit of the kernel size n of a single low-pass filter according to condition X in order 1, and further determining the combination of the filter kernel size n′ and the number of intermediate layers m in order 2, the lower limit of the hyperparameters of the convolutional neural network to be constructed can be determined.

[0158] In addition, if the convolutional neural network is used as a pure low-pass filter, the convolutional neural network composed of connecting 2 levels of n=3 kernels can maintain performance and suppress the amount of calculation compared to the convolutional neural network with a kernel of n=5 in the first level. However, the convolutional neural network finally constructed is used as a noise removal filter, and it cannot be said that the latter is necessarily superior in noise removal performance. The hyperparameters determined in this way are candidate hyperparameters that are candidates for the hyperparameters of the convolutional neural network finally constructed, and the best model of the convolutional neural network is obtained by evaluating each model generated using the candidate hyperparameters.

[0159] In the above, the method of determining the upper limit or lower limit of the hyperparameter according to the problem to be solved by using the neural network is described using a specific example. Next, the processing sequence for implementing this method by the neural network construction device 10 is described. Figure 5 The flowchart of the present embodiment will be used to explain the processing sequence in more detail. Figure 5 The common parts of the description are sometimes simplified.

[0160] First, the acquisition unit 11 acquires condition information and learning data used in neural network construction (S501). The condition information is, for example, information related to the problem to be solved using the convolutional neural network. In the case of the above-mentioned method, the dimension of the input data and the dimension of the output data used in setting the upper limit of the hyperparameter, or the size of the input image, and the truncation frequency f and the minimum truncation rate g used in setting the lower limit of the hyperparameter can be used as the first condition. Based on such information, the acquisition unit 11 calculates the upper limit or lower limit or both of the candidate hyperparameters of the neural network to be constructed to acquire the first condition.

[0161] Next, the setting unit 12 determines a candidate hyperparameter (S502). The candidate hyperparameter determined here is, for example, a hyperparameter having a value greater than or equal to the lower limit obtained by the obtaining unit 11, a hyperparameter having a value less than or equal to the upper limit, or a hyperparameter having a value greater than or equal to the lower limit and less than or equal to the upper limit.

[0162] Next, the generating unit 13 generates a candidate list ( S503 ).

[0163] Next, the generation unit 13 searches for the best candidate hyperparameter from the above-mentioned candidate list, and generates a model of the neural network using the explored candidate hyperparameter (S504). In the case where the candidate hyperparameter included in the candidate list is a hyperparameter that takes a value below the upper limit, for example, the above-mentioned exploration method using Bayesian optimization can also be used. In the case where the candidate hyperparameter included in the candidate list is a hyperparameter that takes a value above the lower limit, for example, a neural network having a structure determined by the lower limit hyperparameter is used as a basis, and a neural network having a structure with an increased number of nodes or layers to ensure higher performance is generated to explore the optimal point. For example, a genetic algorithm can also be used to update the structure of the neural network to explore the optimal point.

[0164] Step S505 and thereafter are processed in the same manner as described above.

[0165] [Implementation Method 2]

[0166] As a second embodiment, a case where a neural network is mainly implemented in an embedded device and information of a CPU and a memory (ROM / RAM) is input as condition information will be described.

[0167] Fig.10 1 is a flowchart of the processing sequence in this embodiment performed by the neural network building device 10. Figure 5 The steps of the processing sequence shown in the flowchart are denoted by common reference numerals and the description is sometimes simplified.

[0168] First, the acquisition unit 11 acquires condition information and learning data used for constructing a neural network ( S501 ).

[0169] The condition information includes resource information such as the CPU frequency, memory (ROM, RAM) size, and memory transfer speed of the embedded device. The information included in the resource information is not limited to this, and may also include other information related to the embedded device. The resource information is an example of the first condition in the present embodiment. In addition, the condition information includes conditions related to the performance when the neural network is executed in the embedded device (also referred to as performance constraints in the present embodiment). As an example of performance constraints, the processing time set as the target can be information related to various performances required for the processing executed in the embedded device. The performance constraint is an example of the second condition in the present embodiment. Such performance constraints use, for example, performance constraints prepared by the user based on the specifications of the embedded device or the product into which the embedded device is assembled and input into the neural network construction device 10.

[0170] Next, the setting unit 12 determines the candidate hyperparameters based on the resource information. (S502) For example, the setting unit 12 can calculate the range of values ​​that the candidate hyperparameters of the fully coupled neural network can take based on the known ROM size using the following formula 4.

[0171] [Number 4]

[0172]

[0173] In formula 4, S ROM Indicates the ROM size, N Li represents the number of neurons in each layer, S DATA Indicates the size of the data type to be processed. In addition, the ROM size is based on S DATA changes, so by dividing by S DATA , which can calculate the maximum number of coupling loads that can be embedded in the neural network corresponding to each data type.

[0174] Next, the generator 13 generates a candidate list including the candidate hyper-parameters determined in step S502 ( S503 ).

[0175] Next, the generator 13 searches for hyperparameters for determining the configuration of the neural network suitable for the embedded device from the candidate list, and generates a neural network model based on the searched candidate hyperparameters (S504). The search uses, for example, the above-mentioned Bayesian optimization method.

[0176] Next, the generator 13 converts the part corresponding to the inference processing of the neural network and generates a temporary source code (S515). The model of the neural network has been constructed by Python, which is a high-level language, until this stage, but is converted into a source code of a language more related to (more dependent on) the operation processing device, such as C language, in this step. The purpose of such conversion is to obtain a more accurate required time by setting it to a language widely used as a program for embedded devices, in this case C language, as a preparation for calculating the processing time in the next step, so as to be close to the actual execution environment.

[0177] Next, the generator 13 uses the source code obtained by the conversion in step S515 to calculate the time required for the inference processing (S516). More specifically, the generator 13 uses the intermediate code generated by compiling the source code to obtain the number of execution cycles required for the inference processing. Then, the generator 13 further uses the information that affects the processing time, such as the operation frequency of the arithmetic processing device included in the resource information obtained in step S501, to calculate the time required for the processing of the number of execution cycles.

[0178] Next, the determination unit 14 determines whether the required time calculated in step S516 satisfies the second condition included in the condition information obtained in step S501, that is, the processing time set as the target as the performance constraint (S517). If the performance constraint is not satisfied (S517: "No"), the model is discarded (S518). After discarding the model, it is confirmed whether the exploration of the neural network has been completed once using all the candidate hyperparameters in the candidate list (S505). If not completed, the processing order returns to step S504, and if completed, it transfers to step S510 described later.

[0179] In addition, if the performance constraint is satisfied (S517: Yes), it is confirmed whether the model is a model for which the prediction accuracy evaluation has been completed (S506). This confirmation is performed based on the evaluation completed model list generated by the learning unit 19 described later. If the evaluation has not been completed, the process proceeds to the next step S507. If the evaluation has been completed, the process proceeds to the step S510 described later.

[0180] Next, the learning unit 19 performs model learning using the learning data acquired in step S501 ( S507 ).

[0181] Next, the learning unit 19 converts the learned model (inference model) to generate source code (S525). The purpose of converting to source code here is basically the same as step S515, which is to get closer to the actual execution environment. Therefore, for example, a model constructed by Python is converted into source code in C language. However, this is not for evaluating the processing time, but for confirming the prediction accuracy of the inference model in an environment close to the actual embedded device. In addition, the source code generated by conversion here, which is based on a language that is more relevant to the operation processing device (more dependent on the operation processing device), such as C language, is saved in the storage device 5 as an inference model output from the neural network construction device 10.

[0182] Next, the learning unit 19 uses the source code obtained by the conversion in step S525 to evaluate the prediction accuracy of the inference model (S508). When the evaluation is completed, the learning unit 19 adds the inference model as an evaluation completed model to the evaluation completed model list (S509). In step S506, the judgment unit 14 uses the evaluation completed model list indicating the model after the learning unit 19 performs learning and performs the evaluation of the prediction accuracy.

[0183] When the evaluation of each model is completed, the output unit 15 outputs the source code of the inference model stored in the storage device 5. However, the output object is not limited to this, and as described above, the model that satisfies the specified conditions among the stored multiple models may be output, or the prediction accuracy results of each inference model may be output. In addition, when there is no inference model that satisfies the performance constraint as the second condition, the output unit 15 may output a warning.

[0184] At this point, the processing of the neural network building method in this embodiment executed by the neural network building device 10 is completed.

[0185] In addition, the above processing order is an example and can be modified in various ways. Figure 5 Various modifications of the processing order can also be applied to the processing order of this embodiment.

[0186] [Implementation method 3]

[0187] Implementation 3 is also the case where the neural network is mainly installed in an embedded device, similar to Implementation 2, and the description will focus on the differences from Implementation 2.

[0188] In this embodiment, in the extraction of hyperparameters in the exploration of neural networks, instead of initially using Bayesian optimization, prediction accuracies corresponding to multiple hyperparameters are obtained once by a method that does not use Bayesian optimization, and Bayesian optimization is performed using the prediction accuracies as prior distribution.

[0189] Fig.11 and Fig.12 1 is a flowchart of the processing sequence in this embodiment performed by the neural network building device 10. Figure 5 or Fig.10 The steps of the processing sequence shown in the flowchart are denoted by common reference numerals and the description is sometimes simplified.

[0190] The acquisition of condition information and learning data by the acquisition unit 11 ( S501 ), the determination of candidate hyperparameters by the setting unit 12 ( S502 ), and the generation of a candidate list by the generation unit 13 ( S503 ) are common to the second embodiment.

[0191] Next, the generator 13 randomly extracts the candidate hyperparameters from the candidate list and generates a model of a neural network based on the extracted candidate hyperparameters (S604). The reason for generating the model of the neural network using the extracted candidate hyperparameters is that the prediction accuracy of multiple models generated using the explored candidate hyperparameters as in Implementation 2 may be roughly the same and cannot be said to be necessarily high. Therefore, by generating a model of a neural network based on appropriately distinguishing the candidate hyperparameters selected using the method used in Implementation 2 and the method used in this implementation, it is possible to generate models with different accuracies more efficiently.

[0192] The generation of source code ( S515 ) and calculation of the time required for inference processing ( S516 ) subsequently performed by the generation unit 13 are common to the second embodiment.

[0193] Next, the judgment (S517) related to the performance constraint performed by the judgment unit 14 is the same as that of the second embodiment, but a part of the next sequence performed according to the result is different. The discarding of the model when the performance constraint is not satisfied (S517: "No", S518) is the same as that of the second embodiment. However, when the performance constraint is satisfied (S517: "Yes"), the confirmation of whether it is an evaluation completed model (step S506 of the second embodiment) is not performed, and the process is transferred to the processing performed by the learning unit 19.

[0194] Next, the learning of the model ( S507 ), the generation of source code ( S525 ), the evaluation of prediction accuracy ( S508 ), and the addition to the evaluated model list ( S509 ) performed by the learning unit 19 are common to the second embodiment.

[0195] Next, in the second embodiment, the process moves to the exploration of the next candidate hyperparameter and the generation of the model (S504), but in the present embodiment, the determination unit 14 determines whether the number of inference models that have completed the prediction accuracy evaluation has reached a predetermined number (S606).

[0196] This prescribed number is also the number of elements of the prior distribution used in the order of hyperparameter exploration based on Bayesian optimization described later, and various determination methods can be used. For example, the judgment unit 14 can also determine it by calculation according to the number of candidate hyperparameters. More specifically, it can also be dynamically determined in a manner that the greater the number of candidate hyperparameters, the larger the number. Alternatively, the prescribed number can also be determined by the user, and the acquisition unit 11 acquires the value input by the user to the neural network construction device 10 as the prescribed number, and the judgment unit 14 uses it.

[0197] If the number of inference models that have been evaluated has not been reached (S606: No), the processing sequence returns to step S604, and the generator 13 extracts the next candidate hyperparameter and generates a neural network model. If it has been reached (S606: Yes), the process proceeds to the next process (S1504) performed by the generator 13.

[0198] On the other hand, after discarding the model (S518), the judgment unit 14 confirms whether the extraction of the neural network has been completed once using all the candidate hyperparameters in the candidate list (S605). If it has not been completed (S605: "No"), the processing order returns to step S604, and the generation unit 13 extracts the next candidate hyperparameter and generates a model of the neural network. If it has been completed (S605: "Yes"), it is transferred to the output performed by the output unit 15 ( Fig.12 S510 is common to implementation mode 2).

[0199] If step S606 is "yes", the prediction accuracy evaluation is completed for the specified number of inference models that satisfy the performance constraint, and then exploration based on Bayesian optimization using the prediction accuracy of these inference models as a prior distribution is performed (S1504). Fig.12 The flowchart of shows an example of the subsequent processing sequence including this exploration. Fig.12 Step S1504 in the embodiment 2 is equivalent to step S504 in the embodiment 2, and step S1515 is equivalent to step S515 in the embodiment 2. Hereinafter, steps S516 to S518, S505 to S507, S525, S508, and S509 in the embodiment 2 are similarly executed as steps S1516 to S1518, S1505 to S1507, S1525, S1508, and S1509 in the present embodiment.

[0200] On the other hand, when step S606 is "No", the number of models that meet the performance constraints among the models generated using the candidate hyperparameters extracted from the candidate list does not reach the specified number. In this case, in step S510, for example, the output may be performed by prompting the user with a notification indicating this situation or by recording in a log information (score) related to the prediction accuracy of the models included in the information evaluation completed model list. In addition, when no model exists in the information evaluation completed model list, the output of step S510 may also be performed by prompting the user with a warning indicating this situation.

[0201] At this point, the processing of the neural network building method in this embodiment executed by the neural network building device 10 is completed.

[0202] In addition, the above processing order is an example and can be modified in various ways. Figure 5Various modifications of the processing order can also be applied to the processing order of this embodiment.

[0203] In addition, in the description of step S604, the extraction method of extracting candidate hyperparameters from the candidate list is set to random, but it is not limited to this. For example, the first one can be arbitrarily selected from the candidate hyperparameters arranged in ascending or descending order of value, and then the candidate hyperparameters with a predetermined interval can be extracted. Alternatively, the extracted candidate hyperparameters can also be manually selected by the user. Such a method does not rely on the posterior distribution, so it can obtain the same effect as random extraction.

[0204] (Other Implementations, etc.)

[0205] As described above, each embodiment is described as an example of the technology involved in the present invention. However, the technology involved in the present invention is not limited to the content of this description, and can also be applied to the embodiment that is appropriately changed, replaced, added, omitted, etc. For example, the following modified example is also included in one embodiment of the present invention.

[0206] (1) In the above embodiment, Python is cited as a language used in constructing a neural network model, and C language is cited as a language for a model operating in an embedded device, but these are examples of languages ​​used in common design and development environments, and are not limited to these languages. For example, in the calculation of processing time, simulation may be performed in a manner that is as close as possible to the execution environment of an actually used embedded device, including the language.

[0207] (2) The memory size, which is one of the first conditions for determining the upper limit of the scale of the model, is not limited to a single value without a range. For example, when there are multiple candidates for the adopted embedded devices with different memory sizes, a range of memory sizes including these embedded devices can be assigned. In this case, as a result of the prediction accuracy evaluation, for example, the correspondence between the memory size and the achievable prediction accuracy can also be expressed. The same is true for the first condition other than the memory size. For example, when a range of the operating speed of the computing processing unit is assigned, the correspondence between the operating speed and the processing time can also be expressed.

[0208] (3) The division of functions among the functional components of the neural network building device described in the above embodiment is merely an example, and the division of functions may be arbitrarily changed.

[0209] (4) The order of various processes shown in the above embodiments (for example Figure 5 , Figures 10 to 12The execution order of the steps (such as the order represented by the example) is not necessarily limited to the order described above. The execution order can be changed, or multiple orders can be performed in parallel, or part of the order can be omitted without departing from the scope of the invention. For example, the confirmation of whether the model is an evaluation completed as step S506 in embodiment 1 can also be implemented between steps S504 and S505. In addition, the confirmation of whether the model is an evaluation completed as step S506 in embodiment 2 can also be implemented between steps S504 and S515, between steps S515 and S516, or between steps S516 and S517. In this case, when the model is an evaluation completed, the generation of source code (S515), the calculation of the required time for inference processing (S516), or the judgment related to performance constraints (S517) can also be skipped. In addition, the judgment of comparing the results of the accuracy evaluation with the prescribed conditions, which is cited as another example of the judgment performed in step S510 in embodiment 2, can also be performed immediately after step S508 or step S509. Then, if the specified conditions are met, the output of step S510 may be performed. In the case of such a modified processing order, step S506 may be omitted. Fig.12 The processing order of implementation mode 3 shown in can also be applied.

[0210] (5) In the above description of the embodiment, an example is given in which the output unit 15 outputs the inference model in the form of source code based on a language related to the operation processing device, but as an example of another form, the inference model may be output after being further converted into a hardware description language. In this way, the constructed inference model can be implemented by hardware based on a dedicated logic circuit.

[0211] (6) In the description of the above embodiment, the depth and number of nodes of the neural network are cited as candidate hyperparameters as objects determined by the setting unit 12, but the present invention is not limited thereto. The setting unit 12 may, for example, treat other parameters related to the depth of the neural network in the convolutional neural network as hyperparameters in the present invention and also make decisions related to them. As more specific examples of such parameters, the size of the kernel, the depth of the kernel, the size of the feature map, the window size of the pooling layer, the amount of padding, and the amount of step length may be cited.

[0212] (7) Part or all of the components constituting each device in the above-mentioned embodiment may be constituted by a system LSI (Large Scale Integration). System LSI is a multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system composed of a microprocessor, ROM, RAM, etc. Computer programs are recorded in the RAM. The system LSI achieves its function by the microprocessor operating according to the computer program.

[0213] In addition, each part of the components constituting the above-mentioned devices can be individually single-chip or in a manner that includes part or all of them. In addition, it is set as system LSI here, but depending on the difference in integration, it is sometimes called IC, LSI, super LSI, and special LSI. In addition, the method of integrated circuit is not limited to LSI, and can also be implemented by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or a reconfigurable processor that can reconfigure the connection or setting of the circuit unit inside the LSI. Furthermore, if a technology for integrated circuitization that replaces LSI emerges with the advancement of semiconductor technology or other derived technologies, it is of course possible to use this technology for the integration of functional modules. The application of biotechnology, etc. is possible.

[0214] (8) A part or all of the components constituting each of the above-mentioned devices may also be constituted by an IC card or a single module that can be detached from each device. The IC card or the module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or the module may also include the above-mentioned super-multifunctional LSI. The IC card or the module achieves the function through the microprocessor operating according to the computer program. The IC card or the module may also have tamper resistance.

[0215] (9) As one embodiment of the present invention, for example, Figure 5 , Figures 10 to 12 A method for constructing a neural network including all or part of the processing sequence shown. For example, the method for constructing a neural network is a method for constructing a neural network executed by a processing unit in a neural network construction device having a processing unit and a storage unit, and includes the following steps: obtaining resource information related to computing resources of an embedded device and performance constraints related to processing performance of the embedded device; setting a scale constraint of the neural network based on the resource information; generating a model of the neural network based on the scale constraint; and determining whether the generated model satisfies the performance constraint, and outputting data based on the result of the determination.

[0216] Furthermore, as one embodiment of the present invention, a program (computer program) for realizing predetermined information processing involved in the neural network construction method by a computer may be used, or a digital signal constituted by the program may be used.

[0217] In addition, as one embodiment of the present invention, the above-mentioned computer program or digital signal may be recorded on a computer-readable recording medium, such as a floppy disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) disc), a semiconductor memory, etc.

[0218] Alternatively, the program or digital signal may be recorded in the recording medium. Alternatively, as one embodiment of the present invention, the program or digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a communication network represented by the Internet, data broadcasting, etc.

[0219] In addition, as one embodiment of the present invention, a computer system having a microprocessor and a memory may be provided, the memory having the above-mentioned program recorded therein, and the microprocessor operating in accordance with the above-mentioned program. In addition, the above-mentioned program or the above-mentioned digital signal may be recorded on the above-mentioned recording medium and transferred, or the above-mentioned program or the above-mentioned digital signal may be transferred via the above-mentioned communication network, etc., so as to be implemented by another independent computer system.

[0220] In addition, as one embodiment of the present invention, an information processing device may be provided that executes a model of a neural network generated by using the apparatus, method, or program involved in the above-mentioned embodiment or its modified example. The information processing device includes a processing unit and a storage unit, in which the model is written, and the processing unit reads and executes the model. For example, the following ECU (Electronic Control Unit) is assumed, which includes a model that takes an image obtained by an image sensor as input and outputs information representing the recognized object.

[0221] (10) A mode achieved by arbitrarily combining the components and functions described in the above-described embodiment and the above-described modified examples is also included in the scope of the present invention.

[0222] Industrial Applicability

[0223] The present invention can be used as a technology for obtaining more appropriate model candidates in a short time when constructing a neural network model.

[0224] Description of reference numerals:

[0225] 1 Computer

[0226] 2 Input Devices

[0227] 3. Processing device

[0228] 4 Output Devices

[0229] 5 Storage Devices

[0230] 6 communication devices

[0231] 7 Bus

[0232] 10 Neural Network Construction Device

[0233] 11 Acquisition Department

[0234] 12. Setting section

[0235] 13. Generation

[0236] 14 Judgment Department

[0237] 15 Output

[0238] 19 Learning Department.

Claims

1. A neural network construction device, comprising: an acquisition unit that acquires a first condition and a second condition, wherein the first condition is a condition used to determine a candidate hyperparameter, the candidate hyperparameter being a candidate for a hyperparameter of a neural network to be constructed, and the second condition is a condition related to performance that a model of the neural network should have; a setting unit, which determines the candidate hyperparameter using the first condition; a generating unit that generates a model of a neural network using the candidate hyperparameters and taking the image data acquired by the image sensor as input data; as well as a judgment unit that judges whether the generated model satisfies the second condition and outputs data based on the result of the judgment, The setting unit calculates at least one of an upper limit and a lower limit of the candidate hyperparameter using the first condition, and determines one or more candidate hyperparameters based on at least one of the calculated upper limit and lower limit. The first condition includes information on at least one of the size of the input data to the neural network and the size of the output data from the neural network. The setting unit calculates an upper limit of the candidate hyperparameter based on at least one of the size of the input data and the size of the output data included in the first condition, and determines at least a portion of the calculated hyperparameters below the upper limit as the one or more candidate hyperparameters.

2. The neural network construction device according to claim 1, The first condition also includes a resource condition related to the computing resources possessed by the embedded device. The setting unit further calculates an upper limit of the candidate hyper-parameters based on the resource conditions, and determines at least a part of the hyper-parameters below the upper limit as the candidate hyper-parameters.

3. The neural network construction device according to claim 2, The resource condition includes information about the memory size of the embedded device. The setting unit calculates an upper limit of the hyperparameters of the neural network that can be accommodated by the memory size as an upper limit of the candidate hyperparameters, and determines at least a part of the hyperparameters below the upper limit as the candidate hyperparameters.

4. The neural network construction device according to claim 1, The size of the input data is the dimension of the input data, the size of the output data is the dimension of the output data, The one or more candidate hyperparameters include one or more numbers of layers and nodes of the neural network.

5. The neural network construction device according to claim 1, The first condition also includes information indicating that the neural network is a convolutional neural network.

6. The neural network construction device according to claim 5, The size of the input data is the number of pixels of the image data, the size of the output data is the number of classes into which the image data is classified, The one or more candidate hyperparameters include at least one of the number of layers of the convolutional neural network, the size of the kernel, the depth of the kernel, the size of the feature map, the window size of the pooling layer, the padding amount, and the stride amount.

7. The neural network construction device according to any one of claims 1 to 6, The first condition comprises an accuracy target for inference based on the neural network model, The setting unit calculates a lower limit of the candidate hyperparameters using the accuracy target, and determines at least a portion of the calculated hyperparameters above the lower limit as the one or more candidate hyperparameters.

8. The neural network construction device according to claim 2 or 3, The second condition includes a time condition related to a reference required time for inference processing of a model using a neural network, The generating unit calculates a time required for an inference process using the generated model based on the resource condition, The determination unit determines whether the generated model satisfies the second condition by comparing the calculated required time with the reference required time.

9. The neural network construction device according to claim 8, The resource condition also includes information on the operating frequency of the computing processing device of the embedded device. The generating unit obtains the number of execution cycles of a portion corresponding to inference processing of the generated model, and calculates the required time using the number of execution cycles and the operation frequency.

10. The neural network construction device according to claim 9, The generation unit generates a first source code in a language related to the arithmetic processing device for a portion corresponding to inference processing of the model, and obtains the number of execution cycles using an intermediate code obtained by compiling the first source code.

11. The neural network construction device according to any one of claims 1 to 6, The neural network construction device further comprises a learning unit and an output unit. The acquisition unit further acquires learning data of the neural network. the judging unit outputs data indicating a model judged to satisfy the second condition among the models generated by the generating unit, The learning unit performs learning of a model represented by the data output by the determination unit using the learning data, The output unit outputs at least a portion of the learned model.

12. The neural network construction device according to claim 11, The learning unit further executes prediction accuracy evaluation of the learned model and generates data related to the executed prediction accuracy evaluation.

13. The neural network construction device according to claim 12, The learning unit further generates a second source code in a language related to the arithmetic processing device for a portion corresponding to the inference process of the learned model, and executes the prediction accuracy evaluation using the second source code.

14. The neural network construction device according to claim 12, The data related to the prediction accuracy evaluation is data indicating a list of evaluated models for which the prediction accuracy evaluation has been completed. The generation unit, the determination unit, or the learning unit excludes, from processing targets, a model that is generated by using a plurality of hyperparameters that are the same combination as that of any of the models represented by the evaluated model list.

15. The neural network construction device according to claim 11, The output unit outputs the output model in the form of source code based on a language related to the calculation processing device.

16. The neural network construction device according to claim 11, The output unit outputs the output model in the form of a hardware description language.

17. The neural network construction device according to claim 13, The determination unit stops the generation of the neural network model by the generation unit when the result of the executed prediction accuracy evaluation satisfies a predetermined condition.

18. The neural network construction device according to claim 17, The acquisition unit acquires an accuracy target indicating a predetermined level of accuracy of the model of the neural network, The predetermined condition is a situation in which the result of the prediction accuracy evaluation does not reach the accuracy target in the models whose generation order is continuous for a predetermined number or more.

19. An information processing device comprising a processing unit and a storage unit, The storage unit stores a model generated by the neural network construction device according to any one of claims 1 to 6, The calculation processing unit reads the model from the storage unit and executes the model.

20. A neural network construction method, the neural network construction method being executed by a neural network construction device having a processing device and a storage device, the processing device of the processing device, obtaining a first condition including a resource condition related to a computing resource possessed by an embedded device and a performance constraint related to a processing performance possessed by the embedded device, wherein the first condition is a condition used to determine a candidate hyperparameter, wherein the candidate hyperparameter is a candidate for a hyperparameter of a neural network to be constructed, Determine the candidate hyperparameter using the first condition, Using the candidate hyperparameters, a model of a neural network is generated that uses image data acquired by an image sensor as input data, determining whether the generated model satisfies the performance constraint, and outputting data based on the result of the determination. In the determination, at least one of an upper limit and a lower limit of the candidate hyperparameter is calculated using the first condition, and one or more candidate hyperparameters are determined based on at least one of the calculated upper limit and lower limit. The first condition includes information on at least one of the size of the input data to the neural network and the size of the output data from the neural network. In the decision, the upper limit of the candidate hyperparameter is calculated based on at least one of the size of the input data and the size of the output data included in the first condition, and at least a portion of the calculated hyperparameters below the upper limit is determined as one or more candidate hyperparameters.

21. A non-volatile computer-readable recording medium recording a program to be executed by a processing unit in a neural network construction device having a processing unit and a storage unit, The neural network construction device is executed by the arithmetic processing device to: obtaining a first condition including a resource condition related to a computing resource possessed by an embedded device and a performance constraint related to a processing performance possessed by the embedded device, wherein the first condition is a condition used to determine a candidate hyperparameter, wherein the candidate hyperparameter is a candidate for a hyperparameter of a neural network to be constructed, Determine the candidate hyperparameter using the first condition, Using the candidate hyperparameters, a model of a neural network is generated that uses image data acquired by an image sensor as input data, determining whether the generated model satisfies the performance constraint, and outputting data based on the result of the determination. In the determination, at least one of an upper limit and a lower limit of the candidate hyperparameter is calculated using the first condition, and one or more candidate hyperparameters are determined based on at least one of the calculated upper limit and lower limit. The first condition includes information on at least one of the size of the input data to the neural network and the size of the output data from the neural network. In the decision, the upper limit of the candidate hyperparameter is calculated based on at least one of the size of the input data and the size of the output data included in the first condition, and at least a portion of the calculated hyperparameters below the upper limit is determined as one or more candidate hyperparameters.

Images