Information processing method, information processing device, and program

The method enables the design of neural networks for diverse tasks by allowing users to select tasks, input data, and structure search methods, optimizing network structure for specific applications.

JP7586079B2Active Publication Date: 2024-11-19SONY GROUP CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2021533930
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-07-22
Filing Date
2020-07-09
Publication Date
2024-11-19
Estimated Expiration
2040-07-09

AI Technical Summary

Technical Problem

Current neural network design methods are mainly focused on image recognition and do not adequately address other tasks such as generative models and speech/language processing, limiting their applicability and efficiency.

Method used

An information processing method and device that allows users to select a task, acquire input data, and display a neural network structure as a default model, with options for structure search methods tailored to the task and data, enabling design of neural networks for a wide range of tasks.

Benefits of technology

Facilitates the design of neural networks compatible with various tasks, including generative models and speech/language processing, by providing a user-friendly interface for selecting tasks, input data, and structure search methods, optimizing network structure for specific applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586079000001
    Figure 0007586079000001
  • Figure 0007586079000002
    Figure 0007586079000002
  • Figure 0007586079000003
    Figure 0007586079000003
Patent Text Reader

Abstract

The present disclosure pertains to an information processing method, an information processing device, and a program that make it possible to easily design a neural network that corresponds to a predetermined task. The information processing device accepts a task selection from a user, acquires input data to be used for learning the task, and displays a neural network having a structure in accordance with the task selected and the input data acquired, as a default model. The present disclosure can be applied to, for example, a GUI that makes it possible for the user to intuitively design a neural network.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an information processing method, an information processing device, and a program, and more particularly to an information processing method, an information processing device, and a program that enable easy design of a neural network corresponding to a desired task. [Background technology]

[0002] Neural networks used for deep learning are known in the past, and various methods have been proposed for searching for an optimal solution from among multiple candidates.

[0003] For example, Patent Document 1 discloses an information processing device that updates an optimal solution of an evaluated neural network based on the evaluation result of another neural network with a different network structure generated from the evaluated neural network. According to the information processing method described in Patent Document 1, it is possible to more efficiently search for a network structure according to the environment.

[0004] In recent years, services have also been provided that automatically design deep learning models for image recognition without the need to design the neural networks (deep learning models) used in deep learning, simply by providing input data and labels. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2017 / 154284 Summary of the Invention [Problem to be solved by the invention]

[0006] In addition to image recognition, there are many other tasks to which deep learning can be applied, including generative models, super-resolution, and speech / language processing.

[0007] However, currently available neural network design methods are mainly aimed at image recognition, and no consideration has been given to designing neural networks for other tasks.

[0008] The present disclosure has been made in consideration of the above circumstances, and makes it possible to easily design a neural network that corresponds to a desired task. [Means for solving the problem]

[0009] In the information processing method disclosed herein, an information processing device receives a task selection by a user, acquires input data to be used for learning the task, and displays a neural network having a structure corresponding to the selected task and the acquired input data as a default model. and displaying, as a selection of structure search methods for the neural network, the structure search methods according to the task and the input data with priority. It is an information processing method.

[0010] The information processing device of the present disclosure includes a reception unit that receives a task selection by a user, an acquisition unit that acquires input data used for learning the task, and a display control unit that displays the selected task and a neural network having a structure corresponding to the acquired input data as a default model. The display control unit preferentially displays the structure search method according to the task and the input data as a selection option of the structure search method of the neural network. It is an information processing device.

[0011] The program disclosed herein causes a computer to receive a task selection by a user, acquire input data to be used for learning the task, and display a neural network having a structure corresponding to the selected task and the acquired input data as a default model. and displaying, as a selection of structure search methods for the neural network, the structure search methods according to the task and the input data with priority. It is a program for executing processing.

[0012] In the present disclosure, a task selection by a user is accepted, input data to be used for learning the task is acquired, and a neural network having a structure corresponding to the selected task and the acquired input data is displayed as a default model. As a selection of the structure search method of the neural network, the structure search method according to the task and the input data is preferentially displayed. . [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing an example of the configuration of an information processing device; [Diagram 3] 4 is a block diagram showing an example of a functional configuration of a control unit; [Figure 4] FIG. 13 is a diagram illustrating an example of a GUI. [Diagram 5] 11 is a flowchart illustrating an automatic model structure search process. [Figure 6] 11 is a flowchart illustrating an automatic model structure search process. [Figure 7] 11 is a flowchart illustrating an automatic model structure search process. [Figure 8] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 9] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 10] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 11] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 12] FIG. 13 is a diagram showing examples of parameters that can be set for structure searching. [Figure 13] FIG. 13 is a diagram showing examples of parameters that can be set for structure searching. [Figure 14] FIG. 13 is a diagram showing examples of parameters that can be set for structure searching. [Figure 15] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 16] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 17] FIG. 13 is a diagram showing examples of parameters that can be set for structure searching. [Figure 18] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 19] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 20] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 21]13 is a flowchart illustrating a model compression process. [Figure 22] 13 is a flowchart illustrating a model compression process. [Figure 23] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 24] FIG. 13 is a diagram illustrating an example of a GUI. [Diagram 25] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 26] FIG. 13 is a diagram illustrating an example of a GUI. [Figure 27] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Modes for carrying out the present disclosure (hereinafter, referred to as embodiments) will be described below. The description will be given in the following order.

[0015] 1. System and Equipment Configuration 2. Automatic structure exploration of models 3. Compress the model 4. Computer configuration examples

[0016] <1. System and device configuration> (Example of information processing system configuration) FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment of the present disclosure.

[0017] 1 includes an information processing terminal 10 and an information processing server 30. The information processing terminal 10 and the information processing server 30 are connected via a network 20 so as to be able to communicate with each other.

[0018] The information processing terminal 10 is an information processing device for presenting a GUI (Graphical User Interface) related to neural network design to a user. The information processing terminal 10 is configured as a PC (Personal Computer), a smartphone, a tablet terminal, or the like.

[0019] The information processing server 30 is an information processing device that executes processing related to the design of a neural network in response to a request from the information processing terminal 10 and supplies the information processing terminal 10 with data necessary for the design of the neural network.

[0020] The network 20 has a function of connecting the information processing terminal 10 and the information processing server 30. The network 20 is composed of public line networks such as the Internet, a telephone line network, and a satellite communication network, various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks), etc. The network 20 may also be composed of a dedicated line network such as an IP-VPN (Internet Protocol - Virtual Private Network).

[0021] (Example of configuration of information processing device) FIG. 2 is a diagram showing an example of the configuration of an information processing device constituting the information processing terminal 10 described above.

[0022] The information processing device 100 in FIG. 2 includes a control unit 110, an input unit 120, a display unit 130, a communication unit 140, and a storage unit 150.

[0023] The control unit 110 is configured with a processor such as a GPU (Graphics Processing Unit) or a CPU (Central Processing Unit), and controls each unit of the information processing device 100.

[0024] The input unit 120 supplies an input signal corresponding to an operational input by a user to the control unit 110. The input unit 120 is configured as, for example, a keyboard, a mouse, or a touch panel.

[0025] The display unit 130 displays a GUI and various information related to the design of a neural network under the control of the control unit 110.

[0026] The communication unit 140 communicates with the information processing server 30 via the network 20 under the control of the control unit 110, and supplies various data provided from the information processing server 30 to the control unit 110.

[0027] The storage unit 150 stores various data used in the processes executed by the control unit 110, as well as programs executed by the control unit 110, etc.

[0028] (Example of functional configuration of the control unit) FIG. 3 is a block diagram showing an example of a functional configuration of the control unit 110 in FIG.

[0029] 3 includes a reception unit 211, an acquisition unit 212, a determination unit 213, an execution unit 214, and a display control unit 215. Each unit of the control unit 110 is realized by a processor constituting the control unit 110 executing a predetermined program stored in the storage unit 150.

[0030] The receiving unit 211 receives an operation input by a user based on an input signal from the input unit 120. Reception information indicating the content of the received operation input by the user is supplied to each unit of the control unit 110. For example, the receiving unit 211 receives an input related to the design of a neural network by the user.

[0031] The acquisition unit 212 acquires data supplied from the information processing server 30 via the communication unit 140, or acquires data stored in the storage unit 150, in response to the reception information from the reception unit 211. The data acquired by the acquisition unit 212 is appropriately supplied to the determination unit 213 and the execution unit 214.

[0032] The determining unit 213 determines, in response to the information received from the receiving unit 211, a model that is to be a candidate for the neural network to be presented to the user.

[0033] The execution unit 214 executes a structure search and compression of the model determined by the determination unit 213 based on the received information from the reception unit 211 and the data from the acquisition unit 212, learning using the model, and the like.

[0034] The display control unit 215 controls the display of GUIs and various information related to the design of a neural network on the display unit 130. For example, the display control unit 215 controls the display of the model determined by the determination unit 213, information related to structural search and compression of the model, the results of learning using the model, and the like.

[0035] In recent years, GUIs have become known that enable users to intuitively design neural networks used in deep learning.

[0036] On the other hand, there are many tasks to which deep learning can be applied, including image recognition, generative models, super-resolution, and speech / language processing.

[0037] However, the GUIs currently available are mainly designed for image recognition, and no consideration has been given to designing neural networks that can handle other tasks.

[0038] Therefore, in the following, an example will be described that provides a GUI that enables the design of neural networks that can handle a wide range of tasks.

[0039] <2. Automatic structure search for models> First, we will explain automatic model structure search. Automatic structure search is a method for automatically searching for the structure of a neural network used in deep learning, and is a technology that uses a specified algorithm to find the optimal network structure from many combinations.

[0040] The automatic structure search of the model is started when the user selects, for example, a menu for executing the automatic structure search of the model on the GUI provided by the information processing device 100.

[0041] Fig. 4 shows an example of a GUI that is displayed on the display unit 130 when a menu for executing an automatic structure search of a model is selected. In the following, a screen such as that shown in Fig. 4 is referred to as an automatic structure search execution screen.

[0042] The automatic structure search execution screen is provided with various GUI parts, such as a drop-down list 311, a text box 312, a check box 313, a check box 314, a text box 315, a check box 316, and a drop-down list 317. In addition, below the drop-down list 317, a model display box 318 is provided.

[0043] A drop-down list 311 is a GUI part for selecting a task. The task here indicates a problem that is the target of deep learning, such as image recognition, generative model, super-resolution, and speech / language processing.

[0044] A text box 312 is a GUI part for inputting the number of computation layers of the neural network to be subjected to the structural search.

[0045] A check box 313 is a GUI part for selecting whether or not to use a skip connection.

[0046] Check box 314 is a GUI part for selecting whether or not to perform a cell-based structural search. When check box 314 is operated to select performing a cell-based structural search, the number of computation layers input in text box 312 represents the number of cells. A cell includes multiple computation layers.

[0047] A text box 315 is a GUI part for inputting the number of nodes (operation layers) in the cell.

[0048] A check box 316 is a GUI part for selecting whether or not to use a skip connection in a cell.

[0049] Note that the text box 315 and the check box 316 are active only when the check box 314 is selected to perform a cell-based structure search.

[0050] A drop-down list 317 is a GUI part for selecting a structure searching method.

[0051] The model display box 318 is an area in which a model of the neural network that is the subject of the structure search is displayed.

[0052] In the following, various GUI parts displayed on the automatic structure search execution screen will be described in detail with reference to the flowcharts of FIGS.

[0053] In step S11, the receiving unit 211 receives a task selection made by a user's operation on the drop-down list 311.

[0054] Specifically, as shown in Fig. 8, four tasks, "Image Recognition", "Generative Model", "Super-Resolution", and "Speech / Language Processing", are displayed in drop-down list 311, and the user can select one of the four tasks. In the example of Fig. 8, "Image Recognition" is selected.

[0055] In step S12, it is determined whether or not a default model is to be used. The default model is a network structure model that is prepared in advance in correspondence with a task that can be selected in the drop-down list 311.

[0056] If it is determined in step S12 that the default model is to be used, the process proceeds to step S13.

[0057] In step S13, the determination unit 213 determines, as a default model, a neural network having a structure corresponding to the task selected in the drop-down list 311 and the input data acquired at a predetermined timing by the acquisition unit 212. Then, the display control unit 215 displays the determined default model in the model display box 318.

[0058] The input data may be prepared by the user or may be supplied from the information processing server 30.

[0059] At this time, in addition to the selected task and the acquired input data, a neural network having a structure corresponding to the hardware information of the information processing device 100 may be determined and displayed as a default model. The hardware information here includes information regarding the processing capacity of the processor constituting the control unit 110 of the information processing device 100 and information regarding the number of processors.

[0060] In the example of FIG. 8, since “image recognition” is selected in the drop-down list 311, a feature extractor (encoder) that extracts image features is displayed in the model display box 318 as the default model corresponding to “image recognition.”

[0061] Also, as shown in FIG. 9, when “super-resolution” is selected in the drop-down list 311, the model display box 318 displays the encoder and decoder that constitute the autoencoder as the default model corresponding to “super-resolution”.

[0062] Note that only some of the computation layers of the default model displayed in the model display box 318 can be targeted for the structural search described below. For example, when a predetermined range is designated by a user's dragging operation in the model display box 318, a bounding box 321 is displayed in the model display box 318 as shown in Fig. 10. In this case, only the computation layers of the default model surrounded by the bounding box 321 are targeted for the structural search.

[0063] Furthermore, although not shown, when "generative model" is selected in the drop-down list 311, a decoder is displayed as a default model corresponding to the "generative model" in the model display box 318. When "speech / language processing" is selected in the drop-down list 311, a model with a recurrent neural network (RNN) structure is displayed in the model display box 318 as the default model corresponding to the "speech / language processing".

[0064] Here, the default model displayed in the model display box 318 is not limited to one, and the reception unit 211 receives a change from the displayed default model to another default model in response to a user operation. As a result, the model display box 318 switches and displays candidate models to be the target of structure search.

[0065] In step S14, the receiving unit 211 receives the selection of a default model by the user, thereby determining the default model to be subjected to the structure search.

[0066] On the other hand, if it is determined in step S12 that the default model is not to be used, the process proceeds to step S15, where the reception unit 211 receives a model design by the user. The model designed by the user is displayed in the model display box 318, similar to the default model.

[0067] After the default model is established in step S14 or the model is designed in step S15, the process proceeds to step S16.

[0068] In step S16, the display control unit 215 displays an outline of the network structure of the model together with the model displayed in the model display box 318. Specifically, the display control unit 215 displays the size of the search space of the model displayed in the model display box 318 and the approximate amount of calculation as the outline of the network structure.

[0069] Thereafter, in step S17, in response to a user operation, it is determined whether or not to add a computation layer to the model displayed in the model display box 318. That is, the reception unit 211 determines whether or not to accept the addition of a computation layer to the default model.

[0070] If it is determined in step S17 that a computation layer is to be added, the process proceeds to step S18 in FIG. 6, where it is determined whether or not to use a preset computation layer.

[0071] If it is determined in step S18 that the preset computation layer is to be used, then in step S19, the reception unit 211 receives a selection of the preset computation layer by the user, and the process returns to step S17.

[0072] On the other hand, if it is determined in step S18 that the preset arithmetic layer is not to be used, then in step S20, the accepting unit 211 accepts the design of the arithmetic layer by the user, and the process returns to step S17.

[0073] Now, if it is determined in step S17 that a computation layer is not to be added, the process proceeds to step S21 in FIG.

[0074] In step S21, the display control unit 215 displays options of structure search methods in the drop-down list 317 according to the model displayed in the model display box 318. Specifically, the display control unit 215 preferentially displays in the drop-down list 317 structure search methods according to the task selected in the drop-down list 311 and the input data acquired by the acquisition unit 212 at a predetermined timing.

[0075] For example, as shown in FIG. 11, drop-down list 317 displays representative structural search methods such as “reinforcement learning,” “genetic algorithm,” and “gradient method,” and the user can select one of these structural search methods.

[0076] For structure search using reinforcement learning, methods such as NASNet proposed in “B. Zoph, V. Vasudevan, J. Shlens, and QV Le. Learning transferable architectures for scalable image recognition. In CVPR, 2018.” and ENAS proposed in “H. Pham, MY Guan, B. Zoph, QV Le, and J. Dean. Efficient neural architecture search via parameter sharing. In ICML, 2018.” are used. For structure search using genetic algorithms, methods such as AmoebaNet proposed in “E. Real, A. Aggarwal, Y. Huang, and QV Le. Regularized evolution for image classifier architecture search. In AAAI, 2019.” are used. In addition, for structure search using gradient methods, methods such as DARTS proposed in “H. Liu, K. Simonyan, and Y. Yang. DARTS: Differentiable architecture search. In ICLR, 2019.” and SNAS proposed in “S. Xie, H. Zheng, C. Liu, and L. Lin. SNAS: Stochastic neural architecture search. In ICLR, 2019.” are used.

[0077] At this time, in addition to the selected task and the acquired input data, the drop-down list 317 may preferentially display a structure search method according to the hardware information of the information processing device 100.

[0078] In step S22, the receiving unit 211 receives a selection of a structure search method through a user's operation on the drop-down list 317. In the example of Fig. 11, "reinforcement learning" is selected.

[0079] After that, in step S23, the reception unit 211 receives an input of the settings of the structure search method selected in the drop-down list 317. At this time, as shown in Fig. 11 for example, a setting input section 331 for inputting the settings of the structure search method is displayed to the right of the model display box 318. In the setting input section 331, parameters that can be set for the structure search method selected in the drop-down list 317 are input by the user.

[0080] Here, examples of parameters that can be set for the structure searching method will be described with reference to FIGS.

[0081] FIG. 12 shows examples of parameters that can be set for structure search using reinforcement learning.

[0082] Configurable parameters for structural search using reinforcement learning include the number of RNN / LSTM layers, the number of child networks, the controller learning rate, the architecture parameter optimizer, the number of searches, and the number of times the child networks are trained.

[0083] The number of RNN / LSTM layers is the number of computation layers of the RNN used in reinforcement learning and its variant, LSTM (Long-short Term Memory), and is set by inputting an int value.

[0084] The number of child networks is the number of child networks (candidate networks) that are output at one time by the controller that is the parent network that predicts the main network structure, and is set by inputting an int type value.

[0085] The learning rate of the controller is a parameter related to the learning by the controller described above, and is set by inputting a float type numerical value.

[0086] The architecture parameter optimizer is a method for adjusting the learning rate, and is set by selecting from a pull-down list. Options include "Adam", "SGD", and "Momentum".

[0087] The number of searches is the number of times a search is performed, and is set by inputting an int type value.

[0088] The number of times the child network is trained is the number of epochs (the number of times one training data item is repeatedly trained) of the child network in one search, and is set by inputting an int value.

[0089] FIG. 13 shows an example of parameters that can be set for structure search using evolutionary computation including a genetic algorithm.

[0090] The parameters that can be set for structure search using evolutionary computation, which involves training multiple candidate networks, include the number of models to be saved, the number of training rounds, the number of populations, the number of samples, and the mutation pattern.

[0091] The number of models to be saved is the number of generated candidate networks (models) to be saved, and is set by inputting an int value. The number of models to be saved is approximately the same as the number of searches.

[0092] The number of learning iterations is the number of epochs for the generated model, and is set by inputting an int value.

[0093] The population number is the size of the population and is set by inputting an int type number.

[0094] The number of samples is the number of models to sample from the current population when selecting a model to mutate, and is set by inputting an int value.

[0095] The mutation pattern is a mutation pattern, and is set by selecting from a pull-down (drop-down list). The options available are "operation and input node", "operation only", "input node only", etc.

[0096] FIG. 14 shows an example of parameters that can be set for structure search using the gradient method.

[0097] The parameters that can be set for the gradient-based structural search include the number of searches, the architecture parameter learning rate, and the architecture parameter optimizer.

[0098] The number of searches, like the number of learning times, is the number of epochs of the generated model, and is set by inputting an int value.

[0099] Architecture parameters The learning rate is a parameter related to learning by the generated model, and is set by a float type numeric input.

[0100] The architecture parameter optimizer is a method for adjusting the learning rate, and is set by selecting from a pull-down list. Options include "Adam", "SGD", and "Momentum".

[0101] The above parameters can be set in the setting input section 331 according to the selected structure search method.

[0102] Returning to the flowchart of Figure 7, when the settings for the structural search method are input, in step S24, the display control unit 215 displays the predicted time required for structural search using the set parameters in accordance with the selected structural search method, for example at a predetermined position in the model display box 318.

[0103] Thereafter, in step S25, it is determined whether or not to change the settings of the structure searching method.

[0104] If it is determined in step S25 that the settings of the structure searching method are to be changed, the process returns to step S23, and the processes of steps S23 and S24 are repeated.

[0105] On the other hand, if it is determined in step S25 that the settings of the structure searching method are not to be changed, the process proceeds to step S26.

[0106] In step S26, the execution unit 214 starts a structure search with the set parameters.

[0107] When the execution of the structure search is completed, in step S27, the display control unit 215 displays a model of the searched structure in the model display box 318.

[0108] Thereafter, in step S28, it is determined whether or not to perform further structure searching.

[0109] If it is determined in step S28 that a further structural search is to be performed, the process returns to step S26, and the processes of steps S26 and S27 are repeated.

[0110] On the other hand, if it is determined in step S28 that no further structure search is to be performed, the process ends.

[0111] According to the above process, in addition to image recognition, it is possible to select tasks such as generative models, super-resolution, and speech / language processing, and a neural network with a structure according to the selected task and input data is displayed as a default model. Furthermore, it is possible to select from various structural search methods proposed in recent years, and structural search is performed using the selected structural search method.

[0112] This makes it possible to easily design a neural network that corresponds to a desired task, and ultimately to optimize the structure of a neural network that corresponds to a wide range of tasks.

[0113] (An example of cell-based structure exploration) An example of a GUI in the case where a cell-based structure search is not performed has been explained above, but an example of a GUI in the case where a cell-based structure search is performed will be explained below.

[0114] FIG. 15 shows an example of a GUI for performing a cell-based structure search.

[0115] On the automatic structure search execution screen of FIG. 15, a check box 314 is operated to select to perform a cell-based structure search.

[0116] Moreover, in the automatic structure search execution screen of FIG. 15, a model display box 341 and a cell display box 342 are provided in place of the model display box 318 in the automatic structure search execution screen described above.

[0117] The model display box 341 is an area in which the entire model of the neural network that is the subject of the structure search is displayed. The model displayed in the model display box 341 is a cell accumulation type model that is configured to include multiple cells (cell blocks).

[0118] In addition to the model made up of a plurality of cells, the model display box 341 also displays the size of the search space of the model displayed in the model display box 341 and the approximate amount of calculation as an outline of the network structure.

[0119] The cell display box 342 is an area for displaying cells that constitute the model displayed in the model display box 341 and are the subject of structural search. The cells displayed in the cell display box 342 are made up of multiple calculation layers.

[0120] 15, an estimate of the worst calculation amount and the like may be displayed, and the user may be allowed to specify a range of the allowable calculation amount. This enables structure search taking into account the constraints on the calculation amount.

[0121] Fig. 16 shows an example of a setting screen used to set the model structure displayed in the model display box 341 and the cell structure displayed in the cell display box 342. The setting screen 350 in Fig. 16 is displayed as a pop-up on the automatic structure search execution screen, for example, by clicking on a predetermined area in the model display box 341 or the cell display box 342.

[0122] The setting screen 350 is provided with text boxes 351 , 352 , 353 , and 354 , and a drop-down list 355 .

[0123] The text box 351 is a GUI part for inputting the number of cells that make up the model displayed in the model display box 341.

[0124] The text box 352 is a GUI part for inputting the number of types of cells that make up the model displayed in the model display box 341.

[0125] The text box 353 is a GUI part for inputting the number of nodes (operation layers) in the cell displayed in the cell display box 342 .

[0126] The text box 354 is a GUI part for inputting the input number for one node in the cell displayed in the cell display box 342 .

[0127] The drop-down list 355 is a GUI part for selecting a reduction operation method at the output node. For example, three reduction operation methods, "element-wise add", "concatenate", and "average", are displayed in the drop-down list 355, and the user can select one of the three reduction operation methods.

[0128] The contents set in this way are reflected in real time in the model displayed in the model display box 341 and in the cells displayed in the cell display box 342.

[0129] It should be noted that it is possible to construct not only a cell accumulation type model but also a multi-layer stacked feedforward type neural network depending on the settings on the setting screen 350. Although not shown, it is also possible to construct a model with, for example, one cell, eight nodes in the cell, and one input to one node in the cell.

[0130] In the above description, parameters for the structure search are set according to the selected structure search method, but parameters independent of the structure search method can also be set.

[0131] FIG. 17 shows examples of parameters that can be set for general structure search, regardless of the structure search method selected.

[0132] Configurable parameters for general structure search include the model learning rate, model parameter optimizer, and number of feature maps.

[0133] The model learning rate is a parameter related to learning by the model that is the target of structure search, and is set by inputting a float type value.

[0134] The model parameter optimizer is a method for adjusting the model learning rate, and is set by selecting from a pull-down list. The available options are "Adam", "SGD", "Momentum", etc.

[0135] The number of feature maps is the number of filters in the hidden layer in the first cell of the constructed model, and is set by a numeric input of type int.

[0136] Such parameters can be set regardless of the structure search method selected.

[0137] (Definition of the search space) The user can select the computation layer to be used in the structural search from among preset computation layers.

[0138] FIG. 18 shows an example of a screen that is displayed when the user selects a computation layer to be used in a structural search from among preset computation layers.

[0139] A selection section 361 is provided at the top of an area 360 on the screen in Fig. 18. Types of computation layers are displayed as options in the selection section 361. In the example of Fig. 18, "Affine", "Convolution", "DepthwiseConvolution", and "Deconvolution" are displayed as options, and "Convolution" is selected.

[0140] A selection unit 362 is provided below the selection unit 361. In the selection unit 362, preset arithmetic layers of the type selected in the selection unit 361 are displayed as options. In the example of Fig. 18, "Convolution_3x3", "Convolution_5x5", "Convolution_7x7", "MaxPooling_3x3", and "AveragePooling_3x3" are displayed as options.

[0141] A model configured with computation layers selected from preset computation layers is displayed in area 370 of the screen in Fig. 18. In the example of Fig. 18, a model configured with an input layer and a convolution layer is displayed.

[0142] Additionally, users can also define their own computational layers to be used in the structural search.

[0143] FIG. 19 shows an example of a screen that is displayed when a user individually defines a computational layer to be used in a structural search.

[0144] A setting section 363 is provided at the bottom of the area 360 on the screen of Fig. 19. The setting section 363 is displayed, for example, by pressing an add computation button (not shown). Various parameters of the computation layer selected by the user are displayed in the setting section 363.

[0145] The user can independently define the computation layer used in the structural search by setting desired values ​​for the parameters of the computation layer in the setting unit 363 .

[0146] In the structure search of the cell accumulation model, it is necessary to prevent the input size and output size from changing due to the calculation in the cell. Therefore, the parameters that the user can set in the setting unit 363 may be limited to a part, and other parameters may be automatically set according to the setting of these parameters. For example, in the parameters of the convolution layer, the filter size is set, and the other parameters are automatically set.

[0147] (Structure search results) As described above, once a structural search has been performed, the network of structures that has been searched is displayed.

[0148] FIG. 20 shows an example of a screen displaying the results of executing a structure search for the above-mentioned cell accumulation type model.

[0149] In the example of FIG. 20, the model and cells of the searched structure are displayed in a model display box 341 and a cell display box 342 .

[0150] Furthermore, in addition to the model and cells of the searched structure, the accuracy, the amount of calculation, etc. may be displayed. In the example of Fig. 20, an accuracy / calculation amount display section 381 is provided above the cell display box 342. The accuracy / calculation amount display section 381 displays the accuracy, the number of parameters (size), FLOPS (Floating-point Operations per Second), power consumption, and intermediate buffer (size).

[0151] The user can check the accuracy and calculation amount displayed in the accuracy / calculation amount display section 381 to determine whether or not to perform the structure search again.

[0152] In particular, conventional GUIs for designing neural networks do not take into account the computational complexity of the hardware that performs the structure search.

[0153] In contrast, according to the above-mentioned configuration, a structure search that takes into account the constraints on the amount of calculation can be realized with a simple operation.

[0154] <3. Model compression> Next, we will explain model compression. Model compression is a method to simplify the structure in a neural network and reduce calculation costs. One example is distillation, which realizes the performance of a large-scale, complex network in a small network.

[0155] Model compression is started by the user selecting, for example, a menu for executing model compression in the GUI provided by the information processing device 100. Model compression may also be started by selecting a button for executing model compression on a screen displaying the execution results of the structure search as shown in Fig. 20.

[0156] 21 and 22 are flowcharts illustrating the model compression processing.

[0157] In step S51, the acquisition unit 212 reads a base model that is a model to be compressed. The base model may be a model designed in advance, or may be a model after the above-mentioned structure search has been performed.

[0158] In step S52, it is determined whether or not a computation layer is to be added to the loaded base model.

[0159] If it is determined that a computation layer is to be added to the base model, the process proceeds to step S53, where the reception unit 211 receives a command to add a computation layer to the base model.

[0160] Steps S52 and S53 are repeated until it is determined that a computation layer is not to be added to the base model, and when it is determined that a computation layer is not to be added to the base model, the process proceeds to step S54.

[0161] In step S54, the display control unit 215 displays the current compression settings.

[0162] Thereafter, in step S55, it is determined whether or not the compression settings are to be changed in response to a user operation.

[0163] If it is determined in step S55 that the compression settings are to be changed, the process proceeds to step S56, where the reception unit 211 receives the selection of the computation layer. At this time, the reception unit 211 also receives the selection of the compression method for the base model.

[0164] Next, in step S57, the reception unit 211 receives input of compression settings for the selected computation layer. At this time, the compression conditions for the selected computation layer are input as the compression settings. After step S57, the process returns to step S55.

[0165] In this manner, the compression settings for the selected computation layer are determined.

[0166] On the other hand, if it is determined in step S55 that the compression settings are not to be changed, the process proceeds to step S58 in FIG.

[0167] In step S58, the execution unit 214 executes compression of the model based on the compression settings set for each of the computation layers.

[0168] In step S59, the execution unit 214 calculates the compression ratio of each of the computation layers. At this time, the display control unit 215 displays the compression ratio of each of the computation layers as a compression result.

[0169] In step S60, the execution unit 214 determines whether or not the calculated compression ratio of each of the computation layers satisfies the compression conditions set for each of the computation layers.

[0170] If it is determined that the compression ratio does not satisfy the condition, the process returns to step S58, and the execution of compression of the model and the calculation of the compression ratio are repeated.

[0171] On the other hand, if it is determined that the compression ratio satisfies the condition, the process proceeds to step S61.

[0172] In step S61, it is determined whether or not further compression is to be performed on the base model in response to a user operation.

[0173] If it is determined that further compression is to be performed, the process returns to step S55 in FIG. 21, and the subsequent processes are repeated.

[0174] On the other hand, if it is determined in step S61 that no further compression is to be performed, the process proceeds to step S62, where the execution unit 214 stores the compressed model, and the process ends.

[0175] (GUI example) An example of the GUI displayed on the display unit 130 during the model compression process will now be described.

[0176] FIG. 23 shows an example of a screen for configuring settings related to model compression.

[0177] A drop-down list 411 and buttons 412 are provided below an area 410 on the screen of Fig. 23. The drop-down list 411 is a GUI part for selecting a compression method.

[0178] The drop-down list 411 displays three compression methods, "pruning," "quantization," and "distillation," and the user can select one of the three compression methods.

[0179] The button 412 is a GUI part for executing compression using the compression method selected in the drop-down list 411 .

[0180] A base model 421 to be compressed is displayed in area 420 on the screen of Fig. 23. The amount of calculation for each calculation layer constituting the base model 421 is shown to the right of the base model 421. The amount of calculation for each calculation layer is shown as a ratio of the memory usage of each calculation layer when the total memory usage is set to 100%.

[0181] By checking the amount of calculation for each calculation layer constituting the base model 421, the user can grasp which calculation layer in the base model 421 may become a bottleneck.

[0182] Furthermore, for compression using the compression method selected from the drop-down list 411, the user may be allowed to set an accuracy degradation tolerance, which is an index of the degree of accuracy degradation to be tolerated, and a target compression ratio.

[0183] In the example of FIG. 23, the entire computation layer constituting the base model 421 can be the target of compression, or only a part of the computation layer can be the target of compression.

[0184] FIG. 24 shows an example of setting compression for each calculation layer constituting the base model 421.

[0185] 24, the "Affine_3" layer is selected from among the computation layers constituting the base model 421, and a child screen 431 is displayed. The child screen 431 is a screen for setting the allowable ranges (compression conditions) for each index of latency, memory, intermediate buffer, and power consumption for the selected computation layer.

[0186] The child screen 431 has a radio button for enabling the setting of the allowable range for each index, and text boxes for inputting the minimum and maximum values ​​of the allowable range. By enabling the setting of the allowable range and inputting the minimum and maximum values ​​of the allowable range, the compression conditions for the selected computation layer are set.

[0187] 25 and 26 show examples of screens on which the compression results are displayed.

[0188] At the bottom of area 410 on the screen in Figures 25 and 26, there are provided an index selection section 441 for selecting which index the compression results are to be displayed for, and an accuracy change rate display section 442 that displays the rate of change in accuracy due to compression.

[0189] 25 and 26, a base model 421 that was the subject of compression is shown in area 420, and to the right of that, the compression results for each computation layer that constitutes the base model 421 are shown. As the compression result for each computation layer, the compression ratio for the index selected in index selection unit 441 is shown.

[0190] Specifically, in the example of FIG. 25, memory is selected in index selection unit 441, and the compression rate for memory is shown as the compression result for each computation layer constituting base model 421.

[0191] In the example of FIG. 26, power consumption is selected in the index selection unit 441, and the compression rate for power consumption is shown as the compression result for each calculation layer constituting the base model 421.

[0192] This allows the user to decide which computational layers to target for further compression.

[0193] According to the above process, compression can be performed not only on models for which a structure search has been performed, but also on existing models, making it possible to reduce calculation costs.

[0194] In the above, the processes related to the automatic structure search and compression of the model and the display of the GUI are performed on the information processing terminal 10 configured as the information processing device 100. Without being limited to this, the information processing server 30 may be configured as the information processing device 100, the processes related to the automatic structure search and compression of the model may be performed on the information processing server 30, and only the display of the GUI may be performed on the information processing terminal 10. Furthermore, each process executed by the above-mentioned information processing device 100 may be performed on either the information processing terminal 10 or the information processing server 30 of the information processing system in FIG. 1.

[0195] <4. Computer configuration> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0196] FIG. 27 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0197] The above-mentioned information processing device 100 is realized by a computer 1000 having the configuration shown in FIG.

[0198] The CPU 1001 , ROM 1002 , and RAM 1003 are connected to each other via a bus 1004 .

[0199] An input / output interface 1005 is further connected to the bus 1004. An input unit 1006 including a keyboard, a mouse, etc., and an output unit 1007 including a display, a speaker, etc. are connected to the input / output interface 1005. In addition, a storage unit 1008 including a hard disk or non-volatile memory, a communication unit 1009 including a network interface, etc., and a drive 1010 that drives a removable medium 1011 are connected to the input / output interface 1005.

[0200] In the computer 1000 configured as above, the CPU 1001 loads, for example, a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes the program, thereby performing the series of processes described above.

[0201] The programs executed by the CPU 1001 are installed in the storage unit 1008, for example, by being recorded on a removable medium 1011, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0202] The program executed by computer 1000 may be a program in which processing is performed chronologically in the order described in this specification, or may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0203] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0204] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0205] Furthermore, the present disclosure can have the following configurations. (1) An information processing device, Accept the user's selection of a task; obtaining input data to be used in learning the task; A neural network having a structure according to the selected task and the acquired input data is displayed as a default model. Information processing methods. (2) In addition to the task and the input data, the neural network having a structure corresponding to hardware information of the information processing device is displayed as the default model. The information processing method described in (1). (3) The hardware information is information regarding the processing capacity of a processor. The information processing method described in (2). (4) The hardware information is information regarding the number of processors. The information processing method described in (2). (5) Displaying at least one of the size of a search space and the computational complexity of the default model together with the default model. An information processing method according to any one of (1) to (4). (6) Accepting changes to the default model by the user An information processing method according to any one of (1) to (5). (7) Accepts the addition of computational layers to the default model. (6) An information processing method according to the above. (8) As options for the neural network structure search method, the structure search method according to the task and the input data is preferentially displayed. An information processing method according to any one of (1) to (7). (9) In addition to the task and the input data, the structure search method according to hardware information of the information processing device is preferentially displayed. (8) An information processing method according to the present invention. (10) Accepting an input of the setting of the structure searching method selected by the user from the options The information processing method according to (8) or (9). (11) A predicted time required for the structure search is displayed according to the structure search method selected by the user from the options. An information processing method according to any one of (8) to (10). (12) execute a structure search based on the structure search method selected by the user from the options; Display the neural network of the structure explored An information processing method according to any one of (8) to (11). (13) The computation layer selected by the user in the neural network is the target of structural search. (12) An information processing method according to (12). (14) The cells included in the neural network are the subject of structural search. (12) An information processing method according to (12). (15) Further accepting a selection of a compression method for the neural network. An information processing method according to any one of (1) to (14). (16) A compression condition setting is accepted for each index selected by the user for a computation layer of the neural network. (15) An information processing method according to (15). (17) performing compression of the neural network with the selected compression technique; Display the compression results of the above computation layer (16) An information processing method according to (16). (18) Displaying the compression ratio of the computation layer for the index selected by the user (17) An information processing method according to (17). (19) a reception unit that receives a task selection by a user; an acquisition unit that acquires input data used for learning the task; a display control unit that displays a neural network having a structure corresponding to the selected task and the acquired input data as a default model; An information processing device comprising: (20) On the computer, Accept the user's selection of a task; obtaining input data to be used in learning the task; A neural network having a structure according to the selected task and the acquired input data is displayed as a default model. A program for executing a process. [Explanation of symbols]

[0206] 10 information processing terminal, 30 information processing server, 100 information processing device, 110 control unit, 120 input unit, 130 display unit, 140 communication unit, 150 storage unit, 211 reception unit, 212 acquisition unit, 213 determination unit, 214 execution unit, 215 display control unit, 1000 computer

Claims

1. An information processing device, Accept the user's selection of a task; obtaining input data to be used in learning the task; Displaying a neural network having a structure according to the selected task and the acquired input data as a default model; As options for the neural network structure search method, the structure search method according to the task and the input data is preferentially displayed. Information processing methods.

2. In addition to the task and the input data, the neural network having a structure corresponding to hardware information of the information processing device is displayed as the default model. The information processing method according to claim 1 .

3. The hardware information is information regarding the processing capacity of a processor. The information processing method according to claim 2 .

4. The hardware information is information regarding the number of processors. The information processing method according to claim 2 .

5. Displaying at least one of the size of a search space and the computational complexity of the default model together with the default model. The information processing method according to claim 1 .

6. Accepting changes to the default model by the user The information processing method according to claim 1 .

7. Accepts the addition of computational layers to the default model. The information processing method according to claim 6.

8. In addition to the task and the input data, the structure search method according to hardware information of the information processing device is preferentially displayed.

8. The information processing method according to claim 1.

9. Accepting an input of the setting of the structure searching method selected by the user from the options 8. The information processing method according to claim 1.

10. A predicted time required for the structure search is displayed according to the structure search method selected by the user from the options.

8. The information processing method according to claim 1.

11. execute a structure search based on the structure search method selected by the user from the options; Display the neural network of the structure explored 8. The information processing method according to claim 1.

12. The computation layer selected by the user in the neural network is the target of structural search. The information processing method according to claim 11.

13. The cells included in the neural network are the subject of structural search. The information processing method according to claim 11.

14. Further accepting a selection of a compression method for the neural network. The information processing method according to claim 1 .

15. A compression condition setting is accepted for each index selected by the user for a computation layer of the neural network. The information processing method according to claim 14.

16. performing compression of the neural network with the selected compression technique; Display the compression results of the above computation layer The information processing method according to claim 15.

17. Displaying the compression ratio of the computation layer for the index selected by the user The information processing method according to claim 16.

18. a reception unit that receives a task selection by a user; an acquisition unit that acquires input data used for learning the task; a display control unit that displays a neural network having a structure corresponding to the selected task and the acquired input data as a default model; Equipped with The display control unit preferentially displays the structure search method according to the task and the input data as a selection option of the structure search method of the neural network. Information processing device.

19. On the computer, Accept the user's selection of a task; obtaining input data to be used in learning the task; Displaying a neural network having a structure according to the selected task and the acquired input data as a default model; As options for the neural network structure search method, the structure search method according to the task and the input data is preferentially displayed. A program for executing a process.

Citation Information

Patent Citations

  • Error tolerant neural network model compression

    US10229356B1

  • Information processing method and information processing device

    WO2017154284A1

  • Program, information processing method, and information processing device

    WO2019035364A1