Information processing device, information processing method, and program

Abstract

An information processing device 10 has: a score calculation unit 11 that inputs training data included in a training data set to a trained neural network model, generates, for each item of the training data, statistical information representing an internal state of the trained neural network model, and calculates, on the basis of the generated statistical information, a score representing the complexity of the training data; and a parameter determination unit 12 that determines, on the basis of the calculated score, a parameter representing the structure of the neural network model.

Description

Information processing device, information processing method, and computer-readable recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a computer-readable recording medium for exploring the structure of a neural network.

[0002] Searching for neural network structures that have high recognition accuracy and can perform recognition processing at high speed has traditionally been performed manually by skilled AI (Artificial Intelligence) researchers, but in recent years, methods have been proposed for automatically searching for the above-mentioned neural network structures using NAS (Neural Architecture Search) and the like.

[0003] However, the search space for neural network structures (the set of structures to be evaluated) is huge, and in order to evaluate the recognition accuracy of a neural network, it is necessary to repeatedly train it during the search.

[0004] Furthermore, currently, each training session takes several hours to several days, and training must be repeated dozens of times during a search, which requires enormous computational costs and search time.

[0005] As a related technique, Non-Patent Document 1 discloses a technique for searching for a neural network structure with higher recognition accuracy than a manually created neural network structure using reinforcement learning.

[0006] Furthermore, as a related technique, Non-Patent Document 2 discloses a technique for performing a search without using reinforcement learning by continuously relaxing the search problem and making it differentiable.

[0007] Furthermore, as a related technique, Non-Patent Document 3 discloses a technique for deriving a neural network structure that can perform high-speed recognition processing with high recognition accuracy by combining components of a neural network structure with high hardware efficiency in a bottom-up manner.

[0008] Barret Zoph and Quoc V. Le, “Neural Architecture Search with Reinforcement Learning,” [online], [Submitted on 5 Nov 2016 (v1), last revised 15 Feb 2017 (this version, v2)], [Retrieved May 16, 2022], Internet <URL: https: / / arxiv.org / abs / 1611.01578> Hanxiao Liu and two others, “DARTS: Differentiable Architecture Search,” [online], [Submitted on 24 Jun 2018 (v1), last revised 23 Apr 2019 (this version, v2)], [Retrieved May 16, 2022], Internet <URL: https: / / arxiv.org / abs / 1806.09055> Xiaofan Zhang and 11 others, “SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems”, [online], [Submitted on 20 Sep 2019 (v1), last revised 29 Feb 2020 (this version, v2)], [Retrieved May 16, 2022], Internet <URL: https: / / arxiv.org / abs / 1909.09709>

[0009] However, in Non-Patent Document 1, a huge number of GPUs (Graphics Processing Units) are used for the search and run for a long period of time, which results in a huge computational cost for the search.

[0010] Furthermore, in Non-Patent Documents 2 and 3, since training is repeated several times or more, it is difficult to perform a search in a short time.

[0011] An example of the object of the present disclosure is to determine parameters that represent the structure of a neural network that has high recognition accuracy and can execute recognition processing at high speed.

[0012] In order to achieve the above object, an information processing device according to one aspect of the present disclosure is characterized by having: a score calculation means for inputting training data included in a training dataset into a trained neural network model, generating statistical information representing the internal state of the trained neural network model for each of the training data, and calculating a score representing the complexity of the training data based on the generated statistical information; and a parameter determination means for determining parameters representing the structure of the neural network model based on the calculated score.

[0013] In addition, to achieve the above object, an information processing method according to one aspect of the present disclosure is characterized in that a computer inputs training data included in a training dataset into a trained neural network model, generates statistical information representing the internal state of the trained neural network model for each piece of training data, calculates a score representing the complexity of the training data based on the generated statistical information, and determines parameters representing the structure of the neural network model based on the calculated score.

[0014] Furthermore, to achieve the above object, a computer-readable recording medium according to one aspect of the present disclosure is characterized in that it causes a computer to execute the following instructions: input training data included in a training dataset into a trained neural network model; generate statistical information representing the internal state of the trained neural network model for each piece of training data; calculate a score representing the complexity of the training data based on the generated statistical information; and determine parameters representing the structure of the neural network model based on the calculated score.

[0015] According to the present disclosure, it is possible to determine parameters that represent the structure of a neural network that has high recognition accuracy and can execute recognition processing at high speed.

[0016] FIG. 1 is a diagram illustrating an example of an information processing device. FIG. 2 is a diagram illustrating an example of a system having an information processing device. FIG. 3 is a diagram illustrating an internal state. FIG. 4 is a diagram illustrating an example of global statistical information. FIG. 5 is a diagram illustrating an example of score determination information. FIG. 6 is a diagram illustrating a method for determining the number of layers. FIG. 7 is a diagram illustrating a method for determining the number of channels. FIG. 8 is a diagram illustrating an example of the operation of the information processing device. FIG. 9 is a block diagram illustrating an example of a computer that realizes the information processing device in an embodiment.

[0017] Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same or corresponding functions are denoted by the same reference numerals, and repeated description thereof may be omitted.

[0018] (Embodiment) The configuration of an information processing device in an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining an example of an information processing device.

[0019] 1 is a device that efficiently searches for a neural network structure that has high recognition accuracy and can execute recognition processing at high speed. Also, as shown in FIG. 1, the information processing device 10 has a score calculation unit 11 and a parameter determination unit 12.

[0020] The score calculation unit 11 inputs the training data included in the training data set into the trained neural network model, generates statistical information representing the internal state (state of layers and channels) of the trained neural network model for each piece of training data, and calculates a score representing the complexity of the training data based on the generated statistical information.

[0021] Based on the calculated score, the parameter determination unit 12 determines parameters that represent the structure of a neural network model that has higher recognition accuracy and can perform recognition processing at high speed for a training data set than the trained neural network model.

[0022] By using a neural network model (second neural network model) obtained by training a neural network having a structure represented by the above-mentioned parameters using a training data set, it is possible to achieve higher recognition accuracy and faster recognition processing than when using the above-mentioned trained neural network model (first neural network model).The parameters include, for example, the number of layers and the number of channels.

[0023] As described above, in the embodiment, it is possible to determine the structure of a neural network that has high recognition accuracy and can perform recognition processing at high speed for a training dataset without training in deep learning.

[0024] In other words, by using a low-dimensional score that represents the complexity of the training data, it is possible to efficiently determine parameters (number of layers, number of channels) that represent the structure of a neural network that can achieve high recognition accuracy and perform recognition processing at high speed for the training data.

[0025] [System Configuration] The configuration of the information processing device 10 in this embodiment will be described in more detail with reference to Fig. 2. Fig. 2 is a diagram illustrating an example of a system including an information processing device.

[0026] 2, the system 100 includes an information processing device 10, a storage device 20, and an output device 30. In the system 100, the information processing device 10, the storage device 20, and the output device 30 are connected via a network.

[0027] The information processing device 10 is, for example, a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, a server computer, a personal computer, a mobile terminal, etc.

[0028] The information processing device 10 is a neural network structure determination device for determining parameters (number of layers, number of channels) for optimizing the structure of a neural network.

[0029] The storage device 20 is a server computer, a circuit having a memory, or the like. The storage device 20 stores, for example, information such as at least a trained neural network model, a training data set, and parameters representing the structure of the neural network. In the example of Fig. 2, the storage device 20 is provided outside the information processing device 10, but it may also be provided inside the information processing device 10.

[0030] The output device 30 acquires output information (described later) converted into an outputtable format by the output information generation unit 13, and outputs generated images, sounds, etc. based on the output information. The output device 30 is, for example, an image display device using a liquid crystal, an organic EL (Electro Luminescence), or a CRT (Cathode Ray Tube). Furthermore, the image display device may also include an audio output device such as a speaker. The output device 30 may also be a printing device such as a printer.

[0031] The network is a general network constructed using communication lines such as the Internet, a LAN (Local Area Network), a dedicated line, a telephone line, an in-house network, a mobile communication network, Bluetooth (registered trademark), or Wi-Fi (Wireless Fidelity).

[0032] The information processing device 10 in FIG. 2 includes a score calculation unit 11, a parameter determination unit 12, and an output information generation unit 13.

[0033] The score calculation unit 11 first inputs training data included in a training data set into a trained neural network model, and generates statistical information representing the internal state of the trained neural network model for each training data set.

[0034] The training data set may be, for example, a collection of image data, a collection of video data, a collection of vibration data, or a collection of audio data.

[0035] The trained neural network model is a neural network model that has already been trained. The trained neural network model may be implemented in the information processing device 10 or in an information processing device other than the information processing device 10.

[0036] As the trained neural network model, it is possible to use, for example, a neural network model such as M2Det or Residual Network (ResNet).

[0037] For more information on M2Det, please refer to the literature: Qijie Zhao and six others, "M2Det: A Single-Shot Object Detector Based on Multi-Level Feature Pyramid Network," pp. 9259-9266, Vol. 33 No. 01: AAAI-19, [Submitted on 12 Nov 2018 (v1), last revised 6 Jan 2019 (this version, v3)].

[0038] For more information on ResNet, please refer to the literature: Kaiming He and three others, "Deep Residual Learning for Image Recognition," [online], [Submitted on 10 December 2015], [Retrieved May 16, 2022], Internet <URL: https: / / arxiv.org / pdf / 1512.03385.pdf>.

[0039] The internal state can be represented using statistical information such as pixel values, average values ​​of pixel values, and variance values ​​of pixel values ​​of an activation map in deep learning, for example.

[0040] 3A and 3B are diagrams for explaining the internal state. An example of training data is shown in A of Fig. 3. An example of an activation map is shown in B of Fig. 3. An example of the internal state is shown in C of Fig. 3.

[0041] FIG. 3A shows image data used as training data, which is an image of a road, a vehicle traveling on the road, and a cloud floating in the sky.

[0042] The example in FIG. 3B shows an activation map of the trained neural network model when the image data described above is input to the trained neural network model and an inference process is performed.

[0043] The activation map shown in Figure 3B represents layers on the horizontal axis and channels on the vertical axis. The trained neural network model consists of M layers, where M is an integer greater than or equal to 2. Each layer consists of D or fewer channels, where D is an integer greater than or equal to 1.

[0044] For convenience, the activation maps shown in FIG. 3B only illustrate activation maps for channel 1 of layer 1, channel D of layer 1, channel 1 of layer M, and channel D of layer M.

[0045] In the example of FIG. 3C, the statistical information shows the maximum and average values ​​of all pixel values ​​of the activation map for each channel included in the trained neural network model.

[0046] However, the statistical information is not limited to the maximum and average values ​​of all pixel values ​​described above. For example, the maximum and average values ​​of all pixel values ​​of an activation map for each layer may be used. Alternatively, the maximum and average values ​​of all pixel values ​​of an activation map for each stage combining multiple layers may be used as statistical information. Furthermore, in addition to the maximum and average values, the mode and median may also be used.

[0047] Alternatively, the statistical information may be the maximum or average of all pixel values ​​of the activation map for all channels in some layers, or the maximum or average of all pixel values ​​of the activation map for some channels in some layers.

[0048] Next, the score calculation unit 11 generates global statistical information for the training data set based on the statistical information calculated for each training data set.

[0049] 4 is a diagram illustrating an example of global statistical information. Fig. 4A shows statistical information representing the internal state of training data included in a training data set. Fig. 4B shows global statistical information for the entire training data set, calculated by taking the maximum and average values ​​of these statistical information.

[0050] In the example of Fig. 4B, the global statistical information includes a global maximum value and a global mean value. The global maximum value is the largest value among the maximum values ​​for the training data included in the training data set. The global mean value is the average value among the mean values ​​for the training data included in the training data set.

[0051] However, the global statistical information is not limited to the global maximum value and global average value. For example, the global statistical information may be the global maximum value among the average values ​​for each of the training data included in the training data set.

[0052] The global statistical information is information that quantifies the degree of activation of the internal state when the training data included in the training dataset is input to the trained neural network model.

[0053] Next, the score calculation unit 11 first calculates a judgment value based on the global statistical information and a preset calculation (mathematical formula).

[0054] Next, the score calculation unit 11 uses the calculated judgment value and refers to the score determination information to determine a score that represents the complexity of the training data corresponding to the judgment value.

[0055] The score determination information is information that associates a judgment range for determining a score with the score. The judgment range is information that indicates a range to which a judgment value calculated based on statistical information representing the internal state and a preset calculation falls.

[0056] The judgment range is determined, for example, by using experiments, simulations, etc. Note that the judgment value differs depending on the type of statistical information used in the calculation and the type of calculation, so the judgment range needs to be changed depending on the calculation method of the judgment value.

[0057] 5 is a diagram illustrating an example of score determination information. In the example of Fig. 5, the score calculation unit 11 first calculates a judgment value (=global average value / global maximum value) by dividing the global average value by the global maximum value.

[0058] 5, if the judgment value is less than 0.01, the score is set to "1," and if the judgment value is equal to or greater than 0.01 and less than 0.05, the complexity score is set to "2." In this way, the score is determined according to the judgment value.

[0059] In the example of C in FIG. 4, the judgment value is 21.4 / 250=0.0856, so the score is "3."

[0060] The score may be expressed not only in one dimension but also in two or more dimensions. For example, the score may be calculated by calculating the maximum and average values ​​for each stage that combines the above-mentioned multiple layers, calculating a score for each stage, and expressing it as a four-dimensional value such as (3, 5, 4, 4).

[0061] Complexity is an index that represents the difficulty of recognizing data in a training dataset. For example, a training dataset containing 80 types of objects will be more complex than a training dataset containing only one type of object. In another example, a training dataset containing only images of pedestrians will be more complex than a training dataset containing only images of apples. Therefore, complexity is defined using, for example, a global mean value, a global maximum value, or the like.

[0062] The score is an index of complexity calculated using statistical information obtained when each training data is input. The score can be expressed as a low-dimensional discrete value.

[0063] The score may be expressed as a one-dimensional discrete value, for example, from 1 to 8 (level 1 to level 8). Alternatively, the score may be expressed as a one-dimensional continuous value, for example, from 1.0 to 100.0. Furthermore, the score may be expressed as a two-dimensional value, for example, (1, 3) or (5, 8).

[0064] In addition, instead of statistical information, the complexity of the training data set may be determined by determining the redundancy of the trained neural network model based on the change in recognition accuracy when part of the trained neural network model is deleted.

[0065] The parameter determination unit 12 determines parameters representing the structure of the neural network model based on the calculated score. The parameters include, for example, the number of layers and the number of channels.

[0066] Specifically, the parameter determination unit 12 first obtains the score from the score calculation unit 11. Next, the parameter determination unit 12 determines parameters that represent the structure of the neural network model based on the calculated score and a preset calculation (mathematical formula).

[0067] (1) Explanation of Determining the Number of Layers Fig. 6 is a diagram for explaining a method for determining the number of layers. In the example of Fig. 6, a case will be described in which the number of layers of a neural network model such as ResNet is determined based on the score.

[0068] In the example of FIG. 6, when the output size of the stage is reduced to 56×56, 28×28, 14×14, and 7×7, the number of layers included in each stage is determined.

[0069] The number of layers is specified by specifying the number of repetitions of the ResNet block, which is composed of a 1 × 1 convolution layer, a 3 × 3 convolution layer, and a 1 × 1 convolution layer, which are basic blocks of ResNet. That is, if the number of repetitions of the ResNet block is 2, the number of layers is 2 × 3 = 6, and if the number of repetitions is 3, the number of layers is 3 × 3 = 9.

[0070] In the example of FIG. 6, a table is shown in which the stage name "Stage name", the output size "Output size", and the number of layers "Number of layers" are associated with each other.

[0071] The parameter determination unit 12 sets the number of layers in the stage conv1 (output size 112×112) to “1”.

[0072] The parameter determination unit 12 calculates the number of layers of the stage conv2_x (output size 56×56) using Equation 1. Note that the function ceil( ) in the equation described below is a function that represents rounding up.

[0073] (Equation 1) Number of layers of conv2_x=3×(ceil(score / 4)+1)

[0074] For example, if the score is 3, the number of layers in the stage conv2_x (output size 56×56) is 3×(ceil(3 / 4)+1)=6 layers.

[0075] The parameter determination unit 12 calculates the number of layers of the stage conv3_x (output size 28×28) using Equation 2.

[0076] (Equation 2) Number of layers of conv3_x=3×(ceil(score / 2)+1)

[0077] For example, if the score is 3, the number of layers in the stage conv3_x (output size 28×28) is 3×(ceil(3 / 2)+1)=9 layers.

[0078] The parameter determination unit 12 calculates the number of layers of the stage conv4_x (output size 14×14) using Equation 3.

[0079] (Equation 3) Number of layers of conv4_x=3×(ceil(score×2))

[0080] For example, if the score is 3, the number of layers in the stage conv4_x (output size 14×14) is 3×(3×2)=18 layers.

[0081] The parameter determination unit 12 calculates the number of layers of the stage conv5_x (output size 7×7) using Equation 4.

[0082] (Equation 4) Number of layers of conv5_x=3×(ceil(score / 4)+1)

[0083] For example, if the score is 3, the number of layers in the stage conv5_x (output size 7×7) is 3×(ceil(3 / 4)+1)=6 layers.

[0084] The coefficients in (Equation 1) to (Equation 4) may be set based on, for example, the ratio of the number of layers for each stage in a neural network model such as ResNet. In the example of Figure 6, the ratio of the number of layers of conv2:conv3:conv4:conv5 is 1:2:4:1, which is set based on the ratio of the number of layers of ResNet152 being 3:8:36:3, the ratio of the number of layers of ResNet101 being 3:4:23:3, and the ratio of the number of layers of ResNet50 being 3:4:6:3. However, the present invention is not limited to the example of Figure 6, and any ratio may be used.

[0085] (2) Explanation of Determining the Number of Channels Fig. 7 is a diagram for explaining a method for determining the number of channels. In the example of Fig. 7, a case will be described in which the number of channels of a neural network model such as ResNet is determined based on the score.

[0086] In the example of FIG. 7, the number of channels for each layer included in a stage is determined using the score and a predetermined linear correlation coefficient.

[0087] The parameter determination unit 12 first calculates a score coefficient by multiplying the score by a predetermined correlation coefficient, and then calculates the number of channels for each layer included in the stage by multiplying the number of basic channels by the score coefficient.

[0088] The number of basic channels can be set based on the configuration of a neural network model such as ResNet. For example, since the number of channels per stage of ResNet is 64, 64, 128, 256, or 512, the number of basic channels can be set so as to maintain the ratio of the number of channels as much as possible.

[0089] The correlation coefficient is set to multiply the number of basic channels determined based on a known model by a constant depending on the complexity. In the example of Fig. 6, the correlation coefficient is set to 0.25.

[0090] The correlation coefficient can be determined, for example, by experiments or simulations. That is, by creating a neural network model with gradually changing numbers of channels, training it on a certain training data set, and conducting an experiment to evaluate the recognition accuracy, it is possible to obtain the number of channels necessary and sufficient to maximize the recognition accuracy. In this case, the correlation coefficient can be obtained from the ratio of the complexity of the training data set to the number of channels that maximizes the recognition accuracy.

[0091] For example, if the score is 3, the number of channels in each layer of the conv5_x stage is 1024×(3×0.25)=768 channels.

[0092] It should be noted that in addition to linear correlation, polynomial correlation, exponential correlation, logarithmic correlation, etc. may also be used.

[0093] The output information generation unit 13 generates output information, such as statistical information, global statistical information, scores, and parameters, to be output to the output device 30. Thereafter, the output information generation unit 13 outputs the output information to the output device 30.

[0094] [Device Operation] Next, the operation of the information processing device in the embodiment will be described with reference to FIG. 8. FIG. 8 is a diagram for explaining an example of the operation of the information processing device. In the following description, reference will be made to the diagram as appropriate. In addition, in the embodiment, a method for determining parameters representing the structure of a neural network is implemented by operating the information processing device. Therefore, the description of the method for determining parameters representing the structure of a neural network in the embodiment will be substituted for the description of the operation of the information processing device below.

[0095] As shown in FIG. 8, first, the score calculation unit 11 acquires training data from the training data set stored in the storage device 20, and inputs the acquired training data to the trained neural network model (step A1).

[0096] Next, the score calculation unit 11 generates statistical information representing the internal state (e.g., the state of layers and channels) of the trained neural network model for each training data (step A2). The internal state represents, for example, the state of layers, channels, etc.

[0097] Next, if the score calculation unit 11 has calculated statistical information for all of the predetermined training data (step A3: Yes), it executes the process of step A5. If the score calculation unit 11 has not calculated statistical information for all of the predetermined training data (step A3: No), it executes the process of step A1.

[0098] Next, the score calculation unit 11 calculates a score representing the complexity of the training data based on the generated statistical information (step A4). Specifically, in step A4, the score calculation unit 11 first calculates a judgment value based on the statistical information representing the internal state and a preset calculation. Next, in step A4, the score calculation unit 11 uses the calculated judgment value and refers to score determination information to determine the score.

[0099] Next, the parameter determination unit 12 determines parameters representing the structure of a neural network model that can achieve high recognition accuracy and high-speed recognition processing for the training data set based on the calculated score (step A5). Specifically, in step A5, the parameter determination unit 12 determines parameters such as (1) the number of layers and (2) the number of channels described above.

[0100] Next, the output information generation unit 13 generates output information to be output to the output device 30, such as statistical information, global statistical information, scores, parameters, etc., and then the output information generation unit 13 outputs the output information to the output device 30 (step A6).

[0101] [Effects of the embodiment] In the embodiment, it is possible to determine the structure of a neural network that has high recognition accuracy and can execute recognition processing at high speed for a training data set without training in deep learning.

[0102] In other words, by using a low-dimensional score that represents the complexity of the training data, it is possible to efficiently determine parameters (number of layers, number of channels) that represent the structure of a neural network that can achieve high recognition accuracy and perform recognition processing at high speed for the training data.

[0103] [Program] The program in the embodiment may be a program that causes a computer to execute steps A1 to A6 shown in Fig. 8. By installing and executing this program on a computer, the information processing device and information processing method in the embodiment can be realized. In this case, the processor of the computer functions as the score calculation unit 11, the parameter determination unit 12, and the output information generation unit 13 and performs processing.

[0104] The program in the embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the score calculation unit 11, the parameter determination unit 12, and the output information generation unit 13.

[0105] [Physical Configuration] A computer that implements an information processing apparatus by executing a program in the embodiment will now be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of a computer that implements an information processing apparatus in the embodiment.

[0106] 9, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU or an FPGA in addition to or instead of the CPU 111.

[0107] The CPU 111 loads the program (code) of this embodiment stored in the storage device 113 into the main memory 112 and executes it in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). The program of this embodiment is provided in a state stored in a computer-readable recording medium 120. The program of this embodiment may be distributed over the Internet connected via the communication interface 117. The recording medium 120 is a non-volatile recording medium.

[0108] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0109] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0110] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0111] The information processing device in the embodiment can be realized by using hardware corresponding to each part, instead of a computer on which a program is installed. Furthermore, the information processing device may be realized in part by a program and in the remaining part by hardware.

[0112] [Supplementary Note] The following supplementary note is further disclosed regarding the above-described embodiment. Part or all of the above-described embodiment can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but is not limited to the following description.

[0113] (Supplementary Note 1) An information processing device having: a score calculation unit that inputs training data included in a training dataset into a trained neural network model, generates statistical information representing an internal state of the trained neural network model for each of the training data, and calculates a score representing the complexity of the training data based on the generated statistical information; and a parameter determination unit that determines parameters representing the structure of the neural network model based on the calculated score.

[0114] (Supplementary Note 2) The information processing device described in Supplementary Note 1, wherein the score calculation unit generates global statistical information for the training data set based on the statistical information calculated for each of the training data, calculates a judgment value based on the global statistical information and a predetermined calculation, and determines the score corresponding to the calculated judgment value.

[0115] (Supplementary Note 3) The information processing device according to Supplementary Note 1 or 2, wherein the parameter determination unit determines the number of layers of the neural network model as the parameter based on the score.

[0116] (Supplementary Note 4) The information processing device according to Supplementary Note 1 or 2, wherein the parameter determination unit determines, as the parameter, a number of channels of the neural network model based on the score.

[0117] (Supplementary Note 5) An information processing method, in which a computer inputs training data included in a training dataset into a trained neural network model, generates statistical information representing the internal state of the trained neural network model for each piece of training data, calculates a score representing the complexity of the training data based on the generated statistical information, and determines parameters representing the structure of the neural network model based on the calculated score.

[0118] (Supplementary Note 6) An information processing method according to Supplementary Note 5, further comprising: generating global statistical information for the training data set based on the statistical information calculated for each of the training data; calculating a judgment value based on the global statistical information and a predetermined calculation; and determining the score corresponding to the calculated judgment value.

[0119] (Supplementary Note 7) The information processing method according to Supplementary Note 5 or 6, further comprising determining the number of layers of the neural network model as the parameter based on the score.

[0120] (Supplementary Note 8) The information processing method according to Supplementary Note 5 or 6, further comprising determining the number of channels of the neural network model as the parameter based on the score.

[0121] (Supplementary Note 9) A computer-readable recording medium having recorded thereon a program including instructions to cause a computer to input training data included in a training dataset into a trained neural network model, generate statistical information representing the internal state of the trained neural network model for each of the training data, calculate a score representing the complexity of the training data based on the generated statistical information, and determine parameters representing the structure of the neural network model based on the calculated score.

[0122] (Supplementary Note 10) A computer-readable recording medium according to Supplementary Note 9, further comprising: generating global statistical information for the training data set based on the statistical information calculated for each of the training data; calculating a judgment value based on the global statistical information and a predetermined calculation; and determining the score corresponding to the calculated judgment value.

[0123] (Supplementary Note 11) The computer-readable recording medium according to Supplementary Note 9 or 10, wherein the number of layers of the neural network model is determined as the parameter based on the score.

[0124] (Supplementary Note 12) The computer-readable recording medium according to Supplementary Note 9 or 10, further comprising determining the number of channels of the neural network model as the parameter based on the determined score.

[0125] Although the embodiments have been described above, the disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the disclosure within the scope of the invention.

[0126] According to the above disclosure, it is possible to efficiently determine parameters representing the structure of a neural network that can achieve high recognition accuracy and high-speed recognition processing. Furthermore, this is useful in fields where optimization of the structure of a neural network model is required.

[0127] REFERENCE SIGNS LIST 10 Information processing device 11 Score calculation unit 12 Parameter determination unit 13 Output information generation unit 20 Storage device 30 Output device 100 System 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. a score calculation means for inputting training data included in a training data set into a trained neural network model, generating statistical information representing an internal state of the trained neural network model for each of the training data, and calculating a score representing the complexity of the training data based on the generated statistical information; A parameter determination means for determining parameters representing a structure of a neural network model based on the calculated score; An information processing device having the above configuration.

2. The score calculation means generating global statistical information for the training data set based on the statistical information calculated for each of the training data; calculating a judgment value based on the global statistical information and a preset calculation, and determining the score corresponding to the calculated judgment value; The information processing device according to claim 1 .

3. The parameter determination means determines the number of layers of the neural network model as the parameter based on the score.

3. The information processing device according to claim 1 or 2.

4. The parameter determination means determines the number of channels of the neural network model as the parameter based on the score.

3. The information processing device according to claim 1 or 2.

5. The computer inputting training data included in a training data set into a trained neural network model, generating statistical information representing an internal state of the trained neural network model for each of the training data, and calculating a score representing the complexity of the training data based on the generated statistical information; determining parameters representing a structure of the neural network model based on the calculated scores; Information processing methods.

6. The computer further comprises: generating global statistical information for the training data set based on the statistical information calculated for each of the training data; calculating a judgment value based on the global statistical information and a preset calculation, and determining the score corresponding to the calculated judgment value; The information processing method according to claim 5.

7. The computer further comprises: determining the number of layers of the neural network model as the parameter based on the score; 7. The information processing method according to claim 5 or 6.

8. The computer further comprises: determining the number of channels of the neural network model as the parameter based on the score; 7. The information processing method according to claim 5 or 6.

9. On the computer, inputting training data included in a training data set into a trained neural network model, generating statistical information representing an internal state of the trained neural network model for each of the training data, and calculating a score representing the complexity of the training data based on the generated statistical information; determining parameters representing a structure of a neural network model based on the calculated score; A program containing instructions.

10. The computer further comprises: generating global statistical information for the training data set based on the statistical information calculated for each of the training data; calculating a judgment value based on the global statistical information and a preset calculation, and determining the score corresponding to the calculated judgment value; The program according to claim 9.

Images