Learning device, generation method, inference device, inference method, and program

Abstract

This technology relates to a training device, a generation method, an inference device, an inference method, and a program that make it possible to select, without human assistance, training data that is suitable for training, and make it possible to efficiently train an inference model using the selected training data. On the basis of a training data group comprising training data that has correct answers, as well as a processing target training data group comprising processing target data that is for training, does not have correct answers, and corresponds to data which will serve as the processing target at the time of an inference, a training device according to one aspect of this technology selects, from the training data group, training data that is suitable for training an inference model to be used at the time of an inference, and outputs the selected training data, together with an inference model obtained by training using the selected training data. This technology can be applied to a computer that performs CNN training.

Description

[Technical field]

[0001] In particular, the present technology relates to a learning device, a generation method, an inference device, an inference method, and a program that enable selection of learning data suitable for learning without manual intervention, and enable efficient learning of an inference model using the selected learning data. [Background technology]

[0002] It is becoming common to use inference models obtained through machine learning such as deep learning to accomplish various tasks.

[0003] There are various data sets of training data that can be used to train an inference model, such as a set of images of handwritten characters that can be used to train an inference model to recognize handwritten characters. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] "Object Recognition with Gradient-Based Learning", (paper on CNN (Convolutional Neural Network)), Yann LeCun et al., http: / / yann.lecun.com / exdb / publis / pdf / lecun-99.pdf Summary of the Invention [Problem to be solved by the invention]

[0005] The training data set contains training data that is not suitable for training. Therefore, it is usually necessary to manually select a group of training data suitable for training in advance. The selection of the training data group is performed for each task.

[0006] If the training data group is not selected, the training time will be longer.

[0007] The present technology has been developed in light of these circumstances, and makes it possible to select training data suitable for learning without manual intervention, and to efficiently train an inference model using the selected training data. [Means for solving the problem]

[0008] A learning device according to one aspect of the present technology includes an information processing unit that selects from the learning data group, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed during inference, the learning data suitable for learning an inference model to be used during inference, and outputs the selected learning data together with the inference model obtained by performing learning using the selected learning data.

[0009] An inference device according to another aspect of the present technology includes an inference unit that selects from the learning data group, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to the data to be processed during inference, selects the learning data suitable for learning an inference model to be used during inference from the learning data group, inputs the data to be processed into the inference model output from a learning device that outputs the selected learning data together with the inference model obtained by learning using the selected learning data, and outputs an inference result representing the result of a predetermined processing.

[0010] In one aspect of the present technology, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to the data to be processed during inference, the learning data suitable for learning an inference model to be used during inference is selected from the learning data group, and the selected learning data is output together with the inference model obtained by learning using the selected learning data.

[0011] In another aspect of the present technology, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to the data to be processed during inference, the learning data suitable for learning an inference model to be used during inference is selected from the learning data group, and the data to be processed is input to the inference model output from a learning device that outputs the selected learning data, together with the inference model obtained by learning using the selected learning data, and an inference result representing the result of a predetermined processing is output. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a learning device according to an embodiment of the present technology; [Diagram 2] FIG. 13 is a diagram illustrating an example of task execution using a task model. [Diagram 3] 13 is a flowchart illustrating a learning process of an optimal data selection task learning unit. [Figure 4] FIG. 13 is a block diagram showing a configuration example of an optimal data selection / Task learning unit. [Diagram 5] FIG. 13 is a block diagram showing another example of the configuration of the learning device. [Figure 6] 13 is a flowchart illustrating a learning process of an optimal data generation task learning unit. [Figure 7] FIG. 13 is a block diagram showing a configuration example of an optimal data generation / Task learning unit. [Figure 8] 13 is a flowchart illustrating a learning process of an optimal data generation task learning unit. [Figure 9] FIG. 13 is a block diagram showing a configuration example of an optimal data generation / Task learning unit. [Figure 10] FIG. 2 is a block diagram showing a configuration example of an inference device. [Figure 11] FIG. 2 is a block diagram showing an example of the configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present technology will be described in the following order. 1. First embodiment: Example in which a set of learning data with correct answers is prepared 2. Second embodiment: Example of generating and preparing a group of learning data with correct answers 3. Configuration of the inference side 4.Other

[0014] <<1. First embodiment: Example in which a group of learning data with correct answers is prepared>> Learning device configuration FIG. 1 is a block diagram showing an example of the configuration of a learning device 1 according to an embodiment of the present technology.

[0015] 1, the learning device 1 is provided with an optimum data selection / task learning unit 11. To the optimum data selection / task learning unit 11, a learning data group #1 and a target data group #2 are input from outside.

[0016] Training data group #1 is a data group consisting of multiple training data that have correct answers (labeled). Each training data is composed of input data of the same type as the target data and output data that represents the correct answer to the task.

[0017] The input data is, for example, any one of various types of data such as RGB data (RGB image), polarized light data, multispectral data, and ultraviolet, near-infrared, and far-infrared data, which is invisible light wavelength data.

[0018] The input data may be data actually detected by a sensor in real space, or data generated by rendering based on a three-dimensional model. For example, when the type of data is RGB data, the input data is an image captured by an image sensor, or a CG (Computer Graphics) image generated by a computer through rendering or the like.

[0019] The output data will be data according to the task. For example, if the task is area segmentation, the output data will be the result of area segmentation on the input data. Similarly, if the task is object normal recognition, the output data will be the result of object normal recognition on the input data, and if the task is depth recognition, the output data will be the result of depth recognition on the input data. If the task is object recognition, the output data will be the result of object recognition on the input data.

[0020] The target data group #2 is a data group consisting of multiple target data of the same type as the input data of the learning data, which does not have a correct answer (unlabeled). The target data is data that is assumed to be used as a processing target during inference as an input of an inference model. Data corresponding to the data used as a processing target during inference is input to the learning device 1 as target data for learning.

[0021] The optimal data selection / task learning unit 11 learns and outputs a task model #3, which is an inference model used to execute a task, based on the learning data group #1 and the target data group #2.

[0022] FIG. 2 is a diagram showing an example of task execution using Task model #3.

[0023] When the task is area segmentation, an inference model with RGB data as input and the result of area segmentation as output is generated as Task model #3, as shown in Fig. 2. In the example of Fig. 2, an image showing the area in which the sofa appears is output in response to an input of an image showing a sofa as an object.

[0024] When the task model # 3 is a convolutional neural network (CNN), information representing the configuration and weights of the neural network is output from the optimal data selection and task learning unit 11 .

[0025] In addition, learning of a type of network other than CNN may be performed in the optimal data selection / task learning unit 11, and machine learning other than deep learning may be performed in the optimal data selection / task learning unit 11.

[0026] Moreover, the optimal data selection / task learning unit 11 selects learning data suitable for learning the Task model #3 from the learning data group #1. The learning of the Task model #3 is performed based on the selected learning data.

[0027] The optimal data selection / Task learning unit 11 outputs a plurality of learning data selected from the learning data group #1 as a Selected learning data group #4 together with the Task model #3. Each of the learning data constituting the Selected learning data group #4 is data having the correct answer.

[0028] In this way, the optimal data selection / Task learning unit 11 functions as an information processing unit that selects learning data suitable for learning the Task model #3 based on the learning data group #1 and the Target data group #2 consisting of target data for learning, and outputs the Selected learning data group #4 together with the Task model #3 obtained by performing learning using the selected learning data.

[0029] Since the learning data suitable for learning the inference model according to the task is automatically selected, manual selection of the learning data is not required. The learning data group #1 is, for example, a data group prepared in advance as a learning data set. The learning data constituting the learning data group #1 is data that has not been manually selected.

[0030] In addition, since the learning data selected as being suitable for learning the inference model is used for learning, efficient learning is possible using a small amount of learning data.

[0031] Since the learning data is selected using the Target data group #2, which is assumed to be the target data to be processed during inference, it is possible to detect the characteristics of the Target data used during inference in advance by analyzing the Selected learning data group #4. For example, if the Task is depth recognition, it is possible to detect in advance the distance range that will be the result of depth recognition. The analysis of the Selected learning data group #4 is performed, for example, in a downstream device that receives the Selected learning data group #4 output from the learning device 1.

[0032] ·Optimal data selection ·Task learning part 11 operation The learning process of the optimum data selection / task learning unit 11 will be described with reference to the flowchart of FIG.

[0033] In step S1, the optimum data selection / task learning unit 11 randomly selects a predetermined number of learning data from the learning data group #1.

[0034] In step S2, the optimal data selection / task learning unit 11 learns the model T based on the learning data selected in step S1. Here, learning of an inference model is performed in which the Input data of the learning data is used as input and the Output data prepared as the correct answer is used as output.

[0035] In step S3, the optimal data selection and task learning unit 11 inputs the target data group #2 to the model T and infers provisional correct answer data. That is, the inference results output in response to inputting each target data to the model T are set as provisional correct answer data.

[0036] In step S4, the optimal data selection / task learning unit 11 learns a model T' using the target data group #2 used in step S3 and the provisional correct answer data as inputs of the model T. Here, an inference model is learned in which each of the target data constituting the target data group #2 is used as input, and the provisional correct answer data obtained when each of the target data is input to the model T is used as output.

[0037] In step S5, the optimum data selection / task learning unit 11 inputs the learning data selected in step S1 to the model T' and performs inference.

[0038] In step S6, the optimum data selection / task learning unit 11 inputs the learning data selected in step S1 to the model T and performs inference.

[0039] In step S7, the optimal data selection / task learning unit 11 calculates the difference between the inference result obtained in step S6 using model T and the inference result obtained in step S5 using model T'. If the inference result when the learning data x is input to model T is T(x) and the inference result when the learning data x is input to model T' is T'(x), the difference s between the two is expressed by the following formula (1).

number

[0040] In step S8, the optimal data selection / task learning unit 11 discards data with large differences, leaving only the learning data with small differences. For example, 50% of the learning data in ascending order of differences is left, and the remaining 50% is deleted. The remaining learning data is held as the learning data constituting Selected learning data group #4.

[0041] In step S9, the optimal data selection task learning unit 11 judges whether or not further learning data with a small difference is necessary. If it is judged in step S9 that further learning data with a small difference is necessary, the process returns to step S1 and the subsequent processes are performed. The processes from step S1 to S9 are repeated as a loop process.

[0042] In the process of step S1 which is repeatedly performed, new learning data which has not been used in the previous learning is randomly selected from the learning data group #1, and the new learning data is added to the remaining learning data. That is, other learning data is selected in place of the learning data which has not been selected as the learning data which constitutes the Selected learning data group #4, and is added to the learning data to be used in the current loop process. Based on the learning data to which the new learning data has been added, the processes from step S2 onwards are performed.

[0043] If it is determined in step S9 that learning data with a small difference is not necessary, then in step S10, the optimal data selection / task learning unit 11 outputs model T at that point in time as Task model #3. Also, the optimal data selection / task learning unit 11 outputs the learning data selected up to that point together with Task model #3 as Selected learning data group #4.

[0044] ·Optimal data selection ·Configuration of Task learning part 11 FIG. 4 is a block diagram showing an example of the configuration of the optimum data selection / task learning unit 11 that performs the processing in FIG.

[0045] 4, the optimal data selection / task learning unit 11 is made up of a learning data acquisition unit 21, a task model learning / inference unit 22, a task model re-learning / inference unit 23, a data comparison unit 24, a data selection unit 25, and a final model / optimal data output unit 26. A learning data group #1 input from outside is supplied to the learning data acquisition unit 21, and a target data group #2 is supplied to the task model learning / inference unit 22 and the task model re-learning / inference unit 23.

[0046] The learning data acquiring unit 21 randomly selects and acquires learning data from the learning data group #1. In the first loop process in the learning process described with reference to Fig. 3, all learning data is randomly selected, and in the second and subsequent loop processes, learning data to be added to the learning data selected by the data selecting unit 25 is randomly selected. The process of step S1 in Fig. 3 is the process performed by the learning data acquiring unit 21.

[0047] The learning data selected by the learning data acquisition unit 21 is supplied to a task model learning and inference unit 22 , a task model re-learning and inference unit 23 , and a data selection unit 25 .

[0048] The task model learning and inference unit 22 learns the model T based on the learning data supplied from the learning data acquisition unit 21. The task model learning and inference unit 22 functions as a first learning unit that learns the model T as a first model. In addition, the task model learning and inference unit 22 inputs the target data group #2 to the model T and infers provisional correct answer data.

[0049] Furthermore, the task model learning and inference unit 22 inputs the learning data selected by the learning data acquisition unit 21 into the model T and performs inference. The processes of steps S2, S3, and S6 in FIG.

[0050] The model T obtained by learning by the task model learning and inference unit 22 is supplied to a final model and optimal data output unit 26, and the provisional correct answer data obtained by inference using the model T is supplied to a task model re-learning and inference unit 23. The inference result (T(x)) obtained by inference using the model T is supplied to a data comparison unit 24.

[0051] The task model re-learning and inference unit 23 learns a model T' using the target data group #2 and the provisional correct answer data supplied from the task model learning and inference unit 22. The task model re-learning and inference unit 23 functions as a second learning unit that learns the model T' as a second model. In addition, the task model re-learning and inference unit 23 inputs the learning data into the model T' and performs inference. The processes of steps S4 and S5 in FIG. 3 are the processes performed by the task model re-learning and inference unit 23.

[0052] The inference result (T′(x)) obtained by inference using the model T′ is supplied to the data comparison unit 24.

[0053] The data comparison unit 24 calculates a difference s between the inference result obtained using the model T supplied from the task model learning / inference unit 22 and the inference result obtained using the model T' supplied from the task model re-learning / inference unit 23. The process of step S7 in FIG. 3 is the process performed by the data comparison unit 24.

[0054] As the difference s, the absolute value of the difference described with reference to the above formula (1) may be calculated, or a square error may be calculated. Information representing the difference s is supplied to the data selection unit 25.

[0055] The data selection unit 25 selects learning data based on the difference s supplied from the data comparison unit 24. For example, the learning data is selected by threshold processing such that learning data whose difference s is equal to or smaller than a threshold is left, or by leaving a predetermined ratio of learning data in ascending order of difference. The processes of steps S8 and S9 in FIG. 3 are the processes performed by the data selection unit 25.

[0056] When the learning termination condition is satisfied, the learning data selected and held by the data selection unit 25 is supplied to the final model and optimal data output unit 26. For example, conditions that may be set as the learning termination condition include that the difference s of all learning data used for processing in the Task model learning and inference unit 22, the Task model re-learning and inference unit 23, etc. is equal to or less than a threshold value, and that the loop processing in Fig. 3 is repeated a predetermined number of times.

[0057] When the learning termination condition is satisfied, the final model / optimum data output unit 26 outputs the model T supplied from the Task model learning / inference unit 22 as Task model #3, and outputs the learning data supplied from the data selection unit 25 as Selected learning data group #4.

[0058] As described above, by using the target data group #2 that does not have a correct answer for learning, it is possible to select and output learning data suitable for learning. Also, it is possible to generate and output an inference model obtained by learning using only the learning data suitable for learning.

[0059] <<2. Second embodiment: Example of generating and preparing a group of learning data with correct answers>> <2-1. Example of generating a training data set by setting parameters randomly> Learning device configuration FIG. 5 is a block diagram showing another example of the configuration of the learning device 1. As shown in FIG.

[0060] 5, learning data used for learning Task model #3 is not prepared in advance, but is generated by the learning device 1 itself. Using the learning data generated by the learning device 1, learning of Task model #3 is performed as described above.

[0061] As shown in Fig. 5, the learning device 1 is provided with an optimal data generation / task learning unit 31 instead of the optimal data selection / task learning unit 11 in Fig. 1. The optimal data generation / task learning unit 31 has a renderer 31A. Target data group #2 is input from outside to the optimal data generation / task learning unit 31. Descriptions that overlap with those described above will be omitted as appropriate.

[0062] The optimum data generation and task learning unit 31 uses a renderer 31A to generate the learning data described above that is made up of Input data of the same type as the Target data and Output data that represents the correct answer to the Task.

[0063] When the input data is, for example, an RGB image, the optimum data generation / task learning unit 31 performs rendering based on a three-dimensional model to generate a CG image (CG RGB image) including a predetermined object. The optimum data generation / task learning unit 31 is provided with three-dimensional model data of various objects.

[0064] For example, when generating a CG image including a sofa as described with reference to Fig. 2, the optimum data generation / task learning unit 31 sets various learning data generation parameters and generates the CG image by performing rendering based on a three-dimensional model of the sofa. The learning data generation parameters are parameters that define the contents of the rendering. The rendering is performed based on multiple types of learning data generation parameters in which predetermined values ​​are set.

[0065] Similarly, when the input data is a type of data other than RGB data, such as polarized light data, multispectral data, or invisible light wavelength data, rendering is performed based on a three-dimensional model and a CG image is generated as the input data.

[0066] The optimal data generation / task learning unit 31 generates output data representing a correct answer by performing a simulation based on the learning data generation parameters used for rendering the input data, and generates learning data consisting of the input data and the output data. The optimal data generation / task learning unit 31 changes the settings of the learning data generation parameters or changes the three-dimensional model used for rendering to generate a learning data group consisting of multiple learning data.

[0067] Except for the generation of learning data, the processing performed in the learning device 1 in Fig. 5 is similar to the processing performed in the learning device 1 in Fig. 1. The optimal data generation / Task learning unit 31 in Fig. 5 outputs, together with the Task model #3, a generated learning data group #11 made up of learning data selected from the generated learning data as being suitable for learning the Task model #3.

[0068] In this way, it is also possible to generate the learning data in the learning device 1, rather than having the learning data prepared in advance.

[0069] ·Optimal data generation ·Task learning unit 31 operation The learning process of the optimum data generation·Task learning unit 31 will be described with reference to the flowchart of FIG.

[0070] In step S21, the optimum data generation / task learning unit 31 randomly sets learning data generation parameters and generates learning data. Here, a plurality of pieces of learning data are generated by changing the settings of the learning data generation parameters.

[0071] The process from step S22 onwards is basically the same as the process from step S2 onwards in FIG.

[0072] That is, in step S22, the optimum data generation / task learning unit 31 learns the model T based on the learning data generated in step S21.

[0073] In step S23, the optimum data generation / task learning unit 31 inputs the target data group #2 to the model T and performs inference of provisional correct answer data.

[0074] In step S24, the optimum data generation / task learning unit 31 learns a model T' using the target data group #2 used in step S23 as an input of the model T and the provisional correct answer data.

[0075] In step S25, the optimal data generation / task learning unit 31 inputs the learning data generated in step S21 to the model T' and performs inference.

[0076] In step S26, the optimal data generation / task learning unit 31 inputs the learning data generated in step S21 to the model T and performs inference.

[0077] In step S27, the optimum data generation·Task learning unit 31 calculates the difference between the inference result obtained in step S26 using the model T and the inference result obtained in step S25 using the model T′.

[0078] In step S28, the optimum data generation / task learning unit 31 leaves only the learning data with a small difference and discards data with a large difference.

[0079] In step S29, the optimum data generation task learning unit 31 judges whether or not further learning data with a small difference is required. If it is judged in step S29 that further learning data with a small difference is required, the process returns to step S21 and the subsequent processes are performed. The processes from step S21 to S29 are repeated as a loop process.

[0080] In the process of step S21 which is repeatedly performed, the learning data generation parameters are set randomly, new learning data is generated, and the new learning data is added to the remaining learning data. That is, other learning data is generated in place of the learning data which was not selected as the learning data constituting the generated learning data group #11, and is added to the learning data used in the current loop process. Based on the learning data to which the newly generated learning data has been added, the processes from step S22 onward are performed.

[0081] If it is determined in step S29 that learning data with a small difference is not necessary, then in step S30, the optimal data generation / task learning unit 31 outputs model T at that point in time as Task model #3. The optimal data generation / task learning unit 31 also outputs the learning data that has been generated and selected up to that point, together with Task model #3, as generated learning data group #11.

[0082] ·Optimal data generation ·Configuration of Task learning part 31 FIG. 7 is a block diagram showing an example of the configuration of the optimum data generation and task learning unit 31 that performs the processing of FIG.

[0083] Among the components shown in Fig. 7, the same components as those explained with reference to Fig. 4 are denoted by the same reference numerals. Duplicate explanations will be omitted as appropriate. The configuration of the optimum data generation / task learning unit 31 shown in Fig. 7 is the same as the configuration of the optimum data selection / task learning unit 11 in Fig. 4, except that a learning data generation unit 41 is provided instead of the learning data acquisition unit 21.

[0084] The learning data generating unit 41 randomly sets learning data generation parameters and performs rendering based on a three-dimensional model to generate input data that constitutes the learning data. The learning data generating unit 41 is realized by the renderer 31A.

[0085] For example, the training data generation parameters include the following parameters:

[0086] Object parameters Object Orientation Object location Material of the object - Object shape High level information (information specifying the type of object (chair, desk, sofa, etc.)) Low level information (information that directly specifies mesh vertices)

[0087] Light source parameters -Type of light source (point, spot, area, environment map, etc.) Light source direction Light source position Characteristics of the light source (wavelength (ultraviolet to visible light to near infrared and far infrared), polarization (Stokes vector))

[0088] Camera parameters -External parameters (camera orientation, position, etc.) - Internal parameters (FoV, focal length, etc.) Image sensor characteristics (noise model, etc.)

[0089] The learning data generation unit 41 also performs a simulation and generates Output data that is the correct answer for each piece of Input data according to the Task. The learning data generation unit 41 generates multiple pieces of learning data by changing the settings of learning data generation parameters or by changing the three-dimensional model used for rendering.

[0090] In the first loop process in the learning process described with reference to Fig. 6, all learning data is generated, and in the second and subsequent loop processes, learning data is generated to be added to the learning data selected by the data selection unit 25. The process of step S21 in Fig. 6 is the process performed by the learning data generation unit 41.

[0091] The learning data generated by the learning data generation unit 41 is supplied to the task model learning and inference unit 22 , the task model re-learning and inference unit 23 , and the data selection unit 25 .

[0092] As described above, even when generating training data, it is possible to select and output training data suitable for training by using the target data group #2 that does not have a correct answer for training. Also, it is possible to generate and output an inference model obtained by training using only training data suitable for training.

[0093] <2-2. Example of generating a learning data set by specifying parameter conditions> Although the learning data generation parameters that define the contents of rendering are set randomly in the above embodiment, they may be set according to conditions.

[0094] In the repeated loop processing, new learning data is generated to replace the learning data determined to be unsuitable for training model T. What kind of learning data should be generated as the new learning data can be specified based on the tendency of the learning data determined to be suitable for training model T. The condition for what kind of learning data (input data) should be generated is specified based on the result of the previous loop processing.

[0095] ·Optimal data generation ·Task learning unit 31 operation The learning process of the optimum data generation·Task learning unit 31 will be described with reference to the flowchart of FIG.

[0096] The process shown in FIG. 8 is similar to the process described with reference to FIG. 6, except that the condition for what kind of learning data should be generated is specified based on the result of the immediately preceding loop process.

[0097] That is, in step S41, the optimum data generation / task learning unit 31 randomly sets learning data generation parameters and generates learning data. The processing of steps S42 to S48 is performed using the learning data generated based on the randomly set learning data generation parameters.

[0098] In step S49, the optimum data generation / task learning unit 31 determines whether or not further learning data with a small difference is required.

[0099] If it is determined in step S49 that further learning data with a small difference is necessary, in step S50, the optimum data generation / task learning unit 31 specifies the conditions for the next learning data to be generated. Then, the process returns to step S41 and the subsequent processes are performed.

[0100] In the process of step S41 which is repeatedly performed, the learning data generation parameters are set according to the conditions, new learning data is generated, and the newly generated learning data is added to the remaining learning data, and the processes from step S42 onwards are performed.

[0101] If it is determined in step S49 that learning data with a small difference is not necessary, then in step S51, the optimal data generation / task learning unit 31 outputs model T at that point in time as Task model #3. Also, the optimal data generation / task learning unit 31 outputs the learning data that has been generated and selected up to that point as generated learning data group #11.

[0102] ·Optimal data generation ·Configuration of Task learning part 31 FIG. 9 is a block diagram showing an example of the configuration of the optimum data generation and task learning unit 31 that performs the processing of FIG.

[0103] The configuration of the optimum data generation / Task learning unit 31 shown in FIG. 9 is the same as that of the optimum data generation / Task learning unit 31 shown in FIG.

[0104] The data generation condition designation unit 42 designates conditions for newly generated learning data based on information supplied from the data selection unit 25. For example, information on the difference s between the retained learning data and the learning data that was discarded without being retained is supplied from the data selection unit 25.

[0105] Specifically, for parameters specifying the camera position and the light (light source) position, the condition is specified that new learning data is generated using parameters in the direction with the smallest error. The parameters in the direction with the smallest error are searched for using a search algorithm such as the hill climbing method.

[0106] For example, assume that external camera parameters include azimuth, zenith, and distance from the subject, and that learning data with azimuths of 40, 45, and 50 degrees already exists (has already been generated). In this case, if the difference s calculated using each learning data is, in order from smallest to largest, learning data with an azimuth of 40 degrees, learning data with an azimuth of 45 degrees, and learning data with an azimuth of 50 degrees, it is specified that learning data with an azimuth of 35 degrees is to be generated next.

[0107] When the light parameters include azimuth, zenith, and distance from the subject, the conditions are specified in a similar manner.

[0108] A condition may be specified that new learning data similar to learning data having a small difference s is generated. Whether the learning data is similar or not is determined using an index such as PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity), MSE (Mean Squared Error), etc. Whether the newly generated learning data is actually used for learning in the task model learning and inference unit 22 or the like may be determined by comparing it with the learning data group generated in the previous loop processing.

[0109] The data generation condition specification unit outputs information specifying such conditions to the learning data generation unit 41. The process of step S50 in FIG.

[0110] The data generation condition designation unit 42 automatically determines what kind of learning data should be generated.

[0111] In this way, by specifying the conditions for the learning data to be generated in the next loop processing based on the processing results of the previous loop processing, learning data can be generated efficiently, and the time required for learning can be shortened.

[0112] Learning as to what kind of learning data should be generated may be performed using a genetic algorithm, etc. This learning is performed based on the difference s calculated using each piece of learning data and the learning data generation parameters used to generate the learning data.

[0113] As described above, the learning device 1 makes it possible to select learning data suitable for learning without manual intervention. In addition, it becomes possible to efficiently learn an inference model using the selected learning data.

[0114] <<3. Configuration of the inference side>> FIG. 10 is a block diagram showing an example of the configuration of the inference device 101.

[0115] 10, the inference device 101 is provided with a task execution unit 111 having a task model #3 output from the learning device 1. Target data #21 is input to the task execution unit 111. The target data #21 is the same type of data as the target data constituting the target data group #2.

[0116] The task execution unit 111 inputs target data #21 input as a processing target to the task model #3 and outputs an inference result #22. For example, when the task model #3 prepared in the task execution unit 111 is an inference model for a region segmentation task and an RGB image is input as the target data #21, the result of the region segmentation is output as the inference result #22.

[0117] <<4.Other>> The learning of the model T and the model T' performed in the learning device 1 is performed by learning a model using any one of regression, decision tree, neural network, Bayes, clustering, and time series prediction.

[0118] The learning of each of the models T and T' may be performed by ensemble learning.

[0119] Computer configuration example The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0120] FIG. 11 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes according to a program.

[0121] The learning device 1 and the inference device 101 are realized by a computer as shown in Fig. 11. The learning device 1 and the inference device 101 may be realized in the same computer, or may be realized in different computers.

[0122] A central processing unit (CPU) 201, a read only memory (ROM) 202, and a random access memory (RAM) 203 are connected to each other via a bus 204.

[0123] An input / output interface 205 is further connected to the bus 204. An input unit 206 including a keyboard, a mouse, etc., and an output unit 207 including a display, a speaker, etc. are connected to the input / output interface 205. In addition, a storage unit 208 including a hard disk, a non-volatile memory, etc., a communication unit 209 including a network interface, etc., and a drive 210 that drives a removable medium 211 are connected to the input / output interface 205.

[0124] In the computer configured as above, the CPU 201 loads a program stored in the storage unit 208 into the RAM 203 via the input / output interface 205 and the bus 204 and executes the program, thereby performing the above-mentioned series of processes.

[0125] The programs executed by the CPU 201 are installed in the storage unit 208 by being recorded on, for example, a removable medium 211, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0126] The program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0127] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0128] The effects described herein are merely examples and are not limiting, and other effects may also be present.

[0129] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0130] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0131] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0132] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0133] Example of configuration combination The present technology can also be configured as follows.

[0134] (1) The present invention further includes an information processing unit that selects, from a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed at the time of inference, the learning data suitable for learning an inference model used at the time of inference from the learning data group, and outputs the selected learning data together with the inference model obtained by performing learning using the selected learning data. Learning device. (2) The information processing unit performs processing including selection of the learning data based on the learning data group and the processing target data group input from outside. The learning device according to (1) above. (3) a data acquisition unit that randomly acquires the learning data from the learning data group; a first learning unit that learns a first model using the randomly acquired learning data; The learning device according to (1) or (2), further comprising: (4) a second learning unit that learns a second model using the processing target data as an input and the provisional answer as an output, by inputting the processing target data into the first model and obtaining an inference result as a provisional answer; The learning device according to (3) above. (5) a data comparison unit that compares a first inference result obtained by inputting the randomly acquired learning data into the first model with a second inference result obtained by inputting the same learning data into the second model; a data selection unit that selects the learning data suitable for learning the inference model based on a comparison result; The learning device according to (4) further comprises: (6) The data selection unit selects the learning data used as an input for inferring the second inference result, the difference of which from the first inference result is smaller than a threshold value, as the learning data suitable for learning the inference model. The learning device according to (5) above. (7) the data acquisition unit randomly selects other learning data in place of the learning data not selected by the data selection unit; the first learning unit repeatedly learns the first model using the learning data selected by the data selection unit and other learning data randomly acquired; the second learning unit repeatedly learns the second model using an inference result of the first model obtained by learning by the first learning unit; The present invention further includes an output unit that outputs the first model obtained by repeating the learning as the inference model together with the learning data selected by the data selection unit. The learning device according to (5) or (6). (8) The learning data is at least one of RGB data, polarization data, multispectral data, and invisible light wavelength data. A learning device according to any one of (1) to (7). (9) The learning data is data detected by a sensor or data generated by a computer. A learning device according to any one of (1) to (8). (10) The first model and the second model are each trained by training a model using any one of regression, decision tree, neural network, Bayes, clustering, and time series prediction. The learning device according to (4) above. (11) a training data generation unit that generates the training data group based on a three-dimensional model of an object; The information processing unit performs processing including selection of the learning data based on the generated learning data group and the input processing target data group. The learning device according to (1) above. (12) The learning data generation unit generates the learning data group including the learning data including the data of the rendering result of the object and having a simulation result of the state of the object as a correct answer. The learning device according to (11) above. (13) a first learning unit that uses the generated learning data to learn a first model; a second learning unit that learns a second model using the processing target data as an input and the provisional answer as an output, by inputting the processing target data into the first model and obtaining an inference result as a provisional answer; The learning device according to (11) or (12). (14) a data comparison unit that compares a first inference result obtained by inputting the generated learning data to the first model with a second inference result obtained by inputting the same learning data to the second model; a data selection unit that selects the learning data suitable for learning the inference model based on a comparison result; The learning device according to (13) above, further comprising: (15) The method further includes a condition designation unit that designates conditions for the newly generated learning data based on the learning data used as an input for inferring the second inference result, the difference of which from the first inference result is smaller than a threshold value. The learning device according to (14) above. (16) The learning device selecting, from the learning data group, learning data suitable for learning an inference model used during inference, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed during inference; Outputting the selected learning data; The inference model is generated by performing learning using the selected learning data. Generation method. (17) On the computer, selecting, from the learning data group, learning data suitable for learning an inference model used during inference, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed during inference; Outputting the selected learning data; The inference model is generated by performing learning using the selected learning data. A program for executing a process. (18) The inference unit selects, from the learning data group, learning data suitable for learning an inference model used at the time of inference, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed at the time of inference, inputs the data to be processed to the inference model output from a learning device that outputs the selected learning data together with the inference model obtained by performing learning using the selected learning data, and outputs an inference result that indicates the result of a predetermined process. Reasoning device. (19) The inference device selecting, from the learning data group, learning data suitable for learning an inference model used during inference, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed during inference, and inputting the data to be processed into the inference model output from a learning device that outputs the selected learning data, together with the inference model obtained by performing learning using the selected learning data; Outputs inference results that represent the results of a given process Reasoning method. (20) On the computer, selecting, from the learning data group, learning data suitable for learning an inference model used during inference, based on a learning data group consisting of learning data having a correct answer and a processing target data group consisting of processing target data for learning that does not have a correct answer and corresponds to data to be processed during inference, and inputting the data to be processed into the inference model output from a learning device that outputs the selected learning data, together with the inference model obtained by performing learning using the selected learning data; Outputs inference results that represent the results of a given process A program for executing a process. [Explanation of symbols]

[0135] 1 Learning device, 11 Optimal data selection and task learning unit, 21 Learning data acquisition unit, 22 Task model learning and inference unit, 23 Task model re-learning and inference unit, 24 Data comparison unit, 25 Data selection unit, 26 Final model and optimal data output unit, 31 Optimal data generation and task learning unit, 41 Learning data generation unit, 42 Data generation condition specification unit, 101 Inference device, 111 Task execution unit

Claims

1. a first learning unit that learns a first model using a learning data group consisting of learning data having a correct answer; a second learning unit that learns a second model using an inference result obtained by inputting processing target data, which corresponds to data to be processed at the time of inference and does not have a correct answer, into the first model as a provisional correct answer; a data selection unit that selects the learning data for training an inference model based on a comparison result between a first inference result obtained by inputting the learning data into the first model and a second inference result obtained by inputting the same learning data into the second model; and Equipped with The first learning unit generates the inference model by performing learning using the selected learning data. Learning device.

2. The training data group and the processing target data group consisting of the processing target data are data input from outside. The learning device according to claim 1 .

3. A data acquisition unit that randomly acquires the learning data from the learning data group, The first learning unit learns the first model using the randomly acquired learning data. The learning device according to claim 1 .

4. The second learning unit learns the second model using the target data as an input and using the provisional correct answer as an output. The learning device according to claim 1 .

5. The data selection unit selects the learning data used as an input for inferring the second inference result, the difference of which from the first inference result is smaller than a threshold, as the learning data for training the inference model. The learning device according to claim 1 .

6. the data acquisition unit randomly acquires other learning data in place of the learning data not selected by the data selection unit; The first learning unit repeatedly learns the first model using the learning data selected by the data selection unit and other learning data randomly acquired; the second learning unit repeatedly learns the second model using an inference result of the first model obtained by learning by the first learning unit; The inference model may further include an output unit that outputs the inference model, which is the first model obtained by repeating the learning, together with the learning data selected by the data selection unit. The learning device according to claim 3.

7. The learning data is at least one of RGB data, polarization data, multispectral data, and invisible light wavelength data. The learning device according to claim 1 .

8. The learning data is data detected by a sensor or data generated by a computer. The learning device according to claim 1 .

9. The first model and the second model are each trained by training a model using any one of regression, decision tree, neural network, Bayes, clustering, and time series prediction. The learning device according to claim 1 .

10. a training data generation unit that generates the training data group based on a three-dimensional model of an object; The training data group is generated data. The learning device according to claim 1 .

11. The learning data generation unit generates the learning data group including the learning data including the data of the rendering result of the object and having a simulation result of the state of the object as a correct answer. The learning device according to claim 10.

12. The method further includes a condition designation unit that designates conditions for the newly generated learning data based on the learning data used as an input for inferring the second inference result, the difference of which from the first inference result is smaller than a threshold value. The learning device according to claim 10.

13. The learning device A first model is trained using a training data group consisting of training data having a correct answer; inputting processing target data corresponding to data to be processed at the time of inference and not having a correct answer into the first model, and learning a second model using an inference result obtained as a provisional correct answer; selecting the learning data for training an inference model based on a comparison result between a first inference result obtained by inputting the learning data into the first model and a second inference result obtained by inputting the same learning data into the second model; The inference model is generated by performing learning using the selected learning data. Generation method.

14. On the computer, A first model is trained using a training data group consisting of training data having a correct answer; inputting processing target data corresponding to data to be processed at the time of inference and not having a correct answer into the first model, and learning a second model using an inference result obtained as a provisional correct answer; selecting the learning data for training an inference model based on a comparison result between a first inference result obtained by inputting the learning data into the first model and a second inference result obtained by inputting the same learning data into the second model; The inference model is generated by performing learning using the selected learning data. A program for executing a process.

15. The inference unit includes: a learning unit that learns a first model using a learning data group consisting of learning data having a correct answer; learns a second model using an inference result obtained by inputting data to be processed that does not have a correct answer and corresponds to data to be processed at the time of inference into the first model as a provisional correct answer; selects the learning data for training the inference model based on a comparison result between a first inference result obtained by inputting the learning data into the first model and a second inference result obtained by inputting the same learning data into the second model; and inputs the data to be processed into the inference model output from a learning device that generates the inference model by performing learning using the selected learning data, and outputs the inference result. Reasoning device.

16. The inference device a first model is trained using a group of training data consisting of training data having a correct answer, and a second model is trained using an inference result obtained by inputting data to be processed that does not have a correct answer and corresponds to data to be processed during inference into the first model as a provisional correct answer, and the training data for training the inference model is selected based on a comparison result between a first inference result obtained by inputting the training data into the first model and a second inference result obtained by inputting the same training data into the second model, and the data to be processed is input into the inference model output from a training device that generates the inference model by performing training using the selected training data; Output the inference result Reasoning method.

17. On the computer, a first model is trained using a group of training data consisting of training data having a correct answer, and a second model is trained using an inference result obtained by inputting data to be processed that does not have a correct answer and corresponds to data to be processed during inference into the first model as a provisional correct answer, and the training data for training the inference model is selected based on a comparison result between a first inference result obtained by inputting the training data into the first model and a second inference result obtained by inputting the same training data into the second model, and the data to be processed is input into the inference model output from a training device that generates the inference model by performing training using the selected training data; Output the inference result A program for executing a process.

Images