Data generation system, industrial machine, data generation method, and data generation program

Abstract

This data generation system comprises an acquisition unit that acquires an evaluation value corresponding to an evaluation of a target object, and a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value on the basis of the acquired evaluation value and a trained model trained to receive input of an evaluation value to output pseudo data representing a target object.

Description

[Technical Field]

[0001] One aspect of the present disclosure relates to a data generation system, an industrial machine, a data generation method, and a data generation program. [Background technology]

[0002] Patent document 1 describes an information processing system that trains a pseudo image generation model, which is a machine learning model that converts a first domain image, which is an image in a first domain of an object to be recognized, into a pseudo second domain image, which is an image that resembles a second domain image, which is an image in a second domain. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-95364 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need for a mechanism for easily generating pseudo data of an object. [Means for solving the problem]

[0005] A data generation system according to one aspect of the present disclosure includes an acquisition unit that acquires an evaluation value corresponding to an evaluation of an object, and a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input.

[0006] A data generation method according to one aspect of the present disclosure is a data generation method executed by a data generation system having at least one processor, and includes the steps of acquiring an evaluation value corresponding to an evaluation of an object, and generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input.

[0007] A data generation program according to one aspect of the present disclosure causes a computer to execute the steps of acquiring an evaluation value corresponding to an evaluation of an object, and generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input. [Effects of the Invention]

[0008] According to one aspect of the present disclosure, pseudo data of an object can be easily generated. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an application and functional configuration of a data generation system. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer used in the data generation system. [Figure 3] FIG. 1 is a diagram illustrating an example of a machine learning model (trained model). [Figure 4] FIG. 1 is a diagram showing an overview of the learning phase. [Figure 5] 10 is a flowchart illustrating an example of a learning phase. [Figure 6] FIG. 1 is a diagram illustrating an overall view of the operation phase. [Figure 7] 10 is a flowchart illustrating an example of an operation phase. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and redundant description will be omitted.

[0011] [System Overview] A data generation system according to the present disclosure is a computer system that generates pseudo data representing a pseudo object. The data generation system acquires an evaluation value corresponding to an evaluation of the object, and generates the pseudo data based on the evaluation value and a trained model that is generated at least in part by machine learning.

[0012] An object refers to a certain type of tangible object. In one example, an object is a tangible object with an indefinite appearance. "Indefinite appearance" means that individual appearances are not exactly the same. An example of a tangible object with an indefinite appearance is fresh produce; therefore, the object may be a type of vegetable, fruit, fish, or meat. A pseudo object refers to a virtual object generated on a computer system to resemble a real object. Pseudo data is electronic data representing a pseudo object. Pseudo data may be an image of the object, and in this disclosure, the image is referred to as a pseudo image. Pseudo data may be model data showing the three-dimensional shape of the object, and in this disclosure, the model data is referred to as a pseudo three-dimensional model.

[0013] The evaluation of an object refers to the value of the object judged qualitatively. In one example, the evaluation of an object is a classification of its appearance. For example, the classification is determined comprehensively based on various elements of the object's appearance, such as color, size, shape, and gloss. The evaluation of an object may be related to its quality. The evaluation value is a value that quantitatively indicates the evaluation of the object. In one example, the evaluation value indicates the quality of the object. For example, the evaluation value may be expressed by a continuous value or a discrete value. Regardless of whether the evaluation value is a continuous value or a discrete value, the evaluation value may indicate a classification of the object's appearance or quality. For example, the classification may be a class (grade) such as rank A, B, or C. When the evaluation value is a continuous value, for example, the class "Rank C" may correspond to an evaluation value between 0 and 1, the class "Rank B" may correspond to an evaluation value between 1 and 2, and the class "Rank A" may correspond to an evaluation value between 2 and 3. When the evaluation value is a discrete value, the correspondence between the discrete value and the class may be one-to-one or N-to-one.

[0014] Machine learning is a method of autonomously discovering laws or rules by repeatedly learning based on given information. A trained model is constructed using an algorithm and a data structure. At least a part of the trained model is generated by machine learning. Generative Adversarial Networks (GAN) may be used as a machine learning architecture for generating pseudo data. By using a trained model, the work of generating pseudo data can be quantified from evaluation values, which are qualitative indicators.

[0015] In one example, a data generation system generates pseudo data based on an evaluation value input by a user and a trained model. When a user specifies an evaluation value based on an ambiguous judgment based on their own intuition or tacit knowledge, the data generation system generates pseudo data corresponding to the evaluation value. The data generation system can automate the generation of pseudo data involving such ambiguous judgment. The user can also compare the generated pseudo data with the ambiguous judgment based on their own intuition or tacit knowledge, and specify or change the evaluation value or its range so that pseudo data closer to their own judgment is generated.

[0016] [System Configuration] FIG. 1 is a diagram illustrating an example of the application and functional configuration of a data generation system 10 according to an embodiment. In this example, the data generation system 10 includes functional modules: a model generation unit 11, an acquisition unit 12, and a data generation unit 13. The model generation unit 11 is a functional module that generates at least a portion of a trained model 20. The trained model 20 is a computational model trained to output pseudo data in response to an input evaluation value. The acquisition unit 12 is a functional module that acquires an evaluation value corresponding to the evaluation of an object. The data generation unit 13 is a functional module that generates pseudo data representing a pseudo object corresponding to the evaluation value based on the acquired evaluation value and the generated trained model 20. The "pseudo object corresponding to the evaluation value" refers to a pseudo object generated to be judged to have a value indicated by the evaluation value. The generation of the trained model 20 by the model generation unit 11 corresponds to the learning phase. The use of the trained model 20 by the acquisition unit 12 and the data generation unit 13 corresponds to the operation phase or the inference phase.

[0017] The data generation system 10 can be realized by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes specific processing. The data generation system 10 may be realized by a single computer or a distributed system having multiple computers.

[0018] In one example, the data generating system 10 connects to a first database 30 and a second database 40 via a communications network.

[0019] The first database 30 is a device that stores a training dataset used to generate the trained model 20. In one example, the training dataset includes multiple records corresponding to multiple real objects. Each record includes data items necessary for generating the trained model 20. For example, each record includes an RGB image, a mask image, a 3D model, an evaluation value, and a class for a given real object. The RGB image represents the projected shape and texture of the object. The mask image represents the two-dimensional shape of the object. In one example, the mask image is a binary image in which pixel values ​​representing the object are a first color (e.g., white) and other pixel values ​​are a second color (e.g., black). The three-dimensional model represents the three-dimensional shape and texture of the object. Texture refers to elements other than shape that characterize an object. For example, texture is represented by color, gloss, etc.

[0020] The second database 40 is a device that stores the simulated data generated by the data generation system 10. In one example, the second database 40 stores at least one of a simulated image and a simulated three-dimensional model for each simulated object.

[0021] In the example of FIG. 1 , the pseudo data is used to operate an industrial machine 60. The second database 40 is connected to a control model generation device 50 via a communication network. The control model generation device 50 is a computer that performs machine learning based on the pseudo data stored in the second database 40 to generate a control model 70. The control model 70, trained based on the pseudo data, has a function of, for example, evaluating a real object corresponding to a pseudo object and estimating a control value related to the evaluation. The control model 70 is provided to the industrial machine 60 via a communication network. The industrial machine 60 is a device that executes predetermined processing based on the control model 70. The industrial machine 60 may be any device, such as a determiner, a robot controller, or a robot. In one example, the industrial machine 60 includes an output unit 61 that outputs a control value according to the control model 70. For example, the output unit 61 may output the control value to another device, such as a monitor or a robot.

[0022] The communication network connecting the devices may be a wired network or a wireless network. The communication network may include at least one of the Internet and an intranet. Alternatively, the communication network may be realized simply by a single communication cable.

[0023] 2 is a diagram showing an example of the hardware configuration of a computer 100 used in the data generation system 10. In this example, the computer 100 includes a main body 110, a monitor 120, and an input device .

[0024] The main body 110 is a device that executes the main functions of the computer. The main body 110 includes a circuit 160. The circuit 160 includes at least one processor 161, a memory 162, a storage 163, an input / output port 164, and a communication port 165. The storage 163 stores programs for configuring each functional module of the main body 110. The storage 163 is a computer-readable recording medium such as a hard disk, a nonvolatile semiconductor memory, a magnetic disk, or an optical disk. The memory 162 temporarily stores programs loaded from the storage 163 and the results of calculations by the processor 161. The processor 161 configures each functional module by executing programs in cooperation with the memory 162. The input / output port 164 inputs and outputs electrical signals to and from the monitor 120 or the input device 130 in response to instructions from the processor 161. The input / output port 164 may also input and output electrical signals to and from other devices. The communication port 165 performs data communication with other devices via a communication network N in response to instructions from the processor 161.

[0025] Monitor 120 is a device for displaying information output from main body 110. Monitor 120 may be any device capable of displaying graphics, and a specific example thereof is a liquid crystal panel.

[0026] The input device 130 is a device for inputting information to the main body 110. The input device 130 may be any device that can input desired information, and specific examples include operation interfaces such as a keypad, a mouse, and an operation controller.

[0027] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main body 110, the monitor 120, and the input device 130 may be integrated as a tablet computer.

[0028] Each functional module of data generation system 10 is realized by loading a data generation program onto processor 161 or memory 162 and having processor 161 execute the program. The data generation program includes code for realizing each functional module of data generation system 10. Processor 161 operates input / output port 164 or communication port 165 in accordance with the data generation program, and executes reading and writing of data from and to memory 162 or storage 163.

[0029] The data generation program may be provided by being permanently recorded on a non-transitory recording medium such as a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the data generation program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0030] [Pre-trained model] 3 is a diagram showing an example of a machine learning model 200 used to generate the trained model 20. The machine learning model 200 can be said to be a pre-complete trained model 20. In one example, the machine learning model 200 includes a conversion model 210, a data generation model 220, and an evaluation model 230.

[0031] The conversion model 210 is a computational model that converts the rating value into a latent variable. The conversion model 210 can be said to be a mapping from the rating value to the latent variable. The latent variable is data that indicates n-dimensional features for generating pseudo data. In one example, the conversion model 210 includes a converter 211 and an adder 212. The converter 211 is a component that converts the rating value into a latent variable. The adder 212 converts another latent variable Z that is set by a probability distribution such as a uniform distribution. s The transformation model 210 is a component that applies the transformed latent variable Z s Outputs the latent variables to which

[0032] The data generation model 220 is a computational model that generates pseudo data based on the latent variables generated by the transformation model 210. In one example, the data generation model 220 includes a shape generation model 221, a texture generation model 223, and a 3D shape generation model 224. The shape generation model 221 is a computational model that outputs a mask image 301 that indicates the shape of an object based on the latent variables generated by the transformation model 210. The texture generation model 223 generates pseudo data based on the mask image 301 and a latent variable Z that is set by a probability distribution such as a uniform distribution. t The 3D shape generation model 224 is a computational model that outputs a pseudo-image 302 that represents the projected shape and texture of the object based on the pseudo-image 302. The pseudo-image 302 may be an RGB image. The 3D shape generation model 224 is a computational model that outputs a pseudo 3D model 303 that represents the 3D shape and texture of the pseudo object based on the pseudo-image 302.

[0033] The evaluation model 230 is a calculation model that calculates an evaluation value based on pseudo data. In this disclosure, in relation to the learning phase, the evaluation value input to the conversion model 210 is also referred to as a "first evaluation value," and the evaluation value output from the evaluation model 230 is referred to as a "second evaluation value," thereby distinguishing between the two types of evaluation values ​​as necessary.

[0034] 3 illustrates a strawberry as an example of an object. A mask image 301 represents the two-dimensional shape of the strawberry, a pseudo image 302 represents the projected shape and texture of the strawberry, and a pseudo three-dimensional model 303 represents the three-dimensional shape and texture of the strawberry.

[0035] [System Operation] (Generating a trained model) Machine learning is performed on at least a part of the machine learning model 200 to generate a trained model 20. Figure 4 shows an overview of the learning phase.

[0036] FIG. 4 shows the transformation model 210, the data generation model 220, and the evaluation model 230, as well as a feature space 410 and a latent space 420. The feature space 410 is a coordinate space that defines evaluation values. In the example of FIG. 4, each point in the feature space 410 represents an evaluation value that changes continuously. That is, the evaluation values ​​in this example are expressed as continuous values. In the example of FIG. 4, these evaluation values ​​are classified into three classes, Qa, Qb, and Qc. The latent space 420 is a coordinate space that defines latent variables. Each point in the latent space 420 represents a latent variable.

[0037] The data generation model 220 is generated by machine learning using individual latent variables and a training dataset. The evaluation model 230 is generated by machine learning using a training dataset. The conversion model 210 is generated by machine learning using evaluation values. In training the conversion model 210, the conversion model 210 converts an input first evaluation value into a latent variable, the data generation model 220 generates pseudo data based on the latent variable, and the evaluation model 230 outputs a second evaluation value based on the pseudo data. The evaluation model 230 is updated based on the first evaluation value and the second evaluation value in the feature space 410.

[0038] 5 is a flowchart showing an example of the learning phase as a process flow S1. That is, the data generation system 10 executes the process flow S1.

[0039] In step S11, the model generation unit 11 generates the shape generation model 221. In one example, the model generation unit 11 generates the shape generation model 221 by machine learning (for example, machine learning using GAN) based on a plurality of latent variables and a plurality of mask images of a training dataset.

[0040] In step S12, the model generation unit 11 generates the texture generation model 223. In one example, the model generation unit 11 generates the texture generation model 223 by machine learning (e.g., machine learning using GAN) based on a plurality of RGB images and mask images of a training dataset.

[0041] In step S13, the model generation unit 11 generates the 3D shape generation model 224. In one example, the model generation unit 11 generates the 3D shape generation model 224 by machine learning (e.g., machine learning using GAN) based on a plurality of RGB images and a plurality of 3D models of a training dataset.

[0042] As shown in steps S11 to S13, the model generation unit 11 generates the data generation model 220 so as to output pseudo data based on latent variables. In this process, by introducing machine learning using GAN, it becomes possible to generate the data generation model 220 that generates pseudo data of a pseudo object that is likely to exist in reality.

[0043] In step S14, the model generation unit 11 generates the evaluation model 230. In one example, the model generation unit 11 generates the evaluation model 230 by machine learning based on a plurality of RGB images and a plurality of evaluation values ​​of a training dataset.

[0044] In step S15, the model generation unit 11 generates the conversion model 210. In one example, the model generation unit 11 executes the following process to complete the conversion model.

[0045] The model generation unit 11 inputs the first evaluation value into the uncompleted transformation model 210 to generate a latent variable. The model generation unit 11 generates a pseudo image (e.g., a pseudo RGB image) corresponding to the first evaluation value based on the latent variable and the data generation model 220. The model generation unit 11 inputs the latent variable into the data generation model 220. The data generation model 220 executes processing using a shape generation model 221 and a texture generation model 223. The model generation unit 11 acquires a pseudo image obtained by the processing. The model generation unit 11 inputs the pseudo image into the evaluation model 230 to calculate a second evaluation value. The model generation unit 11 updates the uncompleted transformation model 210 based on the first evaluation value and the second evaluation value.

[0046] The model generation unit 11 executes this series of processes for each of the multiple first evaluation values, and finally generates the conversion model 210. For example, the model generation unit 11 generates the conversion model 210 so that the first evaluation value and the second evaluation value match. Alternatively, the model generation unit 11 may generate the conversion model 210 so that the error between the first evaluation value and the second evaluation value is less than a predetermined threshold. In this way, by repeatedly updating the uncompleted conversion model 210 while comparing the second evaluation value with the first evaluation value, and finally generating the conversion model 210, a one-to-one correspondence between the first evaluation value and the second evaluation value is achieved. In other words, the conversion model 210 becomes able to output latent variables for generating pseudo data corresponding to a specified evaluation value.

[0047] In processing flow S1, the model generation unit 11 may generate the data generation model 220 and then generate the conversion model 210. That is, the model generation unit 11 may execute steps S11 to S13 and then execute step S15. The model generation unit 11 may generate the data generation model 220 and then generate the evaluation model 230, and then generate the conversion model 210 using the data generation model 220 and the evaluation model 230. That is, the model generation unit 11 may execute steps S11 to S13 and then execute step S14, and then execute step S15.

[0048] The process flow S1 obtains the trained model 20. Note that the trained model 20 is a computational model that is estimated to be optimal, and is not necessarily a "computational model that is actually optimal."

[0049] (Generating pseudo data) In one example, a combination of the conversion model 210 and the data generation model 220 obtained by the learning phase is provided as the trained model 20. The trained model 20 may or may not include an evaluation model 230. In the operation phase (inference phase), pseudo data is generated using the trained model 20. Figure 6 shows an overview of the operation phase.

[0050] In the operation phase, an evaluation value 411 for generating pseudo data is input to the trained model 20. In the present disclosure, the evaluation value 411 is also referred to as a specified evaluation value. Naturally, the evaluation value 411 is a value identified in the feature space 410. In the trained model 20, the conversion model 210 converts the evaluation value 411 into a latent variable 421 in the latent space 420. Next, the data generation model 220 generates at least one of a pseudo image 302 and a pseudo 3D model 303 as pseudo data based on the latent variable 421.

[0051] 7 is a flowchart showing an example of the operation phase as a processing flow S2. That is, the data generation system 10 executes the processing flow S2.

[0052] In step S21, the acquisition unit 12 acquires an evaluation value (specified evaluation value). In one example, the acquisition unit 12 acquires an evaluation value input by a user. Alternatively, the acquisition unit 12 may receive an evaluation value transmitted from another computer, or may read an evaluation value stored in a predetermined storage device.

[0053] In step S22, the data generation unit 13 converts the evaluation value into a latent variable. The data generation unit 13 generates the latent variable based on the evaluation value and the conversion model 210. The data generation unit 13 inputs the evaluation value to the conversion model 210 and obtains the latent variable output from the conversion model 210.

[0054] In step S23, the data generation unit 13 generates a mask image corresponding to the evaluation value based on the latent variables and the shape generation model 221. The data generation unit 13 inputs the latent variables to the shape generation model 221 and acquires the mask image output from the shape generation model 221.

[0055] In step S24, the data generation unit 13 generates a pseudo image corresponding to the evaluation value based on the mask image 301 and the texture generation model 223. The data generation unit 13 generates a pseudo image corresponding to the evaluation value based on the mask image 301 and the latent variable Z tis input to the texture generation model 223, and a pseudo image is output from the texture generation model 223.

[0056] In step S25, the data generation unit 13 generates a pseudo 3D model corresponding to the evaluation value based on the pseudo image and the 3D shape generation model 224. The data generation unit 13 inputs the pseudo image to the 3D shape generation model 224 and acquires the pseudo 3D model output from the 3D shape generation model 224.

[0057] In step S26, the data generation unit 13 stores the pseudo three-dimensional model in the second database 40. As described above, the pseudo three-dimensional model is used to generate the control model .

[0058] In step S27, the user adjusts the evaluation value or its range as needed. The user compares the generated pseudo data (pseudo image or pseudo 3D model) with their own judgment based on intuition or tacit knowledge. Then, the user adjusts the evaluation value or its range so that pseudo data closer to their own judgment is generated. For example, the user specifies or changes the evaluation value or its range. Since the user makes such adjustments as needed, step S27 can be omitted.

[0059] The process flow S2 can be executed repeatedly, and by this repeated process, a plurality of pseudo three-dimensional models of a certain type of object (for example, strawberries) are accumulated in the second database 40.

[0060] The control model generation device 50 performs machine learning based on the pseudo three-dimensional model stored in the second database 40 to generate a control model 70. The industrial machine 60 outputs a control value based on the control model 70.

[0061] The pseudo-3D model in the second database 40 may be used for other purposes. For example, the data generation system 10 may display the pseudo-3D model on the monitor 120 in the form of a computer graphic (CG) moving image or still image.

[0062] As shown in steps S22 to S25, the data generation unit 13 generates a pseudo 3D model based on the latent variables and the data generation model 220. The data generation unit 13 inputs the latent variables to the data generation model 220 and obtains a pseudo 3D model output from the data generation model 220. Therefore, in this example, the data generation unit 13 generates a pseudo 3D model as pseudo data.

[0063] In step S26, the data generation unit 13 may store a pseudo image in addition to or instead of the pseudo 3D model in the second database 40. That is, the data generation unit 13 may generate at least one of a pseudo image and a pseudo 3D model as pseudo data.

[0064] [Variations] Various examples of the present disclosure have been described above in detail. However, the present disclosure is not limited to these examples. Various modifications of the present disclosure are possible without departing from the spirit and scope of the present disclosure.

[0065] The texture generation model uses the mask image and the latent variable Z t In addition to the mask image, the pseudo image may be output based on the latent variable obtained by the transformation model. In this case, the data generation unit generates a mask image and the latent variable Z t and the latent variables obtained by the transformation model are input to the texture generation model, and a pseudo image is output from the texture generation model. Alternatively, the texture generation model may output a pseudo image based on the latent variables obtained by the transformation model without using a mask image. In this example, the shape generation model is omitted. The data generation unit inputs the latent variables obtained by the transformation model to the texture generation model, and a pseudo image is output from the texture generation model.

[0066] The conversion model may be generated without using machine learning. For example, a function that converts the evaluation value into a latent variable so that the first evaluation value and the second evaluation value match or so that the error between the two evaluation values ​​is less than a predetermined threshold may be prepared as the conversion model.

[0067] The data generation model does not have to include a three-dimensional shape generation model. In this case, the data generation unit generates a pseudo image as pseudo data.

[0068] The data generation system does not need to include a model generation unit. Trained models are portable between computer systems. Therefore, the data generation unit may use trained models generated in another computer system.

[0069] The hardware configuration of the system is not limited to the implementation of each functional module by executing a program. For example, at least some of the functional modules may be configured with logic circuits specialized for the respective functions, or may be configured with an ASIC (Application Specific Integrated Circuit) that integrates such logic circuits.

[0070] The processing steps of the method executed by at least one processor are not limited to the above examples. For example, some of the steps or processes described above may be omitted, or the steps may be performed in a different order. Also, two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be performed in addition to the steps described above.

[0071] When comparing the magnitude of two numbers within a computer system or computer, either of the two criteria "greater than or equal to" and "greater than" can be used, or either of the two criteria "less than or equal to" and "under".

[0072] [Note] As can be seen from the various examples above, the present disclosure includes the following aspects. (Appendix 1) an acquisition unit that acquires an evaluation value corresponding to an evaluation of the object; a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; and A data generation system comprising: (Appendix 2) the object is an object with an indefinite appearance, The evaluation is a classification of the appearance. 2. The data generation system of claim 1. (Appendix 3) the acquisition unit acquires the evaluation value input by a user. 3. The data generation system of claim 1 or 2. (Appendix 4) the trained model includes a data generation model that outputs the pseudo data based on latent variables; The data generation unit converting the evaluation values ​​into the latent variables; generating the pseudo data based on the latent variables and the data generation model; 4. A data generation system according to any one of appendices 1 to 3. (Appendix 5) Further comprising a model generation unit that generates at least a part of the trained model, The model generation unit generating the data generation model to output the pseudo data based on the latent variables; After generating the data generation model, a conversion model is generated that converts the evaluation value into the latent variable. 5. The data generation system of claim 4. (Appendix 6) The model generation unit After generating the data generation model, an evaluation model is generated that calculates the evaluation value based on the pseudo data; after generating the evaluation model, generating the transformation model using the data generation model and the evaluation model; 6. The data generation system of claim 5. (Appendix 7) In generating the transformation model, the model generation unit inputting the first evaluation value into the transformation model before completion to generate the latent variables; generating the pseudo data corresponding to the first evaluation value based on the latent variables and the data generation model; inputting the pseudo data into the evaluation model to calculate a second evaluation value; updating the uncompleted transformation model based on the first evaluation value and the second evaluation value to generate the transformation model; 7. The data generation system of claim 6. (Appendix 8) the data generation unit generates a pseudo image as the pseudo data. A data generation system according to any one of appendices 1 to 7. (Appendix 9) the data generation unit generates a pseudo three-dimensional model as the pseudo data. A data generation system according to any one of appendices 1 to 8. (Appendix 10) The data generation model is a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables; a texture generation model that outputs the pseudo-data representing the shape and texture of the object based on the mask image; Equipped with The data generation unit generating the mask image corresponding to the acquired evaluation value based on the latent variables and the shape generation model; generating the pseudo data based on the generated mask image and the texture generation model; A data generation system according to any one of appendices 4 to 9. (Appendix 11) 11. An industrial machine comprising an output unit that outputs a control value according to a control model trained on the basis of pseudo data generated by the data generation system according to any one of Supplementary Notes 1 to 10. (Appendix 12) 1. A data generation method performed by a data generation system comprising at least one processor, comprising: obtaining a rating value corresponding to a rating of the object; generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; A data generation method comprising: (Appendix 13) obtaining a rating value corresponding to a rating of the object; generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; A data generation program that causes a computer to execute the above.

[0073] According to Supplements 1, 12, and 13, pseudo data representing a pseudo object corresponding to an evaluation value is automatically obtained based on a trained model, so that pseudo data of the object can be easily generated.

[0074] According to Supplementary Note 2, pseudo data of objects with variations in appearance are generated. Classification of appearance is impossible or very difficult to quantify. The configuration of Supplementary Note 2 makes it possible to generate pseudo data corresponding to evaluations that are impossible or difficult to quantify.

[0075] According to Supplementary Note 3, pseudo data corresponding to evaluations based on qualitative judgments by users (for example, vague judgments based on the user's feelings) can be generated.

[0076] According to Appendix 4, by using latent variables, it is possible to generate diverse pseudo-data from evaluation values ​​with limited dimensions or ranges.

[0077] According to Appendix 5, a highly accurate trained model can be achieved by generating a conversion model after generating a data generation model.

[0078] According to Appendix 6, by adopting an evaluation model, in addition to the evaluation value used to obtain the latent variable, an evaluation value calculated by the evaluation model can be obtained. By using these two types of evaluation values, a highly accurate conversion model can be realized.

[0079] According to Supplementary Note 7, the conversion model is updated based on the first evaluation value input to generate pseudo data and the second evaluation value calculated based on the generated pseudo data. This update can improve the accuracy of the conversion model for obtaining latent variables.

[0080] According to Appendix 8, a pseudo-image of an object can be easily generated.

[0081] According to Appendix 9, a pseudo-3D model of an object can be easily generated.

[0082] According to Supplementary Note 10, since the shape and texture are generated using separate computational models, pseudo data representing pseudo objects that are likely to exist in reality can be generated more reliably.

[0083] According to Supplementary Note 11, since pseudo data can be easily obtained, learning of a control model for operating industrial machinery can also be easily performed. As a result, industrial machinery that outputs desired control values ​​can be easily realized. [Explanation of symbols]

[0084] 10...data generation system, 11...model generation unit, 12...acquisition unit, 13...data generation unit, 20...trained model, 30...first database, 40...second database, 50...control model generation device, 60...industrial machinery, 61...output unit, 70...control model, 200...machine learning model, 210...conversion model, 220...data generation model, 221...shape generation model, 223...texture generation model, 224...3D shape generation model, 230...evaluation model, 301...mask image, 302...pseudo image, 303...pseudo 3D model.

Claims

1. an acquisition unit that acquires an evaluation value corresponding to an evaluation of the object; a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; and Equipped with the trained model includes a data generation model that outputs the pseudo data based on latent variables; The data generation model is a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables; a texture generation model that outputs the pseudo-data representing the shape and texture of the object based on the mask image; Including, The data generation unit converting the evaluation values ​​into the latent variables; generating the mask image corresponding to the acquired evaluation value based on the latent variables and the shape generation model; generating the pseudo data based on the generated mask image and the texture generation model; Data generation system.

2. the object is an object with an indefinite appearance, The evaluation is a classification of the appearance. The data generation system of claim 1 .

3. the acquisition unit acquires the evaluation value input by a user. The data generation system of claim 1 .

4. Further comprising a model generation unit that generates at least a part of the trained model, The model generation unit generating the data generation model to output the pseudo data based on the latent variables; After generating the data generation model, a conversion model is generated that converts the evaluation value into the latent variable. The data generation system according to any one of claims 1 to 3.

5. The model generation unit After generating the data generation model, an evaluation model is generated that calculates the evaluation value based on the pseudo data; after generating the evaluation model, generating the transformation model using the data generation model and the evaluation model; The data generation system of claim 4 .

6. In generating the transformation model, the model generation unit inputting the first evaluation value into the transformation model before completion to generate the latent variables; generating the pseudo data corresponding to the first evaluation value based on the latent variables and the data generation model; inputting the pseudo data into the evaluation model to calculate a second evaluation value; updating the uncompleted transformation model based on the first evaluation value and the second evaluation value to generate the transformation model; The data generation system of claim 5 .

7. the data generation unit generates a pseudo image as the pseudo data. The data generation system according to any one of claims 1 to 3.

8. the data generation unit generates a pseudo three-dimensional model as the pseudo data. The data generation system according to any one of claims 1 to 3.

9. An industrial machine comprising an output unit that outputs a control value according to a control model trained on the basis of pseudo data generated by the data generation system according to any one of claims 1 to 3.

10. 1. A data generation method performed by a data generation system comprising at least one processor, comprising: obtaining a rating value corresponding to a rating of the object; generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; Including, the trained model includes a data generation model that outputs the pseudo data based on latent variables; The data generation model is a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables; a texture generation model that outputs the pseudo-data representing the shape and texture of the object based on the mask image; Including, In the step of generating the pseudo data, converting the evaluation values ​​into the latent variables; generating the mask image corresponding to the acquired evaluation value based on the latent variables and the shape generation model; generating the pseudo data based on the generated mask image and the texture generation model; Data generation method.

11. obtaining a rating value corresponding to a rating of the object; generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; on the computer, the trained model includes a data generation model that outputs the pseudo data based on latent variables; The data generation model is a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables; a texture generation model that outputs the pseudo-data representing the shape and texture of the object based on the mask image; Including, In the step of generating the pseudo data, converting the evaluation values ​​into the latent variables; generating the mask image corresponding to the acquired evaluation value based on the latent variables and the shape generation model; generating the pseudo data based on the generated mask image and the texture generation model; Data generator.

12. An acquisition unit that acquires an evaluation value corresponding to an evaluation of an object; a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; and a model generation unit that generates at least a portion of the trained model; Equipped with the trained model includes a data generation model that outputs the pseudo data based on latent variables; The model generation unit generating the data generation model to output the pseudo data based on the latent variables; After generating the data generation model, an evaluation model is generated that calculates the evaluation value based on the pseudo data; After generating the evaluation model, using the data generation model and the evaluation model, generate a conversion model that converts the evaluation value into the latent variable; The data generation unit generating the latent variables based on the evaluation values ​​and the transformation model; generating the pseudo data based on the latent variables and the data generation model; Data generation system.

13. A step of obtaining an evaluation value corresponding to an evaluation of an object; generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; generating at least a portion of the trained model; on the computer, the trained model includes a data generation model that outputs the pseudo data based on latent variables; In the step of generating at least a portion of the trained model, generating the data generation model to output the pseudo data based on the latent variables; After generating the data generation model, an evaluation model is generated that calculates the evaluation value based on the pseudo data; After generating the evaluation model, using the data generation model and the evaluation model, generate a conversion model that converts the evaluation value into the latent variable; The step of generating the pseudo data comprises: generating the latent variables based on the evaluation values ​​and the transformation model; generating the pseudo data based on the latent variables and the data generation model; Data generator.

Images