DATA EXTENSION DEVICE, DATA EXTENSION METHOD, AND DATA EXTENSION PROGRAM

Abstract

To provide a data expansion device that suppresses deviation between data before and after conversion.SOLUTION: A data expansion device executes: a first conversion processing for converting a second learning sample into a pseudo first learning sample that is pseudo to a first learning sample by a domain inverse converter, and converting the pseudo first learning sample into a pseudo second learning sample that is pseudo to the second learning sample by a domain forward converter; a first calculation processing for calculating a pseudo first loss value that is a loss value based on a pseudo first output value that is an output value obtained by inputting the pseudo first learning sample output by the first conversion processing to a reference model and a second correct answer value; a second calculation processing for calculating a second loss value based on the second learning sample and the pseudo second learning sample; and a first learning processing for learning the domain inverse converter and the domain forward converter so as to reduce an integrated loss value obtained by integrating the pseudo first loss value calculated by the first calculation processing and the second loss value calculated by the second calculation processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a data expansion device, a data expansion method, and a data expansion program for expanding learning data used in machine learning. [Background technology]

[0002] Machine learning is a technology that realizes predictions based on input data using known learning methods such as neural networks. For example, there are devices that take images collected by an imaging device such as a camera as input and output what objects appear in the images.

[0003] In the above-mentioned machine learning technology, prediction accuracy can be improved by learning various patterns of data, so it is necessary to increase the number of patterns of learning data and reduce the costs associated with collecting data (for example, see Patent Document 1 below).

[0004] Patent Document 1 discloses a data augmentation technique for creating synthetic data similar to training data using a generative adversarial network (GAN). Specifically, the face image generation method of Patent Document 1 includes the steps of receiving a first image including a face input by a user in an interactive scene, presenting the first image to the user, inputting the first image into a pre-trained generative adversarial network in the background, the generative adversarial network being constrained by face attribute information generated based on the input image, and obtaining a second image output from the generative adversarial network, and presenting the second image to the user in response to obtaining the second image output from the generative adversarial network in the background. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-114279 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, data augmentation techniques using generative adversarial networks (GANs) can generate synthetic data that deviates significantly from actual collected real-world data and does not contribute to improving the accuracy of machine learning models.

[0007] An object of the present invention is to suppress the discrepancy between data before and after conversion. [Means for solving the problem]

[0008] A data extension device according to one aspect of the invention disclosed in the present application is a data extension device having a processor that executes a program and a storage device that stores the program, the storage device storing a first learning data set in a first domain that is a set of combinations of a first learning sample and a first correct answer value indicating a correct answer for the first learning sample, a second learning data set in a second domain that is a set of combinations of a second learning sample and a second correct answer value indicating a correct answer for the second learning sample, a reference model, a domain forward converter, and a domain inverse converter, the reference model being a learning model trained to reduce a first loss value based on a first output value output as a result of inputting the first learning sample into the reference model and the first correct answer value, the domain forward converter being a learning model that converts data related to the first domain into data that simulates data related to the second domain, and the domain inverse converter being a learning model that converts data related to the first domain into data that simulates data related to the second domain. the processor executes the following steps: a first conversion process for converting the second training samples into pseudo first training samples that are pseudo to the first training samples, by the domain inverse converter, and converting the pseudo first training samples into pseudo second training samples that are pseudo to the second training samples, by the domain forward converter; a first calculation process for calculating a pseudo first loss value that is a loss value based on a pseudo first output value that is an output value obtained by inputting the pseudo first training samples output by the first conversion process into the reference model, and the second ground truth value; a second calculation process for calculating a second loss value based on the second training samples and the pseudo second training samples; and a first learning process for training the domain inverse converter and the domain forward converter so that an integrated loss value obtained by integrating the pseudo first loss value calculated by the first calculation process and the second loss value calculated by the second calculation process is small. Effect of the Invention

[0009] According to the representative embodiment of the present invention, it is possible to suppress the deviation between data before and after conversion. Other objects, configurations and advantages than those mentioned above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a first example of a functional configuration of a data expansion device. [Diagram 2] FIG. 2 is a block diagram showing a second example of a functional configuration of the data expansion device. [Diagram 3] FIG. 3 is a flowchart showing an example of a learning process procedure performed by the domain reverse converter and the domain forward converter. [Figure 4] FIG. 4 is a block diagram showing a third functional configuration example of the data expansion device. [Diagram 5] FIG. 5 is a block diagram showing a fourth functional configuration example of the data expansion device. [Figure 6] FIG. 6 is a flowchart of an operation example of the data extension device according to the first embodiment. [Figure 7] FIG. 7 is a block diagram illustrating an example of a hardware configuration of the data expansion device. [Figure 8] FIG. 8 is a block diagram showing a fifth functional configuration example of the data expansion device. [Figure 9] FIG. 9 is a flowchart of an operation example of the data extension device according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing a sixth functional configuration example of the data expansion device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0011] <Functional Configuration Example 1 of Data Extension Device 100> 1 is a block diagram showing a functional configuration example 1 of a data expansion device. The data expansion device 100 holds a first training data set 101. The first training data set 101 includes a plurality of first training samples 102 acquired in a first domain, and a first correct answer value 105 for the first training samples 102. Here, the first domain indicates, for example, a specific place or surrounding environment where the first training samples 102 can be acquired, and a data distribution of the first training samples 102 depending on a setting value of a sample acquisition device.

[0012] When the sample acquisition device is an AI (Artificial Intelligence) surveillance camera, the specific location is, for example, the location where the AI ​​surveillance camera is installed. The AI ​​surveillance camera analyzes the captured image data to predict the subject, the subject's behavior, and the events occurring around the subject.

[0013] The surrounding environment is, for example, buildings, roads, and installations around the installation location of the AI ​​surveillance camera. The setting values ​​of the sample acquisition device are, for example, the shooting conditions of the AI ​​surveillance camera, such as the shooting angle, field of view, and frame rate. The first learning sample 102 is, for example, image data captured by the AI ​​surveillance camera. The first correct answer value 105 for the first learning sample 102 is, for example, identification information that specifies the name of an object captured in the image data if the first learning sample 102 is image data.

[0014] The reference model 103 is a machine learning model trained by the first training sample 102 and the first correct answer value 105. For example, the reference model 103 is a model that predicts the number of objects appearing in input image data. In the training process of the reference model 103, the data expansion device 100 inputs the first training sample 102 to the reference model 103 and obtains the first output value 104.

[0015] The first output value 104 is a predicted value output from the reference model 103, for example, a predicted value of the number of objects appearing in the first learning sample 102. The data extension device 100 updates the reference model 103 based on a first loss function 106. The first loss function 106 calculates a loss indicating a deviation between the first output value 104 and a first ground truth value 105 associated with the first learning sample 102 as a first loss value. The data extension device 100 updates the reference model 103 so that the first loss value of the first loss function 106 becomes smaller. As the first loss function 106, for example, a squared error (mean squared error or square root of mean squared error) or a cross entropy error is used.

[0016] <Functional Configuration Example 2 of Data Extension Device 100> 2 is a block diagram showing a functional configuration example 2 of the data extension device 100. The data extension device 100 extends the first training data set 101 by using a second training data set 201. The second training data set 201 includes a plurality of second training samples 202 acquired in a second domain and a second correct answer value 205 for the second training samples 202. Here, the second domain is a domain different from the first domain, and indicates, for example, a specific location or surrounding environment where the second training samples 202 can be acquired, or a setting value of a sample acquisition device.

[0017] For example, in the above example, if the first training sample 102 is image data captured by an AI surveillance camera, the second training sample 202 is also image data captured at a location different from the AI ​​surveillance camera that captured the first training sample 102. If the second training sample 202 is image data, the second correct answer value 205 for the second training sample 202 is, for example, the number of objects captured in the image data.

[0018] In the first embodiment, the data extension device 100 converts the second training data set 201 into a data set of the first domain with high accuracy. This enables data extension in which data including the features of the first domain is newly generated from the second training data set 201. Therefore, the cost of collecting the first training samples 102 at the site of the first domain and the effort of identifying the first correct answer value 105 are reduced.

[0019] The domain inverse converter 203 is a learning model that converts data related to the second domain into data that is pseudo to data related to the first domain. Specifically, for example, the domain inverse converter 203 converts features specific to the second domain included in the second training sample 202 into features of the first domain, and outputs pseudo first training sample 204, which is pseudo data of the first training sample 102, as a conversion result.

[0020] The pseudo first training sample 204 is data that simulates the first training sample 102 included in the first training dataset 101 based on the second training sample 202. For example, if the first training sample 102 is a lush summer landscape and the second training sample 202 is a snowy winter landscape, an image in which the snow is removed while maintaining the object shapes reflected in the second training sample 202 and new leaves are generated is generated as the pseudo first training sample 204. In this way, the pseudo first training sample 204 is not data that is a copy of the first training sample 102, but is data that is similar in geometric composition to the first training sample 102.

[0021] The domain forward converter 206 is a learning model that converts data related to the first domain into data that is pseudo to data related to the second domain. Specifically, for example, the domain forward converter 206 converts features specific to the first domain included in the pseudo first training sample 204 into features of the second domain, and outputs pseudo second training sample 207, which is pseudo data of the second training sample 202, as the conversion result.

[0022] The pseudo second training sample 207 is data that simulates the second training sample 202 included in the second training dataset 201 based on the pseudo first training sample 204. For example, if the second training sample 202 is a lush summer landscape and the pseudo first training sample 204 is a snowy winter landscape, an image in which the snow is removed while maintaining the object shapes reflected in the pseudo first training sample 204 and new leaves are generated is generated as the pseudo second training sample 207. In this way, the pseudo second training sample 207 is not data that is a copy of the second training sample 202, but is data that is similar in geometric composition to the second training sample 202.

[0023] The data extension device 100 executes a learning process by the domain inverse converter 203 and a learning process by the domain forward converter 206 in parallel. The data extension device 100 uses two types of loss functions in the learning process by the domain inverse converter 203 and the learning process by the domain forward converter 206. One is a first loss function 106 and the other is a second loss function 210.

[0024] 2, the first loss function 106 is a loss function that calculates, as a pseudo first loss value, a loss indicating a deviation between the pseudo first output value 208 and a ground truth value associated with the pseudo first training sample 204. The pseudo first output value 208 is an output value output from the reference model 103 as a result of inputting the pseudo first training sample 204 to the reference model 103. The ground truth value associated with the pseudo first training sample 204 may be the second ground truth value 205 associated with the second training sample 202 input to the domain inverse converter 203.

[0025] For example, when a rectangular area surrounding an object in an image is given as a correct value, the same correct value can be used if the object position, size, etc. shown in the second training sample 202 and the pseudo first training sample 204 do not change. In FIG. 2, the correct value associated with the pseudo first training sample 204 is set as the second correct value 205 as an example.

[0026] The second loss function 210 is a loss function that calculates, as a second loss value, a loss indicating a deviation between the second training sample 202 and the pseudo second training sample 207. The pseudo second training sample 207 is output from the domain forward converter 206 by further inputting the pseudo first training sample 204, which is output as a result of inputting the second training sample 202 to the domain inverse converter 203, into the domain forward converter 206. The second loss function 210 is a function for restoring the original data.

[0027] The data extension device 100 performs learning on each of the domain inverse converter 203 and the domain forward converter 206 based on the first loss function 106 and the second loss function 210. For example, the data extension device 100 adds the pseudo first loss value from the first loss function 106 and the second loss value from the second loss function 210 to obtain an integrated loss value. The data extension device 100 updates each of the domain inverse converter 203 and the domain forward converter 206 so as to reduce the integrated loss value.

[0028] In this way, by updating the domain inverse converter 203 based on the first loss function 106, the domain inverse converter 203 can learn conversion to pseudo second training samples 207 that are easy for the reference model 103 to predict. Since the reference model 103 has been trained in advance with the first training dataset 101, it is expected to make highly accurate predictions for samples including features of the first domain.

[0029] Therefore, the domain inverse converter 203 can convert the features of the second domain contained in the second training sample 202 into features of the first domain. Furthermore, by updating the domain inverse converter 203 and the domain forward converter 206 based on the second loss function 210, it is possible to learn to reflect specific features contained in the input samples to each of the domain inverse converter 203 and the domain forward converter 206 in the output.

[0030] <Learning Process by the Domain Inverse Converter 203 and the Domain Forward Converter 206> FIG. 3 is a flowchart showing an example of a learning process procedure performed by the domain reverse converter 203 and the domain forward converter 206.

[0031] Step S301: The data expansion device 100 trains the reference model 103 using the first training data set 101 (see FIG. 1).

[0032] Step S302: The data expansion device 100 reads the second training sample 202 from the second training data set 201 (see FIG. 2).

[0033] Step S303: The data extension device 100 inputs the second training samples 202 to the domain inverse transformer 203 to perform inverse transform, and outputs pseudo first training samples 204 (see FIG. 2).

[0034] Step S304: The data expansion device 100 inputs the pseudo first training sample 204 obtained in step S303 to the reference model 103, and outputs a predicted value for the pseudo first training sample 204 as the pseudo first output value 208 (see FIG. 2).

[0035] Step S305: The data expansion device 100 inputs the pseudo first training samples 204 to the domain forward converter 206, performs forward conversion, and outputs the pseudo second training samples 207 (see FIG. 2).

[0036] Step S306: The data expansion device 100 inputs the pseudo first output value 208 and the second correct answer value 205 acquired in step S304 to the first loss function 106 to calculate a pseudo first loss value (see FIG. 2).

[0037] Step S307: The data extension device 100 inputs the second training sample 202 read in step S302 and the pseudo second training sample 207 acquired in step S305 to the second loss function 210 to calculate a second loss value (see FIG. 2).

[0038] Step S308: The data extension device 100 updates the domain inverse converter 203 and the domain forward converter 206 based on the loss values ​​acquired in steps S306 and S307 (see FIG. 2).

[0039] Step S309: The data extension device 100 judges whether or not steps S302 to S308 have been repeated a predetermined number of times. If steps S302 to S308 have not been repeated a predetermined number of times (step S309: No), the process returns to step S302. If steps S309: Yes, the data extension device 100 ends the learning process by the domain reverse converter 203 and the domain forward converter 206.

[0040] <Functional Configuration Example 3 of Data Extension Device 100> FIG. 4 is a block diagram showing a functional configuration example 3 of the data extension device 100. The data extension device 100 executes data extension using the domain inverse converter 203. Specifically, for example, the data extension device 100 outputs the pseudo first learning sample 204 every time the second learning sample 202 is input to the domain inverse converter 203. A set of combinations of the output pseudo first learning sample 204 and the second correct answer value 205 corresponding to the second learning sample 202 becomes the third learning dataset 401. The third learning dataset 401 may include the first learning dataset 101. Since the pseudo first learning sample 204 includes the features of the first domain, data extension of the first learning dataset 101 is achieved. Note that the data extension device 100 may add the first learning dataset 101 to the third learning dataset 401.

[0041] <Functional Configuration Example 4 of Data Extension Device 100> 5 is a block diagram showing a fourth functional configuration example of the data extension device 100. The data extension device 100 extracts third training samples 501 from the third training data set 401, and outputs the samples to a target model 502. The third training samples 501 are samples stored in the third training data set 401, and include, for example, the pseudo first training samples 204 and the first training samples 102.

[0042] The target model 502 is a machine learning model having an arbitrary structure, and may have the same structure as the reference model 103. Since the domain-specific features included in the third learning sample 501 are converted into the features of the first domain based on the reference model 103, by making the reference model 103 and the target model 502 have the same structure, the accuracy of the target model 502 can be expected to be improved.

[0043] When the third learning sample 501 is input, the target model 502 outputs a second output value 503. The data expansion device 100 updates the target model 502 based on a third loss function 505. The third loss function 505 calculates a loss indicating a deviation between the second output value 503 and a third ground truth value 504 associated with the third learning sample 501 as a third loss value.

[0044] The third superscript 504 is the second superscript 205 if the third training sample 501 is the pseudo first training sample 204, and is the first superscript 105 if the third training sample 501 is the first training sample 102. The target model 502 is updated so that the third loss value of the third loss function 505 becomes smaller.

[0045] In the first embodiment, the target model 502 thus trained is implemented in a sample acquisition device of the first domain.

[0046] <Example of operation of the data extension device 100> FIG. 6 is a flowchart illustrating an example of operation of the data extension device 100 according to the first embodiment.

[0047] Step S301: The data expansion device 100 trains the reference model 103 using the first training data set 101 (see FIG. 1).

[0048] Step S602: The data extension device 100 trains the domain forward converter 206 and the domain inverse converter 203 by using the reference model 103 and the second training data set 201. Specifically, for example, the data extension device 100 executes steps S302 to S309 in FIG.

[0049] Step S603: The data expansion device 100 performs conversion of the second training samples 202 using the domain inverse converter 203 trained in step S602, and outputs pseudo first training samples 204 (see FIG. 4).

[0050] Step S604: The data expansion device 100 creates the third training data set 401 including the pseudo first training sample 204.

[0051] Step S605: The data expansion device 100 trains the target model 502 using the third training data set 401 (see FIG. 5).

[0052] <Hardware Configuration Example of Data Extension Device 100> FIG. 7 is a block diagram showing an example of a hardware configuration of the data expansion device 100. The data expansion device 100 includes a processor 701, a storage device 702, an input device 703, an output device 704, and a communication interface (communication IF) 705. The processor 701, the storage device 702, the input device 703, the output device 704, and the communication IF 705 are connected by a bus 706. The processor 701 controls the data expansion device 100. The storage device 702 serves as a working area for the processor 701. The storage device 702 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 702 include a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and a flash memory. The input device 703 inputs data. Examples of the input device 703 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 704 outputs data. The output device 704 may be, for example, a display, a printer, or a speaker. The communication IF 705 connects to a network and transmits and receives data.

[0053] Although a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) is conceivable as an example of the processor 701, other semiconductor devices may be used as long as they are capable of executing predetermined processes.

[0054] In this way, according to the first embodiment, it is possible to generate synthetic data that contributes to improving the accuracy of a machine learning model. For example, since the domain inverse converter 203 can learn a transformation limited to features that affect the prediction of the reference model 103, it is possible to suppress the divergence between real-world data and synthetic data. Therefore, data expansion of the first training dataset 101 becomes possible. EXAMPLES

[0055] Example 2 is an example corresponding to a case where the second training data set 201 in Example 1 is required to be expanded. For example, even if the number of second training samples 202 included in the second training data set 201 is very small, the second training data set 201 can be data expanded to ensure the amount of samples required for training the reference model 103. In Example 2, differences from Example 1 will be described, and therefore commonalities with Example 1 will not be described.

[0056] <Functional Configuration Example 5 of Data Extension Device 100> FIG. 8 is a block diagram showing a functional configuration example 5 of the data extension device 100. The data extension device 100 executes data extension using the domain order converter 206. Specifically, for example, the data extension device 100 outputs a pseudo second learning sample 207 every time the first learning sample 102 is input to the domain order converter 206. In the second embodiment, a set of the output pseudo second learning samples 207 becomes a third learning data set 401. Since the pseudo second learning sample 207 includes the features of the second domain, data extension of the second learning data set 201 is achieved. Note that the third learning data set 401 may include the second learning data set 201.

[0057] <Example of operation of the data extension device 100> 9 is a flowchart illustrating an example of operation of the data extension device 100 according to the second embodiment. Differences from FIG. 6 will be described.

[0058] Step S903: The data expansion device 100 performs transformation of the first training samples 102 using the domain forward converter 206 trained in step S602, and outputs pseudo second training samples 207 (see FIG. 6).

[0059] Step S904: The data extension device 100 creates a third training dataset 401 including the pseudo second training sample 207. The third training dataset 401 is a combination of the pseudo second training sample 207 and the first correct answer value 105. Note that the data extension device 100 may add the second training dataset 201 to the third training dataset 401.

[0060] By operating the data extension device 100 in this manner, the pseudo second training samples 207 constituting the third training dataset 401 include features of the second domain, and thus data extension of the second training dataset 201 is achieved.

[0061] In the second embodiment, the target model 502 is updated as shown in Fig. 5. In the second embodiment, the third training sample 501 of the third training dataset 401 is the pseudo second training sample 207, and the third correct answer 504 is the first correct answer 105. In the second embodiment, the target model 502 thus trained is implemented in a sample acquisition device of the second domain.

[0062] As a result, the data extension device 100 converts the first training data set 101, which contains a large number of first training samples 102, into a data set for the second domain with high accuracy, even in the field of the second domain, where the number of second training samples 202 is small. This enables data extension in which the first training data set 101 converted to the second domain is applied to an AI surveillance camera at the field of the second domain. Therefore, the cost of collecting the second training samples 202 at the field of the second domain and the effort of identifying the second correct answer value 205 are reduced. EXAMPLES

[0063] Example 3 is an example in which the quality of the pseudo second training samples 207 which are the output of the domain forward converter 206 in Example 2 is improved, and the deviation from the second training samples 202 collected in the real world is suppressed. Example 3 will describe the differences from Examples 1 and 2, and therefore descriptions of the commonalities with Examples 1 and 2 will be omitted.

[0064] <Functional Configuration Example 6 of Data Extension Device 100> 10 is a block diagram showing a functional configuration example 6 of the data extension device 100. In the data extension device 100, the domain forward converter 206 converts the first training samples 102 into pseudo second training samples 207. The domain inverse converter 203 converts the pseudo second training samples 207 output by the domain forward converter 206 into pseudo first training samples 204.

[0065] In the third embodiment, the second loss function 210 calculates, as a fourth loss value, a loss indicating a deviation between the first training sample 102 and the pseudo first training sample 204. The pseudo first training sample 204 is output from the domain inverse converter 203 by inputting, to the domain inverse converter 203, pseudo second training sample 207, which is output as a result of inputting the first training sample 102 to a domain forward converter 206.

[0066] The data extension device 100 alternately executes a learning cycle of the domain inverse converter 203 and the domain forward converter 206 using the pseudo first loss value and second loss value shown in FIG. 2 and a learning cycle of the domain inverse converter 203 and the domain forward converter 206 using the fourth loss value shown in FIG. 10.

[0067] In addition, the data extension device 100 may simultaneously execute a learning cycle of the domain inverse converter 203 and the domain forward converter 206 using the pseudo first loss value and second loss value shown in FIG. 2 and a learning cycle of the domain inverse converter 203 and the domain forward converter 206 using the fourth loss value shown in FIG. 10.

[0068] Specifically, for example, the data extension device 100 adds the pseudo first loss value, the second loss value, and the fourth loss value to obtain an aggregate loss value. Then, the data extension device 100 updates each of the domain inverse converter 203 and the domain forward converter 206 so that the aggregate loss value decreases. In this way, the data extension device 100 can realize the update of each of the domain inverse converter 203 and the domain forward converter 206 in one cycle.

[0069] According to the third embodiment, the domain forward converter 206 learns transformation for the first training samples 102 collected in the real world, thereby improving the quality of the pseudo second training samples 207 which are the output of the domain forward converter 206 in the second embodiment, and suppressing deviation from the second training samples 202 collected in the real world.

[0070] As described above, according to the data extension device 100 of the first to third embodiments, it is possible to generate synthetic data that simulates data actually acquired in the real world with high accuracy and contributes to improving the accuracy of a machine learning model. In other words, according to the data extension device 100 of the first to third embodiments, it is possible to suppress the deviation between data before and after conversion, and it is possible to realize data extension with suppressed image distortion.

[0071] Specifically, the data expansion device 100 can generate synthetic data that contributes to improving the accuracy of a machine learning model. For example, the domain inverse converter 203 and the domain forward converter 206 can learn a transformation limited to features that affect the prediction of the reference model 103, so that it is possible to suppress the deviation between real-world data and synthetic data (for example, the deviation between the first training sample 102 and the pseudo first training sample 204, and the deviation between the second training sample 202 and the pseudo second training sample 207).

[0072] That is, data deviation (deterioration in accuracy) due to differences in domains is said to be caused by differences in data distribution during learning and inference. Therefore, the conversion process (FIG. 3) that improves the inference accuracy for the reference model 103 trained in the first domain means a conversion process that brings the pseudo first training sample 204 closer to the first training sample 102, and therefore can realize conversion to the first domain (second training sample 202 ⇒ pseudo first training sample 204).

[0073] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0074] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0075] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0076] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0077] 100 Data expansion device 101 First training data set 102 First learning sample 103 Reference Model 201 Second training dataset 202 Second Learning Sample 203 Domain Inverse Converter 204 Pseudo 1st Training Sample 206 Domain Order Converter 207 Pseudo Second Learning Sample 401 Third training dataset 501 3rd Learning Sample 502 Target Model

Claims

1. A data extension device having a processor that executes a program and a storage device that stores the program, the storage device stores a first learning data set in a first domain which is a set of combinations of a first learning sample and a first correct answer value indicating a correct answer for the first learning sample, a second learning data set in a second domain which is a set of combinations of a second learning sample and a second correct answer value indicating a correct answer for the second learning sample, a reference model, a domain forward converter, and a domain inverse converter; the reference model is a learning model that is trained so as to reduce a first loss value based on a first output value output as a result of inputting the first learning sample into the reference model and the first correct answer value, the domain forward converter is a learning model that converts data about the first domain into data that resembles data about the second domain; the domain inverse transformer is a learning model that transforms data relating to the second domain into data that resembles data relating to the first domain; The processor, a first conversion process in which the second training samples are converted by the domain inverse converter into pseudo first training samples that are pseudo to the first training samples, and the pseudo first training samples are converted by the domain forward converter into pseudo second training samples that are pseudo to the second training samples; a first calculation process for calculating a pseudo first loss value that is a loss value based on a pseudo first output value that is an output value obtained by inputting the pseudo first learning sample output by the first conversion process to the reference model and the second correct answer value; a second calculation process for calculating a second loss value based on the second training samples and the pseudo second training samples; a first learning process for learning the domain inverse converter and the domain forward converter so that an integrated loss value obtained by integrating the pseudo first loss value calculated by the first calculation process and the second loss value calculated by the second calculation process becomes small; A data expansion device comprising:

2. 2. The data expansion device according to claim 1, The processor, a creation process of creating a third learning data set by associating the pseudo first learning samples output as a result of inputting the second learning samples to the domain inverse converter with the second correct answer value as a third learning sample and the second correct answer value as a third correct answer value; A data expansion device comprising:

3. 3. A data expansion device according to claim 2, In the creation process, the processor adds the first training data set to the third training data set. A data expansion device comprising:

4. 3. A data expansion device according to claim 2, The storage device stores a target model trained using the third training data set; The processor, a second learning process for learning the target model so as to reduce a third loss value based on a second output value output as a result of inputting the third learning sample into the target model and the third correct answer value; A data expansion device comprising:

5. 2. The data expansion device according to claim 1, The processor, a creation process of creating a third learning data set by associating the pseudo second learning samples output as a result of inputting the first learning samples to the domain forward converter with the first correct answer value as a third correct answer value; A data expansion device comprising:

6. 6. A data expansion device according to claim 5, In the creation process, the processor adds the second training data set to the third training data set. A data expansion device comprising:

7. 6. A data expansion device according to claim 5, The storage device stores a target model trained using the third training data set; The processor, a second learning process for learning the target model so as to reduce a third loss value based on a second output value output as a result of inputting the third learning sample into the target model and the third correct answer value; A data expansion device comprising:

8. 2. The data expansion device according to claim 1, The processor, a third calculation process for calculating a fourth loss value based on the first training samples and pseudo first training samples converted from the pseudo second training samples by the domain inverse converter; a third learning process for learning the domain inverse converter and the domain forward converter so as to reduce the fourth loss value calculated by the third calculation process; A data expansion device comprising:

9. 2. The data expansion device according to claim 1, The processor, performing a third calculation process of calculating a fourth loss value based on the first training samples and pseudo first training samples converted from the pseudo second training samples by the domain inverse converter; In the first learning process, the processor learns the domain inverse converter and the domain forward converter so that an integrated loss value obtained by integrating the second loss value, the pseudo first loss value, and the fourth loss value calculated by the third calculation process becomes small. A data expansion device comprising:

10. A data extension method executed by a data extension device having a processor that executes a program and a storage device that stores the program, the storage device stores a first learning data set in a first domain which is a set of combinations of a first learning sample and a first correct answer value indicating a correct answer for the first learning sample, a second learning data set in a second domain which is a set of combinations of a second learning sample and a second correct answer value indicating a correct answer for the second learning sample, a reference model, a domain forward converter, and a domain inverse converter; the reference model is a learning model that is trained so as to reduce a first loss value based on a first output value output as a result of inputting the first learning sample into the reference model and the first correct answer value, the domain forward converter is a learning model that converts data about the first domain into data that resembles data about the second domain; the domain inverse transformer is a learning model that transforms data relating to the second domain into data that resembles data relating to the first domain; The processor, a first conversion process in which the second training samples are converted by the domain inverse converter into pseudo first training samples that are pseudo to the first training samples, and the pseudo first training samples are converted by the domain forward converter into pseudo second training samples that are pseudo to the second training samples; a first calculation process for calculating a pseudo first loss value that is a loss value based on a pseudo first output value that is an output value obtained by inputting the pseudo first learning sample output by the first conversion process to the reference model and the second correct answer value; a second calculation process for calculating a second loss value based on the second training samples and the pseudo second training samples; a first learning process for learning the domain inverse converter and the domain forward converter so that an integrated loss value obtained by integrating the pseudo first loss value calculated by the first calculation process and the second loss value calculated by the second calculation process becomes small; A data augmentation method comprising:

11. A data expansion program for causing a processor to execute data expansion, the processor is capable of accessing a storage device; the storage device stores a first learning data set in a first domain which is a set of combinations of a first learning sample and a first correct answer value indicating a correct answer for the first learning sample, a second learning data set in a second domain which is a set of combinations of a second learning sample and a second correct answer value indicating a correct answer for the second learning sample, a reference model, a domain forward converter, and a domain inverse converter; the reference model is a learning model that is trained so as to reduce a first loss value based on a first output value output as a result of inputting the first learning sample into the reference model and the first correct answer value, the domain forward converter is a learning model that converts data about the first domain into data that resembles data about the second domain; the domain inverse transformer is a learning model that transforms data relating to the second domain into data that resembles data relating to the first domain; The processor, a first conversion process in which the second training samples are converted by the domain inverse converter into pseudo first training samples that are pseudo to the first training samples, and the pseudo first training samples are converted by the domain forward converter into pseudo second training samples that are pseudo to the second training samples; a first calculation process for calculating a pseudo first loss value that is a loss value based on a pseudo first output value that is an output value obtained by inputting the pseudo first learning sample output by the first conversion process to the reference model and the second correct answer value; a second calculation process for calculating a second loss value based on the second training samples and the pseudo second training samples; a first learning process for learning the domain inverse converter and the domain forward converter so that an integrated loss value obtained by integrating the pseudo first loss value calculated by the first calculation process and the second loss value calculated by the second calculation process becomes small; A data expansion program that causes the program to execute the above steps.

Images