Model generation device, classification device, data generation device, model generation method, and model generation program
The model generation device addresses inefficiencies in generating training samples by using encoders and decoders to create comprehensive and reliable datasets, reducing costs and improving classification model performance.
Patent Information
- Application Number
- JP2021041133
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-15
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-03-15
AI Technical Summary
Conventional methods for generating training samples are inefficient in creating comprehensive and controlled datasets for machine learning models, particularly in visual inspection and classification tasks, leading to high costs and suboptimal model performance.
A model generation device that includes a data acquisition section, first and second training processing sections, and a sample generation section to utilize encoders and decoders for converting and generating new training samples, ensuring high coverage and reliability of feature spaces through linear sum calculations and adversarial learning.
Reduces the cost of collecting training samples and improves the performance of trained classification models by generating diverse and reliable datasets, enhancing the accuracy of classification tasks.
Smart Images

Figure 0007593177000015 
Figure 0007593177000016 
Figure 0007593177000017
Abstract
Description
[Technical field]
[0001] The present invention relates to a model generation device, a classification device, a data generation device, a model generation method, and a model generation program. [Background technology]
[0002] Conventionally, in a scene where a product is manufactured, such as on a production line, a technology has been developed to inspect the quality of the product by photographing the product to be manufactured with an imaging device and analyzing the obtained image data. For example, Patent Document 1 proposes an inspection device that uses a trained first neural network to determine whether an object shown in an image is normal or abnormal, and if it is determined to be abnormal, uses a trained second neural network to classify the type of abnormality.
[0003] According to the method using a machine learning model such as a neural network, it is possible to perform visual inspection of a product based on the calculation results of a trained machine learning model without manually specifying in detail the image processing for the obtained image. Therefore, it is possible to simplify the information processing of visual inspection and reduce the effort required to create an inspection program. On the other hand, in order to generate a trained machine learning model (i.e., to perform machine learning), it is required to prepare training samples according to the ability to be acquired by the machine learning model (in the above example, the ability to inspect the appearance of a product). If it is possible to prepare a sufficient number of training samples that comprehensively cover all events that may occur at the site where the trained machine learning model is applied, it is expected that a trained machine learning model capable of performing an inference task with high accuracy will be generated. However, it is costly to prepare such comprehensive and sufficient number of training samples.
[0004] In recent years, therefore, techniques have been studied for increasing the number of training samples by generating new training samples using training samples obtained by various methods such as data augmentation and the use of generative models. For example, Non-Patent Document 1 and Patent Document 2 propose a method for generating training samples of a target class using a trained conditional generative model (AC-GAN in Non-Patent Document 1). According to these methods, it is possible to generate new training samples in which events different from those in the original training samples appear, thereby reducing the cost of preparing a sufficient number of training samples. In addition, by using a sufficient number of obtained training samples for machine learning, it is expected that the performance of the trained machine learning model will improve. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2012-026982 A [Patent Document 2] JP 2019-074945 A [Non-patent literature]
[0006] [Non-Patent Document 1] Quan Kong, Bin Tong, Martin Klinkigt, Yuki Watanabe, Naoto Akira, Tomokazu Murakami, "Active Generative Adversarial Network for Image Classification", arXiv:1906.07133v1 [cs.LG], 17 Jun 2019. Summary of the Invention [Problem to be solved by the invention]
[0007] The present inventors have found that the above-mentioned conventional method for generating training samples has the following problems. That is, in the data augmentation method, new training images (training samples) are generated by applying image processing such as translation, enlargement / reduction, rotation, and noise addition to the original training image. Therefore, although it is possible to generate training samples in which events near the original training sample appear, it is difficult to generate training samples in which other events (e.g., intermediate events) appear. Therefore, it is difficult to obtain comprehensive training samples with the data augmentation method. Moreover, in the above-mentioned generative model method, new training samples are basically generated based on random numbers, so it is difficult to control the generated training samples. Therefore, it is difficult to efficiently obtain comprehensive training samples.
[0008] Note that this problem is not specific to the generation of a trained model that can be used for visual inspection. In addition, this problem is not specific to the case where image data is used as a training sample. For example, a similar problem may occur in any situation where a training sample is collected to generate a trained classification model that has acquired the ability to perform a classification task of classifying features contained in data, such as classifying the type of object shown in an image or classifying the state of an object shown in sensing data. The training sample may be composed of image data, as well as, for example, sound data, numerical data, text data, and other measurement data obtained by a sensor.
[0009] In one aspect, the present invention has been made in consideration of the above circumstances, and its purpose is to reduce the cost of comprehensively collecting training samples, or to improve the performance of a trained classification model by using the training samples obtained thereby for machine learning. [Means for solving the problem]
[0010] In order to solve the above-mentioned problems, the present invention employs the following configuration.
[0011] That is, a model generating device according to one aspect of the present invention includes a data acquisition unit, a first training processing unit, a conversion unit, a point designation unit, a sample generation unit, a labeling unit, and a second training processing unit. The data acquisition unit is configured to acquire a plurality of first data sets each composed of a combination of a training sample and a correct answer label. The training sample includes a feature, and the correct answer label is configured to indicate a class to which the feature included in the training sample belongs. The first training processing unit is configured to perform machine learning of an encoder and a decoder using a plurality of training samples included in the acquired plurality of first data sets. The encoder is configured to convert a sample into a feature. The decoder is configured to decode the sample from the feature. The machine learning of the encoder and the decoder includes converting each of the training samples into a feature by the encoder, and training the encoder and the decoder so that a sample decoded by the decoder from each of the obtained feature matches each of the training samples. The conversion unit is configured to acquire two or more sample points in a feature space corresponding to each of the training samples by converting each of two or more training samples of the plurality of training samples into a feature using the trained encoder. The point designation unit is configured to derive a linear sum of the two or more sample points acquired in the feature space, the linear sum being the maximum sum of distances calculated from the sample points according to a predetermined index. The sample generation unit is configured to generate a decoded sample corresponding to a feature of the linear sum as a new training sample by using the trained decoder. The labeling unit is configured to generate a second dataset by assigning a new correct answer label to the new training sample, the correct answer label being configured to indicate a class to which a feature included in the generated new training sample belongs. The second training processing unit is configured to perform machine learning of a classification model using the multiple first datasets and the generated one or more second datasets.The machine learning of the classification model includes training the classification model for each of the data sets so that the result of classifying the class to which the feature contained in the training sample belongs using the classification model matches the correct label.
[0012] In this configuration, machine learning of the encoder and the decoder is performed using a plurality of original data sets (first data sets). As a result of the machine learning, a trained encoder and a decoder can be generated. The trained encoder can map each training sample to a feature space (latent space) while maintaining information about the included features. The trained decoder can decode a sample from a value (feature) in the feature space. In this configuration, the trained encoder is used to convert each of two or more training samples among the plurality of training samples into a feature, thereby obtaining two or more sample points corresponding to each training sample in the feature space. A linear sum that maximizes the sum of distances calculated from each sample point according to a predetermined criterion is derived. Then, a decoded sample corresponding to the feature of the linear sum is generated as a new training sample by using the trained decoder. That is, according to this configuration, a state in which sample points are evenly present in the feature space can be formed by obtaining a linear sum point that maximizes the sum of distances from each sample point as a new sample point in the feature space. Therefore, by using a trained decoder to add a new data set (second data set) including samples decoded from the features of the linear sum to the training data, it is possible to obtain training data that can be evaluated in a feature space with high comprehensiveness. In addition, in this configuration, by using a trained encoder and decoder, it is possible to automate at least a part of the process of generating such new training samples (datasets including them). Therefore, according to this configuration, it is possible to reduce the cost of comprehensively collecting training samples. In addition, in this configuration, in addition to the original data set (first data set), the new data set (second data set) generated as described above is used for machine learning of the classification model. As a result, according to this configuration, it is possible to improve the performance of the generated trained classification model by using comprehensively obtained training samples for machine learning.
[0013] In the model generating device according to the above aspect, the classification model may be configured to output a posterior probability that a feature included in the sample belongs to each class. The machine learning of the classification model may be performed by dividing it into a first machine learning using the plurality of first data sets and a second machine learning using the one or more second data sets. A provisional trained classification model may be generated by performing the first machine learning. The point designation unit may be further configured to use the provisional trained classification model to calculate a posterior probability that a feature included in a sample decoded by the trained decoder belongs to each class from the derived feature of the linear sum, calculate an entropy of the sample to be decoded from the calculated posterior probability for each class, and correct the derived linear sum so that the entropy increases based on a gradient of the calculated entropy. The sample generating unit may be configured to generate a decoded sample corresponding to the corrected feature of the linear sum as the new training sample by using the trained decoder.
[0014] In this configuration, the entropy of a sample decoded from the features of the linear sum is evaluated by a provisional trained classification model. The entropy indicates the degree to which the features contained in the decoded sample can be classified (in other words, the uncertainty). The higher the uncertainty, the more difficult it is to classify the features contained in the sample, that is, the more insufficient the machine learning of the sample is. In this configuration, the derived linear sum is corrected so that the entropy increases, and a trained decoder is used to generate a sample decoded from the features of the corrected linear sum as a new training sample. This makes it possible to obtain a training sample with insufficient machine learning in the provisional trained classification model, and by using the obtained training sample for machine learning of the classification model, it is possible to increase the expectation of improving the performance of the generated trained classification model.
[0015] In the model generating device according to the above aspect, the first training processing unit may be configured to further perform machine learning of a classifier using a plurality of training samples included in the plurality of first data sets. The machine learning of the classifier may be configured by training the classifier to evaluate the degree to which an input sample belongs to a distribution of the plurality of training samples. The point designation unit may be further configured to use the trained classifier to evaluate the degree to which a sample decoded by the trained decoder belongs to the distribution of the plurality of training samples from the derived feature of the linear sum, calculate the reliability of the sample decoded by the trained decoder based on the result of the evaluation of the degree, and correct the derived linear sum so that the reliability is improved based on a gradient of the calculated reliability. The sample generation unit may be configured to generate a decoded sample corresponding to the corrected feature of the linear sum as the new training sample by using the trained decoder.
[0016] In the feature space, a sample decoded by a trained decoder from the features of points other than the points corresponding to the training sample is, so to speak, pseudo data that imitates true data (training sample). In some cases, this pseudo data may be extremely different from the true data. Therefore, in this configuration, a trained classifier is used to evaluate the reliability of the sample decoded by the trained decoder. In one example, the reliability of the sample is evaluated to be higher the higher the degree to which the decoded sample belongs to the distribution of the training samples, and the reliability of the sample is evaluated to be lower the lower the degree to which the decoded sample belongs to the distribution of the training samples. Then, based on the result of the evaluation, the linear sum is corrected so as to improve the reliability, and a trained decoder is used to generate a sample decoded from the features of the corrected linear sum as a new training sample. This makes it possible to suppress the addition of training samples that are extremely different from the true data to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, it is possible to increase the expectation of improving the performance of the generated trained classification model.
[0017] In the model generating device according to the above aspect, training the classifier may be configured by performing adversarial learning between the encoder and the decoder and the classifier. The adversarial learning may include a step of training the classifier to identify the origin of a plurality of samples decoded by the decoder and the plurality of training samples as input samples, respectively, and a step of training the encoder and the decoder such that the discrimination performance of the classifier decreases when a sample decoded by the decoder is input to the classifier. Evaluating the degree to which the decoded sample belongs to a distribution of the plurality of training samples may be configured by identifying the origin of the decoded sample using the trained classifier.
[0018] A classifier trained by adversarial learning acquires the ability to distinguish between a training sample and a sample (pseudo sample) derived from a decoder, depending on the degree of adversarial learning. Thus, with this trained classifier, the reliability of an input sample can be evaluated according to the result of identifying the origin of the input sample. That is, the more the classifier erroneously identifies the input sample as being derived from a training sample, the higher the reliability is, and the more the classifier correctly identifies the input sample as being derived from a decoder, the lower the reliability is. Therefore, with this configuration, the trained classifier generated by adversarial learning can appropriately evaluate the reliability of a sample to be decoded. This makes it possible to suppress the addition of training samples that are extremely different from the true data to the learning data, and to increase the expectation of improved performance of the generated trained classification model.
[0019] In the model generating device according to the above aspect, the point designation unit may be further configured to evaluate a degree of difference between a sample decoded by the trained decoder and a training sample included in the plurality of first data sets from the derived feature of the linear sum, calculate a reliability of the sample decoded by the trained decoder based on a result of the degree evaluation, and correct the derived linear sum so as to improve the reliability based on a gradient of the calculated reliability. The evaluation of the degree of difference may be configured to convert the decoded sample into a feature by the trained encoder and calculate a difference between the obtained feature of the decoded sample and the feature of the linear sum, or to generate a re-decoded sample from the feature of the decoded sample using the trained decoder, and calculate a reconstruction error between the decoded sample and the generated re-decoded sample.
[0020] The reliability of the decoded sample can also be evaluated by the difference between the decoded sample and the training sample on the feature space or the original space (the dimensional space of the sample). In this configuration, as the difference between the decoded sample and the training sample, the sample decoded by the trained decoder from the derived linear sum is converted into a feature by the trained encoder, and the difference between the obtained feature and the linear sum is calculated. Alternatively, the trained decoder is further used to generate a re-decoded sample from the feature obtained from the decoded sample, and the reconstruction error between the decoded sample and the re-decoded sample is calculated. The trained encoder and decoder are generated by machine learning using the training sample included in the first data set. Therefore, the more similar the sample decoded by the trained decoder from the derived linear sum is to the training sample, the smaller the above-mentioned differences (difference between features / reconstruction error) will be, and the more the decoded sample deviates from the training sample, the larger the above-mentioned differences will be. Therefore, the degree of the difference between the sample decoded from the linear sum and the training sample can be appropriately evaluated by the above-mentioned differences. In this configuration, based on the result of this evaluation, the linear sum is corrected so as to improve reliability, and a trained decoder is used to generate samples decoded from the features of the corrected linear sum as new training samples. This makes it possible to prevent training samples that are significantly different from the true data from being added to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, it is possible to increase the expectation of improving the performance of the generated trained classification model.
[0021] In the model generating device according to the above aspect, the machine learning of the encoder and the decoder may further include: for pairs of training samples obtained from the multiple first datasets, using the encoder to obtain two features corresponding to the two training samples constituting each pair; calculating a composite feature for each pair by combining the two features obtained for each pair at a predetermined internal division ratio; using the decoder to generate a decoded sample for each pair from the calculated composite feature; training an estimator such that, for each pair, a result of estimating the internal division ratio by the estimator from the generated decoded sample conforms to the predetermined internal division ratio; and training the encoder and decoder to generate, for each pair, a decoded sample that degrades the performance of the estimator for estimating the internal division ratio.
[0022] In this configuration, adversarial learning is performed between the encoder and decoder and the estimator. The more the ability of the estimator to distinguish the internal division ratio is improved by the adversarial learning, the more the ability of the encoder and decoder to generate samples whose internal division ratio is difficult to estimate is improved. In other words, for points between each sample point, it is possible to suppress the decoding (generation) of samples that deviate from true data whose internal division ratio is easy to specify (in other words, it is possible to make the decoder decode samples whose internal division ratio is difficult to specify and that are indistinguishable from true data). Therefore, according to this configuration, it is possible to generate trained encoders and decoders that have acquired the ability to generate decoded samples similar to true data (training samples) from the features of a linear sum. This makes it possible to suppress the addition of training samples that are extremely deviated from true data to the learning data when generating new training samples using the trained encoder and decoder. As a result, by using the obtained training samples for machine learning of the classification model, it is possible to increase the expectation of improved performance of the generated trained classification model.
[0023] In the model generating device according to the above aspect, the point designation unit may be configured to extract two sample points near the derived linear sum from among the two or more acquired sample points, and to project the derived linear sum onto a line connecting the two extracted sample points in the feature space, thereby acquiring a projection point corresponding to the linear sum. The sample generating unit may be configured to generate a decoded sample corresponding to a feature of the projection point corresponding to the linear sum as the new training sample by using the trained decoder. According to this configuration, the linear sum can be converted into an internal division ratio (projection point) of two sample points, thereby correcting the value of the linear sum to a value within or near the training range of the adversarial learning. This can increase the certainty of suppressing the addition of a training sample that is extremely deviated from true data to the learning data. As a result, by using the obtained training sample for machine learning of a classification model, it is possible to increase the expectation of improving the performance of the generated trained classification model.
[0024] In the model generating device according to the above aspect, the extraction of the two sample points may be configured by extracting two sample points to which features included in the corresponding training samples belong to different classes. With this configuration, it is possible to suppress the generation of a new training sample that deviates from true data between training samples of different classes. In other words, it is possible to generate a decoded sample that is similar to true data as a new training sample by the trained decoder within a range closer to the classification boundary of the class than each sample point. As a result, it is possible to increase the expectation of improving the performance of the generated trained classification model by using the obtained training sample for machine learning of the classification model.
[0025] The model generating device according to the above aspect may further include an output unit configured to output the two or more training samples and the new training sample after generating the new training sample, after arranging them according to their positional relationship in the feature space. According to this configuration, it is possible to confirm whether or not the decoded sample (the new training sample) can be added to the learning data based on the output result. This makes it possible to prevent a training sample that is extremely different from true data from being added to the learning data. As a result, it is possible to increase the expectation of improving the performance of the generated trained classification model by using the obtained training sample for machine learning of the classification model. In addition, it is possible to confirm the class to which the feature included in the decoded sample (the new training sample) belongs according to its positional relationship in the feature space with the original training sample based on the output result.
[0026] In the model generating device according to the above aspect, the labeling unit may be configured to, after outputting the two or more training samples and the new training sample by the output unit, accept a designation of a class to which a feature included in the new training sample belongs, and assign the new correct label configured to indicate the designated class to the new training sample. According to this configuration, based on the output result, a new correct label indicating an appropriate class can be assigned to the new training sample by accepting a designation of the class while checking a class to which a feature included in the decoded sample (new training sample) belongs according to a positional relationship in a feature space with the original training sample. This makes it possible to prevent an incorrect correct label from being assigned to the new training sample. As a result, the obtained training sample can be appropriately used for machine learning of a classification model, and it is possible to increase the expectation of improved performance of the generated trained classification model.
[0027] The model generating device according to the above aspect may further include a thinning unit configured to determine whether or not a density of sample points corresponding to each training sample in the feature space exceeds a threshold when each training sample included in the plurality of first data sets and the one or more second data sets is converted into a feature by the trained encoder, and to select a target to be thinned out from the one or more second data sets when the density exceeds the threshold. The second training processing unit may be configured to perform machine learning of the classification model, excluding the second data set selected as the target to be thinned out. When there are excessive training samples in a nearby range in the space, some of the training samples may not contribute much to improving the performance of the classification model. According to this configuration, such an excessive state of training samples can be found by the density in the feature space, and some of the excessive training samples can be thinned out. As a result, by reducing the number of data sets used for machine learning of the classification model, it is possible to reduce the amount of calculation for machine learning while suppressing a decrease in performance of the generated trained classification model, and to improve the efficiency of computational resources (processor, memory) required for machine learning.
[0028] In the model generating device according to the above aspect, the classification model may be configured to receive an input of the feature amount and output a result of classifying the class to which the feature amount belongs based on the input feature amount. With this configuration, in a form in which the calculation of the classification task is shared with the processing of the encoder, it is possible to improve the performance of the generated trained classification model. In this form, since the classification model can be trained together with the training of the encoder, it is possible to improve the efficiency of the machine learning processing of the classification model.
[0029] In the model generating device according to the above aspect, the classification model may be configured to receive the input of the sample, and output a result of classifying the input sample into a class to which the feature belongs. With this configuration, in a configuration in which the classification model is configured separately from the encoder and the decoder, it is possible to improve the performance of the generated trained classification model. In this configuration, since the configuration of the classification model does not depend on the encoder, the structure of the classification model may be optimized for the classification task. This may improve the efficiency of the computational effort of the classification model.
[0030] The model generating device according to the above aspect may be configured to repeatedly execute the following: updating the trained encoder and decoder by using the one or more second data sets generated by the first training processing unit to perform machine learning of the encoder and decoder again; acquiring two or more sample points corresponding to two or more training samples among a plurality of training samples included in the plurality of first data sets and the one or more second data sets by the conversion unit using the updated trained encoder; deriving a linear sum of the acquired two or more sample points by the point designation unit; generating a decoded sample corresponding to a feature of the linear sum as a new training sample by the sample generation unit using the updated trained decoder; and generating a new second data set by assigning a new correct label to the generated new training sample by the label assignment unit. When a new training sample is added and machine learning of the encoder and decoder is performed using the added new training sample, the feature space mapped by the encoder may be changed. According to this configuration, by repeating the machine learning of the encoder and decoder and the generation of new training samples, a state in which sample points are evenly present while adding new training samples can be formed. This allows for a better collection of comprehensive training samples.
[0031] In the model generating device according to the aforementioned aspect, the predetermined index may be formed by either Equation 1 or Equation 2 below.
number
number
[0032] In the model generating device according to each of the above aspects, the data type of the training sample is not particularly limited as long as it can be a target of the classification task, and may be appropriately selected according to the embodiment. The training sample may be, for example, image data, sound data, numerical data, text data, other sensing data obtained by a sensor, etc. The same applies to target samples obtained when performing a classification task using a trained classification model.
[0033] For example, in the model generating device according to the above aspect, the training samples may be image samples showing objects, and the class of the features may be related to the type of the object shown in the image samples. With this configuration, it is possible to reduce the cost of comprehensively collecting training samples when generating a trained classification model for object recognition. In addition, by using the training samples obtained in this way for machine learning, it is possible to improve the performance of the generated trained classification model.
[0034] In the model generating device according to the above aspect, the object may be a product, and the class of the feature may relate to a type of defect contained in the product. With this configuration, when generating a trained classification model for performing visual inspection as a form of object recognition, it is possible to reduce the cost of comprehensively collecting training samples. Furthermore, by using the training samples thus obtained for machine learning, it is possible to improve the performance of the generated trained classification model.
[0035] The embodiment of the present invention may not be limited to a model generating device configured to execute all of the above series of processes. An apparatus according to an aspect of the present invention may be configured by extracting a part from the model generating apparatus according to each of the above embodiments, such as a part that generates a trained encoder and a decoder, a part that generates a new training sample (second data set) by the trained encoder and the decoder, and a part that performs machine learning of a classification model using learning data including the new training sample. In one example, an apparatus corresponding to the part that generates a trained encoder and a decoder may be referred to as a first model generating apparatus. The part that generates a new training sample by the trained encoder and the decoder may be referred to as a data generating apparatus. The part that performs machine learning of a classification model using learning data including the new training sample may be referred to as a second model generating apparatus. Furthermore, one aspect of the present invention may be a classification apparatus that uses a trained classification model generated by the model generating apparatus according to any of the above embodiments. The classification apparatus may be appropriately interpreted according to the type of classification task (for example, an inspection apparatus, etc.).
[0036] For example, a classification device according to one aspect of the present invention includes an acquisition unit configured to acquire a target sample, a classification unit including the trained classification model generated by the model generation device according to any one of the above aspects, configured to classify a class to which a feature included in the acquired target sample belongs by using the trained classification model, and an output unit configured to output information related to a result of classifying the class. With this configuration, it is expected that a classification task can be performed with high accuracy by using the trained classification model generated as described above.
[0037] Also, for example, a data generating device according to one aspect of the present invention includes a data acquiring unit configured to acquire two or more training samples, a conversion unit configured to acquire two or more sample points in a feature space corresponding to each training sample by converting each of the two or more training samples into a feature using a trained encoder, a point designation unit configured to derive a linear sum in the feature space of the two or more acquired sample points, which is a linear sum that maximizes the sum of distances calculated from each sample point according to a predetermined index, and a sample generating unit configured to generate a decoded sample corresponding to the feature of the linear sum as a new training sample by using a trained decoder. The trained encoder and decoder are generated by machine learning using a plurality of training samples including the two or more training samples. The machine learning includes converting each training sample into a feature by the encoder, and training the encoder and decoder so that a sample decoded by the decoder from each obtained feature matches each training sample. With this configuration, it is possible to reduce the cost required for comprehensively collecting training samples.
[0038] As another aspect of each of the devices according to the above embodiments, one aspect of the present invention may be an information processing method or a program for realizing all or part of each of the above configurations, or may be a storage medium storing such a program and readable by a computer or other device, machine, etc. Here, a storage medium readable by a computer, etc. is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action. Another aspect of the present invention may be a classification system including a model generation device (first model generation device, data generation device, and second model generation device) according to any of the above embodiments and a classification device.
[0039] For example, a model generation method according to one aspect of the present invention is an information processing method comprising the steps of: acquiring a plurality of first datasets, each of which is composed of a combination of the training samples and a ground truth label; performing machine learning of the encoder and the decoder using a plurality of training samples included in the acquired plurality of first datasets; acquiring two or more sample points in a feature space corresponding to each training sample by converting each of two or more training samples among the plurality of training samples into features using the trained encoder; deriving a linear sum in the feature space, which is a linear sum of the acquired two or more sample points and which maximizes a sum of distances calculated from each sample point according to a predetermined index; generating a decoded sample corresponding to the feature of the linear sum as a new training sample by using the trained decoder; generating a second dataset by assigning a new ground truth label to the new training sample configured to indicate a class to which a feature included in the generated new training sample belongs; and performing machine learning of the classification model using the plurality of first datasets and the generated one or more second datasets.
[0040] Furthermore, for example, a model generation program according to one aspect of the present invention is a program for causing a computer to execute the steps of: acquiring a plurality of first datasets, each of which is composed of a combination of the training samples and a ground truth label; performing machine learning of the encoder and the decoder using a plurality of training samples included in the acquired plurality of first datasets; acquiring two or more sample points in a feature space corresponding to each training sample by converting each of two or more training samples among the plurality of training samples into features using the trained encoder; deriving a linear sum in the feature space, which is a linear sum of the acquired two or more sample points and which maximizes the sum of distances calculated from each sample point according to a predetermined index; generating a decoded sample corresponding to the feature of the linear sum as a new training sample by using the trained decoder; generating a second dataset by assigning a new ground truth label to the new training sample configured to indicate a class to which a feature included in the generated new training sample belongs; and performing machine learning of the classification model using the plurality of first datasets and the generated one or more second datasets. Effect of the Invention
[0041] According to the present invention, it is possible to reduce the cost of comprehensively collecting training samples, and by using the training samples thus obtained for machine learning, it is possible to improve the performance of a generated trained classification model. [Brief description of the drawings]
[0042] [Figure 1] FIG. 1 illustrates a schematic example of a situation in which the present invention is applied. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of the model generating device according to the embodiment. [Diagram 3]FIG. 3 illustrates an example of a hardware configuration of the classification device according to the embodiment. [Figure 4] FIG. 4 illustrates an example of a software configuration of the model generating device according to the embodiment. [Figure 5A] FIG. 5A illustrates a schematic diagram of an example of a machine learning process for an encoder, a decoder, and a classification model. [Figure 5B] FIG. 5B illustrates a schematic diagram of an example of the machine learning process of the encoder and decoder. [Figure 5C] FIG. 5C illustrates a schematic diagram of an example of a data generation process. [Figure 5D] FIG. 5D illustrates a schematic diagram of an example of a data generation process. [Figure 5E] FIG. 5E illustrates a schematic example of an output format of new training samples. [Figure 6] FIG. 6 illustrates an example of a software configuration of the classification device according to the embodiment. [Figure 7A] FIG. 7A is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 7B] FIG. 7B is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a processing procedure of the classification device according to the embodiment. [Figure 9] FIG. 9 illustrates a schematic example of another scene to which the present invention is applied. [Figure 10] FIG. 10 illustrates a schematic example of another scene to which the present invention is applied. [Figure 11] FIG. 11 illustrates a schematic example of another scene to which the present invention is applied. [Figure 12] FIG. 12 illustrates an example of a configuration of a model generating device according to another embodiment. [Figure 13] FIG. 13 illustrates an example of a configuration of a classification model according to another embodiment. [Figure 14]FIG. 14 illustrates an example of a process of machine learning (adversarial learning) of a classifier according to another embodiment. [Figure 15] FIG. 15 illustrates an example of a data generation process according to another embodiment. [Figure 16] FIG. 16 is a diagram for explaining an example of an optimization method for obtaining a corrected linear sum in another embodiment. [Figure 17] FIG. 17 illustrates an example of a data generation process according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0043] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.
[0044] §1 Examples of application 1 is a schematic diagram illustrating an example of a situation in which the present invention is applied. As shown in FIG. 1, a classification system 100 according to the present embodiment includes a model generating device 1 and a classification device 2.
[0045] The model generating device 1 according to the present embodiment is a computer configured to perform machine learning of the encoder 5 and the decoder 6, generation of new training samples 36 using the trained encoder 5 and the decoder 6, and machine learning of the classification model 7. Specifically, the model generating device 1 according to the present embodiment acquires a plurality of first data sets 30. Each of the first data sets 30 is composed of a combination of a training sample 31 and a correct answer label 32. The training sample 31 includes a feature. The correct answer label 32 is configured to indicate a class to which a feature included in the corresponding training sample 31 belongs (i.e., a correct answer to a classification task for the feature).
[0046] As long as each training sample 31 can be a target of a classification task, the data type may not be particularly limited and may be appropriately selected according to the embodiment. Each training sample 31 may be, for example, image data, sound data, numerical data, text data, other sensing data obtained by a sensor, etc. In one example, each training sample 31 may be sensing data obtained by observing an arbitrary object by a sensor. The sensor may be, for example, an image sensor (camera), an infrared sensor, a sound sensor (microphone), an ultrasonic sensor, a light sensor, a pressure sensor, a barometric pressure sensor, a temperature sensor, etc. In addition, the sensor may be, for example, an environmental sensor, a vital sensor, a medical testing device, an in-vehicle sensor, a home security sensor, etc. The environmental sensor may be, for example, a barometer, a thermometer, a hygrometer, a sound pressure meter, a sound sensor, an ultraviolet sensor, a luminance meter, a rain gauge, a gas sensor, etc. The vital sensor may be, for example, a blood pressure monitor, a pulse rate monitor, a heart rate monitor, an electrocardiograph, an electromyograph, a thermometer, a skin electrodermal response monitor, a microwave sensor, an electroencephalograph, a magnetoencephalograph, an activity monitor, a blood glucose level meter, an electrooculography sensor, an eye movement measuring instrument, or the like. The medical examination device may be, for example, a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, or the like. The in-vehicle sensor may be, for example, an image sensor, a light detection and ranging (Lidar) sensor, a millimeter wave radar, an ultrasonic sensor, an acceleration sensor, or the like. The home security sensor may be, for example, an image sensor, an infrared sensor, an activity (audio) sensor, a gas (CO2, etc.) sensor, a current sensor, a smart meter (a sensor that measures the amount of power used by home appliances, lighting, etc.), or the like.
[0047] The contents of the classification task may be appropriately determined according to the embodiment. In one example, each training sample 31 may be an image sample in which an object is shown. In this case, the classification task may be to identify the type of object. The class of the feature may relate to the type of object shown in the image sample. The correct answer label 32 may be configured to indicate the type of object shown in the corresponding training sample 31. In another example, each training sample 31 may be a sample of sensing data (e.g., vital data obtained by a vital sensor, image data obtained by an image sensor, etc.) reflecting the state of the subject (subject). In this case, the classification task may be to identify the state of the subject (e.g., to determine whether the subject is healthy or not, whether there is a sign of illness or not, whether the subject is normal or not, etc.). The class of the feature may relate to the type of the state of the subject. The correct answer label 32 may be configured to indicate the state of the subject (subject) appearing in the corresponding training sample 31.
[0048] In this embodiment, the model generating device 1 performs machine learning of the encoder 5 and the decoder 6 using a plurality of training samples 31 included in the acquired plurality of first data sets 30. The encoder 5 is configured to convert the input sample into a feature (i.e., a latent representation). The decoder 6 is configured to decode the sample from the feature. The machine learning of the encoder 5 and the decoder 6 includes converting each training sample 31 into a feature by the encoder 5, and training the encoder 5 and the decoder 6 so that a sample decoded by the decoder 6 from each obtained feature matches each training sample 31. As a result of this machine learning, a trained encoder 5 and a trained decoder 6 are generated. The trained encoder 5 can map each training sample 31 into a feature space (latent space) while maintaining information about the included features. The trained decoder 6 can decode a sample from the value (feature) of the feature space.
[0049] In this embodiment, the model generating device 1 uses a trained encoder 5 to convert two or more training samples 31 out of a plurality of training samples 31 into features, thereby acquiring two or more sample points 311 corresponding to the two or more training samples 31 in the feature space. The model generating device 1 derives a linear sum 312, which is a linear sum of the acquired two or more sample points 311 in the feature space and maximizes the sum of distances calculated from the two or more sample points 311 according to a predetermined index. The model generating device 1 uses a trained decoder 6 to generate a decoded sample (i.e., a sample decoded from the linear sum 312) corresponding to the feature of the linear sum 312 as a new training sample 36.
[0050] In this embodiment, the model generating device 1 assigns a new correct answer label 37 to the new training sample 36 generated so as to indicate a class to which a feature included in the new training sample 36 belongs. In this way, the model generating device 1 generates a second dataset 35 composed of a combination of the new training sample 36 and the correct answer label 37. The first dataset 30 may be referred to as an original dataset, and the second dataset 35 may be referred to as a new dataset. However, "original" and "new" merely indicate a relationship in one cycle of processing in which a trained encoder 5 and a decoder 6 are generated by the above machine learning, and a new training sample 36 is generated using the trained encoder 5 and the decoder 6. The multiple first datasets 30 may include at least one of the second dataset 35 generated previously and a dataset generated by another method.
[0051] The model generation device 1 performs machine learning of the classification model 7 using a plurality of first data sets 30 and one or more second data sets 35. The machine learning of the classification model 7 includes training the classification model 7 so that the result of classifying the class to which the features included in the training samples (31, 36) belong for each data set (30, 35) by the classification model 7 matches the correct label (32, 37). As a result of the machine learning, a trained classification model 7 can be generated that has acquired the ability to perform the task of classifying the features included in the data samples of the same type as the training samples (32, 37). The generated trained classification model 7 may be provided to the classification device 2 at any timing.
[0052] The classification device 2 is a computer configured to perform a classification task using a trained classification model 7. Specifically, the classification device 2 according to this embodiment acquires a target sample 221. The target sample 221 is the same type of data as the training samples (31, 36). The classification device 2 includes a trained classification model 7 generated by the model generation device 1, and classifies a class to which a feature contained in the acquired target sample 221 belongs by using the trained classification model 7. The classification device 2 outputs information related to the result of classifying the class. Note that the classification device 2 may be interpreted as appropriate depending on the type of classification task (for example, an inspection device, etc.).
[0053] As described above, according to the model generating device 1 of this embodiment, the linear sum 312 with the maximum total distance from each sample point 311 is obtained as a new sample point, so that a state in which sample points (sample points 311, linear sum 312) are evenly present in the feature space can be formed. Therefore, by using the trained decoder 6 to add a new data set (second data set 35) including a sample (training sample 36) decoded from the feature amount of the linear sum 312 to the learning data, learning data that can be evaluated in the feature space as being highly comprehensive can be obtained. Furthermore, according to the model generating device 1 of this embodiment, by using the trained encoder 5 and decoder 6, at least a part of the process of generating such a new training sample (data set including the new training sample) can be automated. Therefore, it is possible to reduce the cost required for comprehensively collecting training samples. Furthermore, in the model generating device 1 of this embodiment, in addition to the original data set (first data set 30), the new data set (second data set 35) generated as described above is used for machine learning of the classification model 7. As a result, according to the model generation device 1 of this embodiment, the comprehensively obtained training samples (31, 36) are used for machine learning of the classification model 7, thereby improving the performance of the generated trained classification model 7. In the classification device 2 of this embodiment, by using the trained classification model 7 generated in this manner, it can be expected to perform a classification task with high accuracy for the target sample 221.
[0054] In the example of FIG. 1, the model generating device 1 and the classification device 2 are connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of exchanging data between the model generating device 1 and the classification device 2 is not limited to this example, and may be appropriately selected depending on the embodiment. For example, data may be exchanged between the model generating device 1 and the classification device 2 using a storage medium.
[0055] 1, the model generating device 1 and the classification device 2 are each configured by separate computers. However, the configuration of the classification system 100 according to this embodiment is not limited to this example and may be appropriately determined according to the embodiment. For example, the model generating device 1 and the classification device 2 may be an integrated computer. Also, for example, at least one of the model generating device 1 and the classification device 2 may be configured by multiple computers.
[0056] §2 Configuration Example [Hardware configuration] <Model generation device> Fig. 2 illustrates a schematic example of a hardware configuration of the model generating device 1 according to this embodiment. As shown in Fig. 2, the model generating device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. In Fig. 2, the communication interface and the external interface are described as a "communication I / F" and an "external I / F".
[0057] The control unit 11 includes a hardware processor (processor resource) such as a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and is configured to execute information processing based on programs and various data. The storage unit 12 is an example of a memory (memory resource), and is configured, for example, by a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, a plurality of first data sets 30, one or more second data sets 35, learning result data 125, and the like.
[0058] The model generation program 81 is a program for causing the model generation device 1 to execute information processing (FIGS. 7A and 7B) related to machine learning of the encoder 5 and the decoder 6, generation of new training samples 36 using the trained encoder 5 and the decoder 6, and machine learning of the classification model 7, which will be described later. The model generation program 81 includes instructions for an example of the information processing. Note that at least one part related to each of the above information processing may be configured as a separate program. In this case, the subprograms related to the machine learning of the encoder 5 and the decoder 6 and the subprograms related to the machine learning of the classification model 7 may be referred to as a model generation program, a learning program, or the like. The subprograms related to the generation of new training samples 36 using the trained encoder 5 and the decoder 6 may be referred to as a data generation program.
[0059] The multiple first data sets 30 are used for the encoder 5 and the decoder 6, and for machine learning of the classification model 7. The multiple first data sets 30 used for the machine learning of the encoder 5 and the decoder 6 and the multiple first data sets 30 used for the machine learning of the classification model 7 may not necessarily be completely identical. The second data set 35 is generated as a result of executing information processing related to the generation of new training samples 36 using the trained encoder 5 and the decoder 6. The second data set 35 is used for machine learning of the classification model 7. The learning result data 125 indicates information related to the trained classification model 7 generated by machine learning. In this embodiment, the learning result data 125 is generated as a result of executing the model generation program 81. Details will be described later.
[0060] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. The model generating device 1 can use the communication interface 13 to perform data communication via a network with other information processing devices. The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting to an external device. The type and number of the external interfaces 14 may be selected arbitrarily. In one example, the training sample 31 may be obtained by a sensor. In another example, the training sample 31 may be generated by another computer. In these cases, the model generating device 1 may be connected to the sensor or another computer via at least one of the communication interface 13 and the external interface 14.
[0061] The input device 15 is a device for performing input, such as a mouse, a keyboard, etc. The output device 16 is a device for performing output, such as a display, a speaker, etc. An operator such as a user can operate the model generation device 1 by using the input device 15 and the output device 16.
[0062] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as a program stored in a storage medium 91. The storage medium 91 is a medium for storing information such as a program by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read the various information such as the stored program. At least one of the model generation program 81, the first data set 30, and the second data set 35 may be stored in the storage medium 91. The model generation device 1 may read at least one of the model generation program 81, the first data set 30, and the second data set 35 from the storage medium 91. Note that in FIG. 2, a disk-type storage medium such as a CD or a DVD is illustrated as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type, and may be other than the disk type. An example of a storage medium other than the disk type may be a semiconductor memory such as a flash memory. The type of the drive 17 may be selected arbitrarily according to the type of the storage medium 91.
[0063] In addition, regarding the specific hardware configuration of the model generating device 1, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 12 may be configured with a RAM and a ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The model generating device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The model generating device 1 may be an information processing device designed exclusively for the service provided, as well as a general-purpose server device, a PC (Personal Computer), or the like.
[0064] <Classification device> Fig. 3 illustrates an example of a hardware configuration of the classification device 2 according to this embodiment. As illustrated in Fig. 3, the classification device 2 according to this embodiment is a computer to which a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a drive 27 are electrically connected.
[0065] The control unit 21 to the drive 27 and the storage medium 92 of the classification device 2 may be configured similarly to the control unit 11 to the drive 17 and the storage medium 91 of the model generation device 1. The control unit 21 includes a CPU, RAM, ROM, etc., which are hardware processors, and is configured to execute various information processing based on programs and data. The storage unit 22 is configured, for example, with a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 22 stores various information such as a classification program 82, learning result data 125, etc.
[0066] The classification program 82 is a program for causing the classification device 2 to execute information processing ( FIG. 8 ), which will be described later, for performing a classification task using the trained classification model 7. The classification program 82 includes a series of instructions for the information processing. At least one of the classification program 82 and the learning result data 125 may be stored in a storage medium 92. The classification device 2 may read at least one of the classification program 82 and the learning result data 125 from the storage medium 92.
[0067] Similar to the training samples 31, in one example, the subject samples 221 may be obtained by a sensor. In another example, the subject samples 221 may be generated by another computer. In these cases, the classification device 2 may be connected to the sensor or the other computer via the communication interface 23 and / or the external interface 24.
[0068] In addition, regarding the specific hardware configuration of the classification device 2, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, FPGA, DSP, etc. The storage unit 22 may be configured with a RAM and a ROM included in the control unit 21. At least one of the communication interface 23, the external interface 24, the input device 25, the output device 26, and the drive 27 may be omitted. The classification device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The classification device 2 may be an information processing device designed exclusively for the service provided, as well as a general-purpose server device, a general-purpose PC, an industrial PC, a tablet PC, a terminal device (e.g., a smartphone, etc.), a PLC (programmable logic controller), etc.
[0069] [Software configuration] <Model generation device> FIG. 4 illustrates an example of a software configuration of the model generating device 1 according to the present embodiment. The control unit 11 of the model generating device 1 expands the model generating program 81 stored in the storage unit 12 into the RAM. Then, the control unit 11 interprets and executes the instructions included in the model generating program 81 expanded into the RAM by the CPU to control each component. As a result, as shown in FIG. 4, the model generating device 1 according to the present embodiment operates as a computer including a data acquiring unit 111, a first training processing unit 112, a converting unit 113, a point designating unit 114, a sample generating unit 115, an output unit 116, a labeling unit 117, a thinning unit 118, a second training processing unit 119, and a saving processing unit 1110 as software modules. That is, in the present embodiment, each software module of the model generating device 1 is realized by the control unit 11 (CPU).
[0070] The data acquisition unit 111 is configured to acquire a plurality of first data sets 30 each composed of a combination of a training sample 31 and a correct answer label 32. The training sample 31 includes a feature. The correct answer label 32 is configured to indicate a class to which a feature included in the corresponding training sample 31 belongs.
[0071] The first training processing unit 112 is configured to perform machine learning of the encoder 5 and the decoder 6 using a plurality of training samples 31 included in the acquired plurality of first datasets 30. The encoder 5 is configured to receive input of samples and convert the input samples into features. The decoder 6 is configured to receive input of features and decode samples from the input features. The machine learning of the encoder 5 and the decoder 6 includes converting each training sample 31 into a feature by the encoder 5, and training the encoder 5 and the decoder 6 so that samples decoded by the decoder 6 from each obtained feature match each training sample 31.
[0072] The conversion unit 113 is configured to convert each of two or more training samples 31 out of the multiple training samples 31 into a feature using the trained encoder 5, thereby acquiring two or more sample points 311 in the feature space corresponding to each training sample 31. The point designation unit 114 is configured to derive a linear sum 312 of the acquired two or more sample points 311 in the feature space, the linear sum 312 being a maximum sum of distances calculated from each sample point 311 according to a predetermined index.
[0073] The sample generation unit 115 is configured to generate a decoded sample corresponding to the feature quantity of the linear sum 312 as a new training sample 36 by using the trained decoder 6. The output unit 116 is configured to output the new training sample 36 after generating the new training sample 36, after arranging the two or more training samples 31 and the new training sample 36 according to their positional relationship in the feature space. The label assignment unit 117 is configured to generate a second dataset 35 by assigning to the new training sample 36 a new correct answer label 37 configured to indicate a class to which a feature included in the generated new training sample 36 belongs.
[0074] The thinning unit 118 is configured to determine whether or not the density of sample points corresponding to each training sample (31, 36) in the feature space exceeds a threshold when each training sample (31, 36) included in the multiple first datasets 30 and the one or more generated second datasets 35 is converted into a feature by the trained encoder 5, and to select a target to be thinned out from the one or more second datasets 35 if the density exceeds the threshold.
[0075] The second training processing unit 119 is configured to perform machine learning of the classification model 7 using the multiple first datasets 30 and the one or more generated second datasets 35. When a target to be thinned is selected by the thinning unit 118, the second training processing unit 119 is configured to perform machine learning of the classification model 7, excluding the second dataset 35 selected as the target to be thinned. The machine learning of the classification model 7 includes training the classification model 7 so that the result of classifying the class to which the feature included in the training sample (31, 36) belongs by the classification model 7 for each dataset (30, 35) matches the correct label (32, 37). The saving processing unit 1110 generates information about the trained classification model 7 generated by the machine learning as learning result data 125. Then, the saving processing unit 1110 saves the generated learning result data 125 in a predetermined storage area.
[0076] (An example of the machine learning process) 5A illustrates an example of a training process for the machine learning of the encoder 5 and the decoder 6, and for the machine learning of the classification model 7. FIG. 5B illustrates an example of a further training process for the machine learning of the encoder 5 and the decoder 6.
[0077] As shown in FIG. 5A, training samples 31 are used for training the encoder 5 and the decoder 6. First, the first training processing unit 112 inputs each training sample 31 to the encoder 5 and executes the arithmetic processing of the encoder 5. Based on the arithmetic results of the encoder 5, the first training processing unit 112 acquires features corresponding to each training sample 31. Next, the first training processing unit 112 inputs each acquired feature to the decoder 6 and executes the arithmetic processing of the decoder 6. Based on the arithmetic results of the decoder 6, the first training processing unit 112 acquires a sample decoded from each feature. Then, the first training processing unit 112 calculates an error (reconstruction error) between each decoded sample and each corresponding training sample 31, and adjusts the parameter values of the encoder 5 and the decoder 6 so that the calculated error becomes small. This allows the encoder 5 to convert each training sample 31 into a feature, and the encoder 5 and the decoder 6 to be trained so that the sample decoded by the decoder 6 from the acquired feature matches each training sample 31.
[0078] Additionally, in this embodiment, in the machine learning of the encoder 5 and the decoder 6, adversarial learning as exemplified in Fig. 5B may be implemented. In this case, an estimator 600 may be provided corresponding to the encoder 5 and the decoder 6. The estimator 600 may be appropriately configured to estimate the internal division ratio from the decoded sample when the decoder 6 generates a decoded sample from a combined feature obtained by combining features corresponding to two samples at an arbitrary internal division ratio.
[0079] As an example of the adversarial learning process, first, the first training processing unit 112 acquires two features corresponding to the two training samples (31_1, 31_2) constituting each pair of training samples 31 obtained from a plurality of first data sets 30 by using the encoder 5. Next, the first training processing unit 112 calculates a composite feature for each pair by combining the two features acquired for each pair at a predetermined internal division ratio (α:1-α). The predetermined internal division ratio (α value) may be appropriately determined or changed for each pair / each training iteration. Next, the first training processing unit 112 generates a decoded sample for each pair from the calculated composite feature by using the decoder 6.
[0080] Then, the first training processing unit 112 inputs the decoded samples generated for each pair to the estimator 600, and executes the calculation process of the estimator 600. As the calculation result of this estimator 600, the first training processing unit 112 obtains the result of estimating the internal division ratio from the decoded sample for each pair. The first training processing unit 112 calculates the error (estimated error) between the obtained estimation result of the internal division ratio and the corresponding true value of the internal division ratio (α / 1-α), and adjusts the parameter value of the estimator 600 so that the calculated error becomes small. In this way, the first training processing unit 112 trains the estimator 600 so that the result of estimating the internal division ratio by the estimator 600 from the generated decoded samples for each pair matches the predetermined internal division ratio.
[0081] In addition, the first training processing unit 112 inputs the decoded samples generated for each pair to the estimator 600, and executes the calculation process of the estimator 600. As the calculation result of this estimator 600, the first training processing unit 112 obtains the result of estimating the internal division ratio from the decoded sample for each pair. The first training processing unit 112 adjusts the parameter values of the encoder 5 and the decoder 6 so that the error between the obtained estimation result of the internal division ratio and the true value of the corresponding internal division ratio becomes large. In one example, the first training processing unit 112 calculates the error between the obtained estimation result of the internal division ratio and a constant value (0 in FIG. 5B), and adjusts the parameter values of the encoder 5 and the decoder 6 so that the calculated error becomes small. In this way, the first training processing unit 112 trains the encoder 5 and the decoder 6 so as to generate decoded samples for each pair that reduce the performance of the estimator 600 that estimates the internal division ratio (i.e., reduce the performance of the estimator 600 that estimates the internal division ratio from the generated decoded samples).
[0082] On the other hand, as shown in FIG. 5A, each data set (30, 35) is used for training the classification model 7. In this embodiment, the classification model 7 is configured to receive input of features obtained by the encoder 5 and output a result of classifying the class to which the features belong from the input features. That is, the calculation process of the classification task is configured to include the calculation of the encoder 5. Therefore, first, the second training processing unit 119 inputs each training sample (31, 36) included in each data set (30, 35) to the encoder 5 and executes the calculation process of the encoder 5. Based on the calculation result of the encoder 5, the second training processing unit 119 acquires the feature corresponding to each training sample (31, 36). Next, the second training processing unit 119 inputs each acquired feature to the classification model 7 and executes the calculation result of the classification model 7. Based on the calculation result of the classification model 7, the second training processing unit 119 acquires the result of classifying the class to which the features included in each training sample (31, 36) belong. Then, the second training processing unit 119 calculates the error (classification error) between the obtained classification result and the true value (correct classification) indicated by each corresponding correct label (32, 37), and adjusts the parameter values of the classification model 7 so as to reduce the calculated error. This makes it possible to train the classification model 7 so that the result of classifying the classes to which the features included in the training samples (31, 36) belong for each data set (30, 35) by the classification model 7 matches the correct label (32, 37).
[0083] The learning result data 125 is configured to include information for executing the calculation of the classification task. In this embodiment, as described above, in response to the classification model 7 being configured to use the calculation result of the encoder 5, the storage processing unit 1110 generates information for reproducing the trained encoder 5 and the trained classification model 7 as the learning result data 125. Note that information regarding other trained machine learning models (in this embodiment, the decoder 6 and the estimator 600) may or may not be included in the learning result data 125.
[0084] A part of the machine learning of the classification model 7 (specifically, a part using the first dataset 30) may be executed simultaneously with the machine learning of the encoder 5 and the decoder 6. In this case, in the machine learning process, the parameter values of the encoder 5 may be adjusted so as to reduce the classification error. According to this configuration, at least a part of the training of the classification model 7 can be performed together with the training of the encoder 5, so that the efficiency of the machine learning process of the classification model 7 can be improved.
[0085] However, the timing of the machine learning of the classification model 7 may not be limited to such an example. The machine learning of the classification model 7 may be performed separately from the machine learning of the encoder 5 and the decoder 6. Furthermore, the configuration of the classification model 7 may not be limited to the above example. In another example, the calculation process of the classification task may be configured not to include the calculation of the encoder 5. As a specific example, the classification model 7 may be configured to receive the input of a sample and output a result of classifying the class to which the feature belongs from the input sample. In this case, information on the trained encoder 5 may be omitted from the learning result data 125. Note that the output format of the classification model 7 may not be particularly limited as long as the result of classifying the class to which the feature included in the sample belongs can be specified, and may be appropriately determined according to the embodiment. In one example, the classification model 7 may be configured to output the posterior probability that the feature included in the sample belongs to each class as a result of classifying the feature included in the input sample.
[0086] (An example of a process for generating new training samples) 5C and 5D are schematic diagrams illustrating an example of a process for generating new training samples 36. As shown in FIG. 5C, first, the conversion unit 113 converts each of two or more training samples 31 among the multiple training samples 31 into a feature using a trained encoder 5. As a result, the conversion unit 113 obtains two or more sample points 311 in the feature space, each of which corresponds to each training sample 31.
[0087] In addition, although FIG. 5C shows an example in which three sample points 311 are acquired, the number of sample points 311 to be acquired is not limited to three as long as it is two or more, and may be appropriately determined according to the embodiment. Furthermore, the two or more training samples 31 from which the sample points 311 are acquired may be appropriately selected from a plurality of training samples 31. In one example, in order to improve the classification accuracy of the classification model 7, it is preferable to generate a new training sample 36 in a range in which there are no samples near the discrimination boundary of different classes. Therefore, it is preferable that the two or more training samples 31 are selected so as to include training samples 31 associated with correct answer labels 32 indicating different classes.
[0088] Next, the point designation unit 114 derives a linear sum 312_1, which is a linear sum 312 of two or more acquired sample points 311 in the feature space and maximizes the sum of distances calculated from each sample point 311 according to a predetermined index. In this embodiment, this directly derived linear sum 312_1 may be used to generate a new training sample 36. That is, the sample generation unit 115 inputs the derived linear sum 312_1 to the trained decoder 6 and executes the arithmetic processing of the trained decoder 6. As a result, the sample generation unit 115 may generate a decoded sample corresponding to the linear sum 312_1 as a new training sample 36 by using the trained decoder 6.
[0089] Here, in this embodiment, the classification model 7 may be configured to output, for each class, a posterior probability to which a feature included in an input sample belongs. In this case, the above machine learning of the classification model 7 may be divided into a first machine learning using a plurality of first data sets 30 and a second machine learning using one or more second data sets 35. The first machine learning may be referred to as pre-learning for the second machine learning. The second machine learning may be referred to as re-learning or additional learning for the first machine learning. In the second machine learning, at least one of the plurality of first data sets 30 may be used again.
[0090] As shown in FIG. 5C, in this embodiment, the point designation unit 114 may use the generated provisional trained classification model 7 to evaluate the entropy of the sample to be decoded, and correct the linear sum 312_1 so that the entropy increases. Specifically, the point designation unit 114 uses the provisional trained classification model 7 to calculate the posterior probability to which the feature included in the sample to be decoded by the trained decoder 6 belongs for each class from the feature amount of the derived linear sum 312_1. In this embodiment, the point designation unit 114 inputs the derived linear sum 312_1 to the provisional trained classification model 7, and executes the calculation process of the provisional trained classification model 7. Through this calculation process, the point designation unit 114 can obtain the result of calculating the posterior probability to which the feature included in the sample to be decoded belongs for each class. Next, the point designation unit 114 calculates the entropy of the sample to be decoded from the calculated posterior probability for each class.
[0091] The entropy indicates the degree of whether or not the features included in the decoded sample can be classified (in other words, the uncertainty). The higher the uncertainty, the more difficult it is to classify the features included in the sample, that is, the machine learning of the sample is insufficient in the provisional trained classification model 7. The point designation unit 114 may correct the derived linear sum 312_1 so that the entropy increases based on the gradient of the calculated entropy. By this process, the point designation unit 114 may obtain the corrected linear sum 312_2. In this embodiment, the corrected linear sum 312_2 may be used to generate a new training sample 36. That is, the sample generation unit 115 may input the corrected linear sum 312_2 to the trained decoder 6 and execute the arithmetic process of the trained decoder 6. As a result, the sample generation unit 115 may generate a decoded sample corresponding to the corrected linear sum 312_2 as a new training sample 36 by using the trained decoder 6.
[0092] Furthermore, as shown in FIG. 5D, in response to the execution of the adversarial learning, the point designation unit 114 may extract two sample points (311_1, 311_2) adjacent to the linear sum 312_2 derived by the above calculation from among the two or more sample points 311 acquired. Then, the point designation unit 114 may acquire a projection point 313 corresponding to the linear sum 312_2 by projecting the derived linear sum 312_2 onto a straight line connecting the two extracted sample points (311_1, 311_2) in the feature space. When the correction based on the entropy is not executed, the point designation unit 114 may extract two sample points (311_1, 311_2) adjacent to the linear sum 312_1 and acquire a projection point 313 corresponding to the linear sum 312_1.
[0093] The extraction of the two sample points (311_1, 311_2) may be performed by extracting two sample points having different classes to which features included in the corresponding training samples 31 belong. That is, the point designation unit 114 may extract two sample points (311_1, 311_2) derived from training samples 31 having different true values of classes indicated by the associated correct answer labels 32 from the sample points 311 existing in the vicinity of the linear sum 312_2 (or the linear sum 312_1).
[0094] In this embodiment, the acquired projection point 313 may be used to generate a new training sample 36. That is, the sample generation unit 115 may input the value of the projection point 313 to the trained decoder 6 and execute the arithmetic processing of the trained decoder 6. In this way, the sample generation unit 115 may use the trained decoder 6 to generate a decoded sample corresponding to the feature amount of the projection point 313 as the new training sample 36.
[0095] (Example of output) 5E illustrates an example of an output format of the new training sample 36. After generating the new training sample 36 by the above calculation, the output unit 116 outputs two or more training samples 31 and the new training sample 36 while maintaining their positional relationship in the feature space. In response to this, the label assignment unit 117 may be configured to, after outputting the two or more training samples 31 and the new training sample 36 by the output unit 116, accept a designation of a class to which the feature included in the new training sample 36 belongs (i.e., a new correct answer label 37), and assign a new correct answer label 37 configured to indicate the designated class to the new training sample 36.
[0096] In the example of FIG. 5E, three training samples 31 are displayed. In one example, all the training samples 31 used to generate the new training sample 36 may be output. However, if the number of training samples 31 used to generate the new training sample 36 is large, the output of the training samples 31 may occupy a large portion of the output resource (for example, the display area of the display), and the area for outputting the new training sample 36 may be limited. Therefore, in another example, the output unit 116 may select one training sample 31 that is closest to the new training sample 36 in the feature space for each different class from the training samples 31 used to generate the new training sample 36, and output only the selected training sample 31 for each class together with the new training sample 36. This can improve the efficiency of the output resource. As a result, when comparing the training sample 31 for each class with the new training sample 36, it is possible to make each training sample (31, 36) easier to see.
[0097] (Examples of each model configuration) In this embodiment, the encoder 5, the decoder 6, the classification model 7, and the estimator 600 are used as machine learning models (models in which parameter values are adjusted by machine learning). As long as the encoder 5, the decoder 6, the classification model 7, and the estimator 600 can execute their respective arithmetic processes, the types of machine learning models constituting each of them are not particularly limited and may be appropriately selected depending on the embodiment.
[0098] In one example, as shown in FIG. 5A and FIG. 5B, a neural network may be used for the machine learning model constituting each of the encoder 5, the decoder 6, the classification model 7, and the estimator 600. In the example of FIG. 5A and FIG. 5B, the encoder 5, the decoder 6, the classification model 7, and the estimator 600 include an input layer (51, 61, 71, 601), one or more intermediate (hidden) layers (52, 62, 72, 602), and an output layer (53, 63, 73, 603). The number of the intermediate layers (52, 62, 72, 602) may be appropriately selected depending on the embodiment. The number of the intermediate layers (52, 62, 72, 602) may be different between the encoder 5, the decoder 6, the classification model 7, and the estimator 600, or may be the same between at least some of them. The intermediate layers (52, 62, 72, 602) may be omitted. The number of the encoder 5, the decoder 6, the classification model 7, and the neural network that constitutes the estimator 600 may be appropriately determined depending on the embodiment.
[0099] Each layer (51-53, 61-63, 71-73, 601-603) includes one or more nodes (neurons). The number of nodes included in each layer (51-53, 61-63, 71-73, 601-603) is not particularly limited and may be appropriately selected according to the embodiment. Each node included in each layer (51-53, 61-63, 71-73, 601-603) may be connected to all nodes in the adjacent layer. As a result, the encoder 5, the decoder 6, the classification model 7, and the estimator 600 may be configured by a fully connected neural network. However, the connection relationship of each node is not limited to such an example and may be appropriately determined according to the embodiment. For example, each node may be connected to a specific node in an adjacent layer, or to a node in a layer other than the adjacent layer.
[0100] A weight (connection weight) is set for each connection in each layer (51-53, 61-63, 71-73, 601-603). A threshold is set for each node, and basically, the output of each node is determined depending on whether the sum of the products of each input and each weight exceeds the threshold. The threshold may be expressed by an activation function. In this case, the output of each node is determined by inputting the sum of the products of each input and each weight to the activation function and executing the calculation of the activation function. The type of activation function may be selected arbitrarily. The weight of the connection between each node included in each layer (51-53, 61-63, 71-73, 601-603) and the threshold of each node are examples of parameters used in the calculation processing of the encoder 5, the decoder 6, the classification model 7, and the estimator 600.
[0101] When each machine learning model (encoder 5, decoder 6, classification model 7, and estimator 600) is configured by a neural network, the computation process of each machine learning model is configured by forward propagation computation. That is, in the computation process of each machine learning model, data is input to each input layer (51, 61, 71, 601), and computation processes such as firing judgment of neurons included in each layer (51-53, 61-63, 71-73, 601-603) are executed in order from the input side. As a result of this forward propagation computation, the execution result of the task of each machine learning model is obtained from the output layer (53, 63, 73, 603). In this embodiment, the input data for the encoder 5 and the estimator 600 are samples, and the input data for the decoder 6 and classification model 7 are features. The task of the encoder 5 is to convert the samples into features. The task of the decoder 6 is to decode the samples from the features. The classification model 7 is to classify the classes of features included in the samples. The estimator 600 estimates the internal division ratio of the decoded samples. In each of the above machine learning processes, the parameter values of each machine learning model are adjusted by the error backpropagation method so that each error is reduced.
[0102] In this embodiment, the storage processing unit 1110 generates learning result data 125 for reproducing the trained encoder 5 and classification model 7 generated by machine learning. As long as the trained encoder 5 and classification model 7 can be reproduced, the configuration of the learning result data 125 is not particularly limited and may be appropriately determined according to the embodiment. In one example, the learning result data 125 may include information indicating the values of each parameter of the encoder 5 and classification model 7 obtained by adjusting the above machine learning. In some cases, the learning result data 125 may further include information indicating the structure of the encoder 5 and classification model 7. The structure may be specified by, for example, the number of layers from the input layer to the output layer in the neural network, the type of each layer, the number of nodes included in each layer, the connection relationship between nodes in adjacent layers, etc. When the structure of each machine learning model is common between devices, the information indicating the structure of the encoder 5 and classification model 7 may be omitted.
[0103] <Classification device> 6 illustrates an example of a software configuration of the classification device 2 according to the present embodiment. The control unit 21 of the classification device 2 loads the classification program 82 stored in the storage unit 22 into the RAM. Then, the control unit 21 interprets and executes instructions included in the classification program 82 loaded into the RAM by the CPU to control each component. As a result, as shown in FIG. 6, the classification device 2 according to the present embodiment operates as a computer including an acquisition unit 211, a classification unit 212, and an output unit 213 as software modules. That is, in the present embodiment, each software module of the classification device 2 is also realized by the control unit 21 (CPU) like the model generating device 1.
[0104] The acquisition unit 211 is configured to acquire a target sample 221. The classification unit 212 includes a trained classification model 7 generated by the model generation device 1 by holding the learning result data 125. The classification unit 212 is configured to classify a class to which a feature included in the acquired target sample 221 belongs by using the trained classification model 7. In this embodiment, the classification unit 212 includes a trained encoder 5 and classification model 7 by holding the learning result data 125. The classification unit 212 is configured to perform a classification task on the target sample 221 by using the trained encoder 5 and classification model 7. The output unit 213 is configured to output information related to a result of classifying the class (i.e., performing the classification task).
[0105] <Other> Each software module of the model generating device 1 and the classification device 2 will be described in detail in an operation example described later. In this embodiment, an example is described in which each software module of the model generating device 1 and the classification device 2 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors (e.g., graphics processing units). Each of the above modules may be realized as a hardware module. Furthermore, with regard to the software configurations of the model generating device 1 and the classification device 2, software modules may be omitted, replaced, or added as appropriate depending on the embodiment.
[0106] §3 Example of operation [Model generation device] 7A and 7B are flowcharts showing an example of a processing procedure related to machine learning and data generation by the model generation device 1 according to this embodiment. The processing procedure of the model generation device 1 described below is an example of a model generation method. However, the processing procedure of the model generation device 1 described below is merely an example, and each step may be changed as much as possible. Furthermore, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.
[0107] (Step S101) In step S101, the control unit 11 operates as the data acquisition unit 111 and acquires a plurality of first data sets 30 each of which is composed of a combination of the training samples 31 and the correct labels 32.
[0108] Each first data set 30 may be generated as appropriate. For example, assuming an environment in which a classification task is performed in real space or virtual space, each training sample 31 can be acquired by observing an arbitrary object in the environment. The observed object may be selected as appropriate according to the classification task. An arbitrary sensor may be used to acquire each training sample 31. As an example, when the classification task is to identify the type of an object appearing in an image, the training sample 31 can be acquired by photographing the object with a camera. Next, a result of performing the classification task on each acquired training sample 31, that is, information indicating the correct answer (true value) of the class to which the feature included in each training sample 31 belongs, is acquired as a correct answer label 32. In this situation, the classification task may be performed manually by an operator or the like. Then, the obtained correct answer label 32 is associated with the corresponding training sample 31. In this way, each first data set 30 can be generated.
[0109] Each of the first data sets 30 may be generated automatically by the operation of a computer, or may be generated manually by at least partially including an operation of an operator. Furthermore, each of the first data sets 30 may be generated by the model generating device 1, or may be generated by a computer other than the model generating device 1. When each of the first data sets 30 is generated by the model generating device 1, the control unit 11 acquires a plurality of first data sets 30 by executing the above-mentioned series of generation processes automatically or manually by an operation of an operator via the input device 15. On the other hand, when each of the first data sets 30 is generated by another computer, the control unit 11 acquires a plurality of first data sets 30 generated by the other computer, for example, via a network, a storage medium 91, or the like. Some of the first data sets 30 may be generated by the model generating device 1, and the other first data sets 30 may be generated by one or more other computers.
[0110] There may be any number of first data sets 30 to be acquired. When a plurality of first data sets 30 are acquired, the control unit 11 advances the process to the next step S102.
[0111] (Step S102) In step S102, the control unit 11 operates as a first training processing unit 112, and performs machine learning of the encoder 5 and the decoder 6 using a plurality of training samples 31 included in the acquired plurality of first data sets 30. For the machine learning of the encoder 5 and the decoder 6, at least some of the training samples 31 acquired in step S101 are used.
[0112] As an example of machine learning processing, first, the control unit 11 performs initial settings of a machine learning model to be processed by machine learning. In this embodiment, the control unit 11 performs initial settings of the encoder 5, the decoder 6, and the estimator 600. The structure and initial values of parameters of each machine learning model may be provided by a template or may be determined by input by an operator. Furthermore, when performing re-learning or additional learning, the control unit 11 may perform initial settings of each machine learning model based on learning result data obtained by past machine learning.
[0113] Next, the control unit 11 converts each training sample 31 into a feature by the encoder 5, and trains the encoder 5 and the decoder 6 so that a sample decoded by the decoder 6 from each obtained feature matches each training sample 31. In addition, in this embodiment, the control unit 11 trains the estimator 600 as adversarial learning so that a result of estimating an internal division ratio in a composite feature from a decoded sample generated by the decoder 6 by the estimator 600 matches a predetermined internal division ratio. Furthermore, the control unit 11 trains the encoder 5 and the decoder 6 so as to reduce the performance of the estimator 600 that estimates an internal division ratio from a generated decoded sample. For each training process, a stochastic gradient descent method, a mini-batch gradient descent method, or the like may be used.
[0114] As an example of the training process, first, the control unit 11 inputs each training sample 31 to the input layer 51 of the encoder 5, and executes the forward propagation arithmetic process of the encoder 5. Through the forward propagation arithmetic process of the encoder 5, the control unit 11 obtains an output corresponding to the result of converting each training sample 31 into a feature from the output layer 53. Next, the control unit 11 inputs each obtained feature to the input layer 61 of the decoder 6, and executes the forward propagation arithmetic process of the decoder 6. Through the forward propagation arithmetic process of the decoder 6, the control unit 11 obtains an output corresponding to the result of decoding a sample from each feature from the output layer 63. Next, the control unit 11 calculates an error (reconstruction error) between each obtained decoded sample and each corresponding training sample 31. The control unit 11 further calculates the gradient of the calculated error, and backpropagates the calculated gradient of the error by the error backpropagation method, thereby calculating the error of the parameter values of the encoder 5 and the decoder 6 in order from the output layer 63. Then, the control unit 11 updates the values of the parameters of the encoder 5 and the decoder 6 based on the calculated error of each parameter so as to reduce the sum of the calculated errors.
[0115] As the adversarial learning, the control unit 11 inputs each of the two training samples (31_1, 31_2) constituting each pair to the input layer 51 of the encoder 5, and executes the forward propagation arithmetic process of the encoder 5. Through this forward propagation arithmetic process, the control unit 11 acquires an output corresponding to the result of converting each of the two training samples (31_1, 31_2) constituting each pair into a feature from the output layer 53. It is preferable that each pair is composed of two training samples (31_1, 31_2) whose included features belong to different classes (i.e., whose true values indicated by the correct answer labels 32 are different). However, the configuration of the pairs is not limited to this example. Each pair may be composed of two training samples (31_1, 31_2) whose included features belong to the same class. The pairs may be created at the training sample 31 stage, or at the feature stage. The pairs may be created at any timing before the combination is performed by the internal division ratio.
[0116] The control unit 11 calculates a composite feature for each pair by combining two features acquired for each pair at a predetermined internal division ratio (α:1-α). The predetermined internal division ratio (α value) may be appropriately determined or changed for each pair / each training iteration. In one example, the predetermined internal division ratio may be determined randomly or according to a predetermined rule. In another example, the predetermined internal division ratio may be determined by an operator. For example, a plurality of internal division ratios may be designated by the operator, and one of the designated plurality of internal division ratios may be used as the predetermined internal division ratio. The control unit 11 inputs the composite feature to the input layer 61 of the decoder 6 and executes a forward propagation calculation process of the decoder 6. Through this forward propagation calculation process, the control unit 11 obtains an output from the output layer 63 corresponding to the result of decoding a sample from the composite feature of each pair.
[0117] The control unit 11 inputs the decoded samples generated for each pair to the input layer 601 of the estimator 600, and executes the forward propagation arithmetic process of the estimator 600. Through this forward propagation arithmetic process, the control unit 11 obtains an output corresponding to the result of estimating the internal division ratio from the decoded samples of each pair from the output layer 603. The control unit 11 calculates an error (estimated error) between the obtained estimation result of the internal division ratio and the corresponding true value of the predetermined internal division ratio (α / 1-α). The control unit 11 further calculates the gradient of the calculated error, and backpropagates the gradient of the calculated error by the error backpropagation method, thereby calculating the error of the parameter value of the estimator 600 from the output layer 603 in order. Then, the control unit 11 updates the parameter value of the estimator 600 based on the calculated parameter error so that the sum of the calculated errors becomes smaller.
[0118] Furthermore, the control unit 11 inputs the decoded samples generated for each pair to the input layer 601 of the estimator 600, and executes the forward propagation arithmetic process of the estimator 600. Through this forward propagation arithmetic process, the control unit 11 obtains an output corresponding to the result of estimating the interior division ratio from the decoded samples of each pair from the output layer 603. The control unit 11 calculates an error from the obtained estimation result of the interior division ratio so that the estimation accuracy of the estimator 600 decreases. In one example, the control unit 11 calculates an error between the obtained estimation result of the interior division ratio and a constant value (0 in FIG. 5B). The control unit 11 further calculates the gradient of the calculated error, and backpropagates the gradient of the error calculated via the estimator 600 to the encoder 5 and the decoder 6 by the error backpropagation method, thereby calculating the error of the values of the encoder 5 and the decoder 6 in order from the output layer 63 of the decoder 6. Then, the control unit 11 updates the values of each parameter of the encoder 5 and the decoder 6 based on the calculated error of each parameter so that the estimated result of the internal division ratio deviates from the true value (so that the sum of the calculated errors becomes smaller when the error between a constant value is calculated). In another example, the control unit 11 may calculate the error between the obtained estimated result of the internal division ratio and the corresponding true value of a predetermined internal division ratio. Then, the control unit 11 may update the values of each parameter of the encoder 5 and the decoder 6 by the error backpropagation method so that the sum of the calculated errors increases.
[0119] A loss function may be used to calculate the reconstruction error, the estimation error, and the error for reducing the performance of the estimator 600. The loss function may be set appropriately according to each task, the format of the correct answer, and the like. The degree to which the value of each parameter is updated may be adjusted by a learning rate. The learning rate may be given by an operator or may be given as a specified value in the program. The control unit 11 may repeat the adjustment of the value of each parameter by each series of the update processes until a predetermined condition is satisfied, for example, a specified number of times is executed, or the sum of the calculated errors is equal to or less than a threshold value.
[0120] By adjusting the parameters based on the reconstruction error, the encoder 5 and the decoder 6 are trained to acquire the ability to convert each training sample 31 into a feature by the encoder 5 and to decode a sample that matches each training sample 31 from each obtained feature by the decoder 6. By adjusting the parameters of the estimator 600 in the adversarial learning, the estimator 600 is trained to acquire the ability to estimate a predetermined internal division ratio from a generated decoded sample for each pair according to the capabilities of the encoder 5 and the decoder 6. Also, by adjusting the parameters of the encoder 5 and the decoder 6 in the adversarial learning, the encoder 5 and the decoder 6 are trained to acquire the ability to generate a decoded sample that reduces the performance of the estimator 600 in estimating the internal division ratio (i.e., the internal division ratio is difficult to estimate by the estimator 600) for each pair according to the capabilities of the estimator 600.
[0121] The training of the estimator 600 in the adversarial learning and the training of the encoder 5 and the decoder 6 may be alternately and repeatedly performed. In this case, the arithmetic process of obtaining an estimation result of the internal division ratio by the estimator 600 may be performed for each training. A series of processes of generating a decoded sample from two training samples 31 may be performed separately for each training, or may be performed commonly for each training. The series of processes of generating the decoded sample may be performed for each training iteration. In the training of the estimator 600, the parameter values of the encoder 5 and the decoder 6 may be fixed, and in the training of the encoder 5 and the decoder 6, the parameter values of the estimator 600 may be fixed.
[0122] Alternatively, a gradient inversion layer (not shown) may be provided between the decoder 6 and the estimator 600. The gradient inversion layer may be appropriately configured to pass values as is during forward propagation calculations and invert values during backward propagation. In this case, the training of the estimator 600 and the training of the encoder 5 and the decoder 6 may be performed at the same time. That is, during each training, the calculation process from the two training samples 31 to obtaining an estimation result of the internal division ratio may be performed in common. The series of processes for generating the decoded samples may be performed for each training iteration.
[0123] In addition, in one example, the adversarial learning training (FIG. 5B) may be performed simultaneously with the reconstruction error-based training (FIG. 5A). In this case, the computation process for calculating the feature amount of each pair may be performed in common with the computation process of the encoder 5 in the reconstruction error-based training. In another example, the adversarial learning training may be performed separately from the reconstruction error-based training.
[0124] Note that the machine learning of the encoder 5 and the decoder 6 does not need to be limited to training based on the above errors. In one example, in addition to the update process based on the above errors, the control unit 11 may calculate the error between the feature value obtained from the encoder 5 and a value obtained from a predetermined probability distribution (e.g., Gaussian distribution, etc.), and further adjust the parameter value of the encoder 5 so that the sum of the calculated errors becomes smaller. In this way, the control unit 11 may normalize the output value of the encoder 5.
[0125] As a result of the above machine learning, the control unit 11 can generate a trained encoder 5 and a trained decoder 6. When the machine learning process is completed, the control unit 11 proceeds to the next step S103.
[0126] (Step S103) In step S103, the control unit 11 operates as the second training processing unit 119, and performs machine learning of the classification model 7 using a plurality of first data sets 30. The machine learning in step S103 is an example of the above-mentioned first machine learning of the classification model 7. Note that the plurality of first data sets 30 used in the machine learning of the classification model 7 may not necessarily be identical to the plurality of first data sets 30 (training samples 31) used in the machine learning of the encoder 5 and the decoder 6. A portion of the first data sets 30 used in the machine learning of the classification model 7 may not be used in the machine learning of the encoder 5 and the decoder 6.
[0127] Except for differences in error calculation and adjustment target, the machine learning of the classification model 7 may be executed in the same manner as the machine learning of the encoder 5 and the decoder 6. As an example, the control unit 11 first performs initial setting of the classification model 7 to be processed by machine learning. The structure and initial values of the parameters of the classification model 7 may be provided by a template or may be determined by input from an operator. Furthermore, when performing re-learning or additional learning, the control unit 11 may perform initial setting of the classification model 7 based on learning result data obtained by past machine learning.
[0128] Next, the control unit 11 converts the training samples 31 for each first data set 30 into features using the encoder 5, and trains the classification model 7 so that the results of classifying the classes of features included in the training samples 31 from the obtained features by the classification model 7 match the true values indicated by the corresponding correct labels 32. For the training process, a stochastic gradient descent method, a mini-batch gradient descent method, or the like may be used.
[0129] As an example of the training process, first, the control unit 11 inputs the training samples 31 included in each first data set 30 to the input layer 51 of the encoder 5, and executes the forward propagation arithmetic process of the encoder 5. Through the forward propagation arithmetic process of the encoder 5, the control unit 11 obtains an output corresponding to the result of converting each training sample 31 into a feature from the output layer 53. Next, the control unit 11 inputs each obtained feature to the input layer 71 of the classification model 7, and executes the forward propagation arithmetic process of the classification model 7. Through the forward propagation arithmetic process of the classification model 7, the control unit 11 obtains an output corresponding to the result of classifying the class of the feature included in each training sample 31 from the output layer 73. Next, the control unit 11 calculates an error (classification error) between the obtained classification result and the true value indicated by the corresponding correct label 32. A loss function may be used to calculate the error. The loss function may be set appropriately. The control unit 11 further calculates the gradient of the calculated error, and back-propagates the calculated gradient of the error by the back-propagation method, thereby calculating the error of the parameter values of the classification model 7 in order from the output layer 73. Then, the control unit 11 updates the parameter values of the classification model 7 based on the calculated parameter errors. The degree to which the parameter values are updated may be adjusted by a learning rate. The learning rate may be specified by an operator, or may be provided as a default value in the program.
[0130] The control unit 11 adjusts the parameter values of the classification model 7 by the series of update processes so that the sum of errors calculated for each first data set 30 is reduced. The control unit 11 may repeat the adjustment of the parameter values by each series of update processes until a predetermined condition is met, such as, for example, performing the process a specified number of times or the sum of errors calculated is equal to or less than a threshold. As a result of this machine learning, a trained classification model 7 that has acquired the ability to classify the classes of features contained in the samples within the range of the first data set 30 used can be generated. That is, the control unit 11 can generate a provisional trained classification model 7. When the machine learning process is completed, the control unit 11 proceeds to the next step S104.
[0131] The machine learning process of the classification model 7 in step S103 may be executed simultaneously with the machine learning process of the encoder 5 and the decoder 6 in step S102. In this case, the process of converting each training sample 31 into a feature by the encoder 5 may be executed in common between the machine learning in step S102 and step S103. In addition, in the machine learning in step S103, the control unit 11 may back-propagate the calculated error gradient to the encoder 5 and calculate the error of the parameter value of the encoder 5. Then, the control unit 11 may also update the parameter value of the encoder 5 based on the calculated parameter error. Thereby, the control unit 11 may adjust the parameter value of the encoder 5 together with the classification model 7 based on the classification error.
[0132] (Step S104-Step S105) In step S104, the control unit 11 operates as the conversion unit 113, and converts each of two or more training samples 31 out of the multiple training samples 31 into a feature using the trained encoder 5 by the machine learning. In this embodiment, the control unit 11 inputs each of the two or more training samples 31 to the input layer 51 of the trained encoder 5, and executes a forward propagation arithmetic process of the trained encoder 5. As a result, the control unit 11 acquires two or more sample points 311 in the feature space, each corresponding to each training sample 31. When the two or more sample points 311 are acquired, the control unit 11 proceeds to the next step S105.
[0133] The two or more training samples 31 to be used may be appropriately selected from the multiple first data sets 30. It is preferable that at least some of the two or more training samples 31 are selected so that the classes to which the features contained therein belong are different. However, the method of selecting the training samples 31 to be used is not limited to this example. The two or more training samples 31 may be selected so that the classes to which the features contained therein belong are all the same.
[0134] In step S105, the control unit 11 operates as the point designation unit 114 and derives a linear sum 312 (312_1) of two or more acquired sample points 311 in the feature space, which is a linear sum 312 (312_1) that maximizes the sum of distances calculated from each sample point 311 according to a predetermined index. i ) so as to satisfy the following Equations 3 to 5.
[0135]
number
number
number
[0136] Furthermore, the predetermined index may be appropriately set to derive sample points of the linear sum 312 (312_1) that are appropriately separated from each sample point 311 and contribute to forming a state in which the sample points are evenly distributed on the feature space. In one example, the predetermined index may be configured by either of the above formula 1 and formula 2. When formula 1 is adopted as the predetermined index, the control unit 11 calculates the center of gravity of each sample point 311 as the linear sum 312_1. When formula 2 is adopted as the predetermined index, the control unit 11 calculates the Voronoi point of each sample point 311 as the linear sum 312_1. When the derivation of the linear sum 312 (312_1) is completed, the control unit 11 advances the process to the next step S106.
[0137] (Step S106-Step S108) In step S106, the control unit 11 operates as the point designation unit 114, and calculates, for each class, the posterior probability that a feature included in a sample decoded by the trained decoder 6 belongs from the feature amount of the derived linear sum 312_1, using the provisional trained classification model 7. In this embodiment, the control unit 11 inputs the derived linear sum 312_1 to the input layer 71 of the provisional trained classification model 7, and executes a forward propagation calculation process of the provisional trained classification model 7. By this forward propagation calculation process, the control unit 11 acquires an output corresponding to the result of calculating, for each class, the posterior probability that a feature included in a sample decoded by the trained decoder 6 belongs from the derived linear sum 312_1 (i.e., the classification result) from the output layer 73. When the calculation of the posterior probability is completed, the control unit 11 proceeds to the next step S107.
[0138] In step S107, the control unit 11 operates as the point designation unit 114 and calculates the entropy of the sample to be decoded from the calculated posterior probability for each class. As long as a value indicating the uncertainty of classification can be derived, the method of calculating the entropy is not particularly limited and may be appropriately determined according to the embodiment. In one example, the control unit 11 may calculate the entropy L according to the following Equation 6.
[0139]
number
[0140] In step S108, the control unit 11 operates as the point designation unit 114 and calculates the gradient (∂L / ∂α i The control unit 11 further calculates the gradient (∂L / ∂α i), the control unit 11 corrects the linear sum 312_1 derived so that the entropy L increases. The amount of correction may be appropriately determined depending on the embodiment. In one example, the correction amount of the linear sum 312_1 may be proportional to the value of the gradient of the entropy L and a correction rate. The correction rate may be given by the operator's designation or may be given as a default value in the program. In another example, the control unit 11 may correct the linear sum 312_1 so that the entropy L is maximized. As a result of the correction process, the control unit 11 can obtain the corrected linear sum 312_2. The control unit 11 may derive the linear sum 312_2 so that the relationship between the above formulas 4 and 5 holds even after the correction. When the corrected linear sum 312_2 is obtained, the control unit 11 proceeds to the next step S109.
[0141] (Step S109-Step S110) In step S109, the control unit 11 operates as the point designation unit 114, and extracts, from the two or more acquired sample points 311, two sample points (311_1, 311_2) adjacent to the derived linear sum 312 (312_2).
[0142] In one example, when at least some of two or more training samples 31 are selected such that the classes to which the features included therein belong are different, the control unit 11 may extract two sample points 311 to which the classes to which the features included therein belong are different as two sample points (311_1, 311_2) near the linear sum 312 (312_2). In another example, the control unit 11 may extract two sample points 311 to which the classes to which the features included in the corresponding training samples 31 belong are the same.
[0143] Furthermore, the two nearby sample points (311_1, 311_2) may be composed of two sample points 311 that are closest to the derived linear sum 312 (312_2). Alternatively, the two nearby sample points (311_1, 311_2) may be composed of any two sample points 311 that exist within a proximity range defined by a predetermined distance from the linear sum 312 (312_2). The predetermined distance may be set appropriately depending on the embodiment.
[0144] When the extraction of the two sample points (311_1, 311_2) is completed, the control unit 11 advances the process to the next step S110.
[0145] In step S110, the control unit 11 operates as the point designation unit 114, and acquires a projection point 313 corresponding to the linear sum 312 (312_2) by projecting the derived linear sum 312 (312_2) onto a straight line connecting the two extracted sample points (311_1, 311_2) in the feature space. The projection point 313 is the foot of a perpendicular line. That is, the projection point 313 can be calculated by extending a perpendicular line from the linear sum 312 (312_2) to the straight line connecting the two sample points (311_1, 311_2) and determining the intersection point between the perpendicular line and the straight line. A known method may be adopted as a method for calculating such a projection point 313. When the calculation of the projection point 313 is completed, the control unit 11 advances the process to the next step S111.
[0146] (Step S111) In step S111, the control unit 11 generates a decoded sample corresponding to the linear sum 312 (feature of the projection point 313) as a new training sample 36 by using the trained decoder 6. In this embodiment, the control unit 11 inputs the value (feature) of the projection point 313 to the input layer 61 of the trained decoder 6, and executes a forward propagation arithmetic process of the trained decoder 6. By this forward propagation arithmetic process, the control unit 11 obtains an output corresponding to the result of decoding the sample from the feature of the projection point 313 from the output layer 63. This makes it possible to generate the decoded sample as a new training sample 36. When the generation of the new training sample 36 is completed, the control unit 11 proceeds to the next step S112.
[0147] (Step S112) In step S112, the control unit 11 operates as the output unit 116, and outputs two or more training samples 31 and the new training sample 36 after arranging them according to their positional relationship in the feature space.
[0148] As long as the positional relationship of each sample (31, 36) can be presented, the output form and the output destination are not particularly limited and may be appropriately selected according to the embodiment. In one example, the output destination may be a display. This display may be at least one of the output device 16 and the output device of another computer. In this case, the control unit 11 may appropriately represent the feature space in a coordinate space in the display area of the display, arrange each sample (31, 36) by reflecting the positional relationship in the feature space in the coordinates, and draw each sample (31, 36) at the position of its coordinates. In another example, the output destination may be a print medium (e.g., a paper medium, etc.). In this case, the control unit 11 may output a print command to a printer (not shown) to arrange each sample (31, 36) by reflecting the positional relationship in the feature space in the coordinates, and draw each sample (31, 36) at the position of its coordinates.
[0149] In addition, the training sample 31 to be output may be appropriately selected. In one example, the control unit 11 may output all the training samples 31 used to generate the new training sample 36. In another example, the output of some of the training samples 31 used to generate the new training sample 36 may be omitted. For example, the control unit 11 may select one training sample 31 that is closest to the new training sample 36 in the feature space for each different class from the training samples 31 used to generate the new training sample 36. Then, the control unit 11 may output only the training sample 31 of each selected class together with the new training sample 36. According to this method, the training samples 31 to be output can be narrowed down, so that the efficiency of output resources can be improved. As a result, when comparing the training sample 31 of each class with the new training sample 36, each training sample (31, 36) can be easily viewed, which makes it easier to identify the class to which the feature included in the training sample 36 belongs.
[0150] When the output of each sample (31, 36) is completed, the control unit 11 advances the process to the next step S113.
[0151] (Step S113) In step S113, the control unit 11 operates as the label assignment unit 117 and generates a new correct label 37 configured to indicate a class to which a feature included in the generated new training sample 36 belongs. Then, the control unit 11 assigns the generated new correct label 37 to the new training sample 36. In this way, the control unit 11 generates a second dataset 35.
[0152] In this embodiment, the operator can visually check the new training sample 36 and the training sample 31 by the output of step S112, and identify the class to which the feature included in the new training sample 36 belongs. The control unit 11 may accept the designation of the class to which the feature included in the new training sample 36 belongs in response to the output of this step S112. A method for designating the class may be appropriately selected according to the embodiment. In one example, the class may be directly designated by the operator's operation via the input device 15. In another example, the class may be selected from a list. In yet another example, the class may be designated by selecting a training sample of the same class as the designated class from the output training samples 31. Then, the control unit 11 may generate a new correct answer label 37 configured to indicate the designated class, and assign (associate) the generated new correct answer label 37 to the new training sample 36. In this way, the second data set 35 can be generated.
[0153] In this embodiment, the control unit 11 can generate one or more second data sets 35 by one cycle of the processes of steps S104 to S113. The number of second data sets 35 generated in one cycle is not particularly limited and may be set appropriately depending on the embodiment. When the generation of one or more second data sets 35 is completed, the control unit 11 proceeds to the next step S114.
[0154] (Step S114) In step S114, the control unit 11 determines whether or not to repeat the series of processes from step S102 to step S113.
[0155] The repeat criteria may be set appropriately depending on the embodiment. As an example, whether or not to repeat the series of processes may be selected by the operator's specification. In this case, the control unit 11 may determine whether or not to repeat the series of processes depending on the operator's selection. As another example, the number of repetitions may be set. The number of repetitions may be given by any method, such as, for example, the operator's specification, a set value in a program, etc. In this case, the control unit 11 may determine whether or not to repeat the series of processes depending on whether or not the number of times the series of processes have been executed has reached a set number. As yet another example, the number of second data sets to be generated may be set. The set number of second data sets may be given by any method, such as, for example, the operator's specification, a set value in a program, etc. In this case, the control unit 11 may determine whether or not to repeat the series of processes depending on whether or not the total number of second data sets generated by the series of processes has reached a set number.
[0156] When it is determined that the series of processes is not to be repeated, the control unit 11 advances the process to the next step S115. On the other hand, when it is determined that the series of processes is to be repeated, the control unit 11 returns the process to step S102, and repeats the series of processes.
[0157] When repeating a series of processes, in step S102, the control unit 11 may update the trained encoder 5 and decoder 6 by performing machine learning of the encoder 5 and decoder 6 again using the generated one or more second datasets 35 (training samples 36). In step S103, the control unit 11 may update the provisionally trained classification model 7 by performing machine learning of the classification model 7 again using the generated one or more second datasets 35. Each re-machine learning (retraining) may be performed using only the generated one or more second datasets 35, or may be performed using at least a part of the multiple first datasets 30 together with the generated one or more second datasets 35.
[0158] In step S104, the control unit 11 may use the updated trained encoder 5 to obtain two or more sample points 311 corresponding to two or more training samples among the multiple training samples (31, 36) included in the multiple first data sets 30 and one or more second data sets 35. In step S105, the control unit 11 may derive a linear sum 312 (312_1) of the obtained two or more sample points 311. In step S106, the control unit 11 may use the updated provisional trained classification model 7 to calculate the posterior probability that the feature included in the sample decoded by the trained decoder 6 belongs to each class from the derived linear sum 312_1. The processes of steps S107 to S110 may be performed in the same manner as above. In step S111, the control unit 11 may use the updated trained decoder 6 to generate a decoded sample corresponding to the linear sum 312 (feature amount of the projection point 313) as a new training sample 36. In step S112, the control unit 11 may output two or more training samples selected from the multiple training samples (31, 36) and a new training sample 36 generated in this cycle after arranging them according to their positional relationship in the feature space. In step S113, the control unit 11 may generate a new second dataset 35 by assigning a new correct answer label 37 to the generated new training sample 36.
[0159] Note that the form of repetition does not need to be limited to this example. When repeating a series of processes, at least one of the retraining of the encoder 5 and the decoder 6 in step S102 and the retraining of the classification model 7 in step S103 may be omitted. When both retrainings are omitted, the control unit 11 may repeat the execution of the process from step S104. In this case, in the second and subsequent processes, the trained encoder 5, the trained decoder 6, and the provisionally trained classification model 7 generated in the first process may be used.
[0160] (Step S115-Step S116) In step S115, the control unit 11 operates as a thinning unit 118, and converts each training sample (31, 36) included in the multiple first data sets 30 and the one or more generated second data sets 35 into a feature using the trained encoder 5. The process of converting each training sample (31, 36) into a feature may be executed in the same manner as in step S104. In addition, the calculation result of step S104 may be reused for at least a part of the feature conversion result. Based on the conversion result, the control unit 11 calculates the density of sample points corresponding to each training sample (31, 36) in the feature space. When the calculation of the density is completed, the control unit 11 proceeds to the next step S116.
[0161] In step S116, the control unit 11 determines the branch destination of the process according to the calculation result of the density by the process of step S115. If the density exceeds the threshold, the control unit 11 advances the process to the next step S117. On the other hand, if the density is less than the threshold, the control unit 11 omits the process of step S117 and advances the process to the next step S118. If the density is equal to the threshold, the process may branch to any one of the following.
[0162] The method of calculating the density by the process of step S115 is not particularly limited as long as it is possible to evaluate the number of sample points included in an arbitrary range, and may be appropriately determined depending on the embodiment. In one example, in step S115, the control unit 11 may calculate the density by counting the number of sample points in a region of a predetermined size in the feature space that includes at least one sample point corresponding to the new training sample 36. In step S116, the control unit 11 may determine whether the number of counted sample points exceeds a threshold value.
[0163] In another example, in step S115, the control unit 11 may extract any two sample points in the feature space from the sample points obtained using the trained encoder 5, at least one of which corresponds to the new training sample 36. Then, the control unit 11 may calculate the distance D and cross entropy H between the two extracted sample points as density evaluation indexes by the following Equations 7 and 8.
[0164]
number
number
[0165] In this case, in step S116, the control unit 11 may determine that the density exceeds the threshold when the calculated distance D is less than the first threshold and the cross entropy H is less than the second threshold. On the other hand, the control unit 11 may determine that the density is less than the threshold when the calculated distance D exceeds the first threshold or the cross entropy H exceeds the second threshold. As described above, when each index is equal to each threshold, the process may branch to any one of the following. According to this evaluation method, the second data set 35 can be excluded in a dense region using the distance D and the cross entropy H as indexes.
[0166] (Step S117) In step S117, the control unit 11 operates as the thinning unit 118 and selects targets to be thinned out from targets whose density is determined to exceed the threshold. Here, the training samples 36 of the second data set 35 are pseudo samples generated by the trained decoder 6. Therefore, it is preferable that the targets to be thinned out are selected from the second data set 35. That is, it is preferable that the control unit 11 selects targets to be thinned out from one or more second data sets 35 generated within a range whose density is determined to exceed the threshold.
[0167] The second data set 35 to be thinned out may be selected arbitrarily. When the density is calculated within the region of the predetermined size, the control unit 11 may arbitrarily select a target to be thinned out from the second data set 35 present within the region. When the distance D between the two sample points and the cross entropy H are calculated as density evaluation indexes, the control unit 11 may select the second data set 35 of the two data sets corresponding to the two sample points as a target to be thinned out. When the two corresponding data sets are both second data sets 35, the control unit 11 may select one of the two second data sets 35 as a target to be thinned out.
[0168] However, the target to be thinned out does not have to be limited to the second data set 35. In this step S117, the control unit 11 may select the first data set 30 as the target to be thinned out. In this case, in the above step S115, the control unit 11 may calculate the density of the target that does not include the new training sample 36. Furthermore, the target to be thinned out may be appropriately selected so that the density is less than a threshold value. After selecting the target to be thinned out, the control unit 11 proceeds to the next step S118.
[0169] (Step S118) In step S118, the control unit 11 operates as a second training processing unit 119 and performs machine learning of the classification model 7 using the generated one or more second datasets 35. The machine learning in step S118 is an example of the above-mentioned second machine learning of the classification model 7.
[0170] The machine learning in step S118 may be performed using only the one or more finally generated second data sets 35, or may be performed using at least a part of the multiple first data sets 30 together with the one or more finally generated second data sets 35. In addition, the control unit 11 may execute retraining of the provisionally trained classification model 7 as the machine learning in step S118. Alternatively, the control unit 11 may newly train an initialized classification model 7 as the machine learning in step S118. When newly training an initialized classification model 7, the control unit 11 uses at least a part of the multiple first data sets 30 together with the one or more finally generated second data sets 35 for the machine learning in step S118.
[0171] When a target to be thinned out is selected by the process of step S117, the control unit 11 executes machine learning of the classification model 7, excluding the second data set 35 selected as the target to be thinned out. The machine learning process of step S118 may be the same as step S103 described above, except that the finally generated second data set 35 is included as learning data to be used for machine learning. As a result of this machine learning, a trained classification model 7 can be generated. When the generation of the trained classification model 7 is completed, the control unit 11 proceeds to the next step S119.
[0172] (Step S119) In step S119, the control unit 11 operates as a storage processing unit 1110, and generates information about the trained classification model 7 generated by machine learning as the learning result data 125. In this embodiment, the control unit 11 generates information about the trained encoder 5 generated in the processing of step S102 and the classification model 7 generated in the processing up to step S118 as the learning result data 125. Then, the control unit 11 stores the generated learning result data 125 in a predetermined storage area.
[0173] The predetermined storage area may be, for example, a RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD, a DVD, or the like, and the control unit 11 may store the learning result data 125 in the storage medium via the drive 17. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). In this case, the control unit 11 may use the communication interface 13 to store the learning result data 125 in the data server via a network. In addition, the external storage device may be, for example, an external storage device connected to the model generating device 1 via the external interface 14. Note that the information on the trained encoder 5 and the information on the trained classification model 7 may be stored in separate files. Each file may be stored in a different storage destination.
[0174] When the storage of the learning result data 125 is completed, the control unit 11 ends the processing procedure of the model generating device 1 according to this operation example.
[0175] The generated learning result data 125 may be provided to the classification device 2 at any timing. For example, the control unit 11 may transfer the learning result data 125 to the classification device 2 as part of the process of step S103 or separately from the process of step S103. The classification device 2 may acquire the learning result data 125 by receiving this transfer. Also, for example, the classification device 2 may acquire the learning result data 125 by accessing the model generation device 1 or a data server via a network using the communication interface 23. Also, for example, the classification device 2 may acquire the learning result data 125 via the storage medium 92. Also, for example, the learning result data 125 may be incorporated in advance into the classification device 2.
[0176] Furthermore, the control unit 11 may update or newly generate the learning result data 125 by periodically or irregularly repeating the processes of steps S101 to S119. During this repetition, at least a portion of the data sets (30, 35) used for machine learning may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may update the learning result data 125 held by the classification device 2 by providing the updated or newly generated learning result data 125 to the classification device 2 by any method.
[0177] [Classification device] FIG. 8 is a flowchart showing an example of a processing procedure for performing a classification task by the classification device 2 according to this embodiment. The processing procedure of the classification device 2 described below is an example of a classification method. However, the processing procedure of the classification device 2 described below is merely an example, and each step may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.
[0178] (Step S201) In step S201, the control unit 21 operates as the acquisition unit 211 and acquires the target sample 221. The target sample 221 is a sample of a predetermined type of data to be subjected to a classification task. A method for acquiring the target sample 221 may be appropriately determined according to the type of data. As an example, the target sample 221 may be generated by observing an object with a sensor (for example, a camera). In addition, a route for acquiring the target sample 221 may not be particularly limited and may be appropriately selected according to the embodiment. In one example, the control unit 21 may directly acquire the target sample 221 from a device such as a sensor. In another example, the control unit 21 may indirectly acquire the target sample 221 via another computer, a storage medium 92, or the like. Any pre-processing may be applied to the target sample 221. After acquiring the target sample 221, the control unit 21 advances the process to the next step S202.
[0179] (Step S202) In step S202, the control unit 21 operates as the classification unit 212. In this embodiment, the control unit 21 refers to the learning result data 125 to set the trained encoder 5 and the classification model 7. Then, the control unit 21 uses the trained encoder 5 and the classification model 7 to classify the class to which the feature included in the target sample 221 belongs. Specifically, the control unit 21 inputs the target sample 221 to the input layer 51 of the trained encoder 5, and executes a forward propagation arithmetic process of the trained encoder 5 and the classification model 7. As a result of this arithmetic process, the control unit 21 acquires an output corresponding to the result of classifying the class to which the feature included in the target sample 221 belongs from the output layer 73 of the trained classification model 7. The class classification result may be obtained appropriately according to the output format of the classification model 7. In this embodiment, the control unit 21 can obtain the class classification result for the target sample 221 by extracting the class with the highest calculated posterior probability. When the classification process is completed, the control unit 21 proceeds to the next step S203.
[0180] (Step S203) In step S203, the control unit 21 operates as the output unit 213 and outputs information related to the result of classifying the classes.
[0181] The output destination and the contents of the information to be output may be appropriately determined according to the embodiment. For example, the control unit 21 may output the classification result obtained in step S202 directly to the output device 26. The control unit 21 may also execute any information processing based on the obtained classification result. The control unit 21 may then output the result of the information processing as information related to the classification result. The output of the result of the information processing may include controlling the operation of the controlled device according to the classification result. The output destination may be, for example, the output device 26, an output device of another computer, the controlled device, etc.
[0182] When the output of information related to the classification result is completed, the control unit 21 ends the processing procedure of the classification device 2 according to this operation example. The control unit 21 may continuously and repeatedly execute the series of information processing from step S201 to step S203. The timing of the repetition may be appropriately determined according to the embodiment. In this way, the classification device 2 may be configured to continuously and repeatedly execute the classification task.
[0183] [Features] As described above, according to this embodiment, in the model generating device 1, the linear sum 312 having the maximum total distance from each sample point 311 is obtained as a new sample point by the process of step S105, so that a state in which sample points (sample points 311, linear sum 312) are evenly present in the feature space can be formed. Therefore, by using the trained decoder 6 to add a new data set (second data set 35) including a sample (training sample 36) decoded from the feature amount of the linear sum 312 to the learning data by the process of steps S111 and S113, learning data that can be evaluated in the feature space as being highly comprehensive can be obtained. Furthermore, according to this embodiment, by using the trained encoder 5 and decoder 6 by the series of processes of steps S104 to S113, at least a part of the process of generating such a new training sample (data set including the new training sample) can be automated. Therefore, it is possible to reduce the cost required for comprehensively collecting training samples. Furthermore, in this embodiment, in the processes of steps S103 and S118, in addition to the original dataset (first dataset 30), the new dataset (second dataset 35) generated as described above is used for machine learning of the classification model 7. In this way, by using the comprehensively obtained training samples (31, 36) for machine learning of the classification model 7, it is possible to improve the performance of the generated trained classification model 7.
[0184] In addition, in this embodiment, the calculation method of the above formula 1 or formula 2 may be adopted as a method for calculating the linear sum 312 (312_1) in the process of step S103. According to these calculation methods, it is possible to derive the linear sum 312 (312_1) in which the sum of the distances from each sample point 311 is maximum in the region inside each sample point 311. As a result, it is possible to obtain an appropriate linear sum 312 (312_1) and add sample points to a region in the feature space where sample points are sparse. In addition, by repeating this operation, it is possible to form a state in which sample points are evenly present. Therefore, it is possible to appropriately collect comprehensive training samples. In addition, in the processes of steps S103 and S118, it is possible to improve the performance of the trained classification model to be generated by using the comprehensively obtained training samples (31, 36) for machine learning.
[0185] In addition, in this embodiment, the provisional trained classification model 7 is generated by the process of step S103. The derived linear sum 312_1 is corrected by the process of steps S106 to S108 using the provisional trained classification model 7 so that the entropy increases. Then, by the process of step S111, a sample decoded from the feature amount derived from the corrected linear sum 312_2 is generated as a new training sample 36 by using the trained decoder 6. Since the entropy is calculated so as to indicate the uncertainty of whether classification is possible or not, a training sample with insufficient machine learning can be generated in the provisional trained classification model 7 by performing the correction based on the entropy as described above. Therefore, by using the obtained new training sample 36 in the process of step S118 (also in the process of step S103 when the training sample 36 is used during the repetition), the expectation of improvement in the performance of the generated trained classification model 7 can be increased.
[0186] In addition, in this embodiment, in the process of step S102, adversarial learning is performed between the encoder 5 and the decoder 6 and the estimator 600. By training the estimator 600, the ability to distinguish the internal division ratio is acquired according to the capabilities of the encoder 5 and the decoder 6 (i.e., the accuracy of distinguishing the internal division ratio is improved). On the other hand, by training the encoder 5 and the decoder 6, the ability to generate decoded samples whose internal division ratios are difficult to estimate is acquired according to the capabilities of the estimator 600. The more the ability of the estimator 600 to distinguish the internal division ratio is improved by the adversarial learning, the more the abilities of the encoder 5 and the decoder 6 to generate samples whose internal division ratios are difficult to estimate are improved. In other words, for points between each sample point, it is possible to suppress the decoding of samples that deviate from the true data whose internal division ratios are easy to identify. Therefore, according to this embodiment, it is possible to generate a trained encoder 5 and a decoder 6 that have acquired the ability to generate decoded samples similar to the true data (training sample 31) from the linear sum 312. This makes it possible to prevent the generation of training samples 36 that are extremely deviated from the true data in the process of step S111. As a result, by using the obtained new training samples 36 in the machine learning of the classification model 7, it is possible to increase the expectation of improving the performance of the generated trained classification model 7.
[0187] In addition, in this embodiment, by converting the linear sum 312 (312_2) into the internal division ratio (projection point 313) of two sample points (311_1, 311_2) in steps S109 and S110, the linear sum 312 (312_2) can be corrected to a value in the training range of the adversarial learning or in the vicinity thereof. This can increase the reliability of suppressing the addition of a training sample 36 that is extremely deviated from the true data to the learning data in the process of step S111. As a result, by using the obtained new training sample 36 in the machine learning of the classification model 7, the expectation of improving the performance of the generated trained classification model 7 can be further increased.
[0188] Furthermore, in this embodiment, in the process of step S109, two sample points 311 of different classes may be extracted as two sample points (311_1, 311_2). As a result, in the process of step S111, a decoded sample similar to true data can be generated as a new training sample 36 in a range closer to the classification boundary of the class than each sample point 311. As a result, by using the obtained new training sample 36 in the machine learning of the classification model 7, it is possible to further increase the expectation of improving the performance of the generated trained classification model 7.
[0189] In addition, in this embodiment, based on the output result obtained by the process of step S112, it is possible to confirm whether or not the new training sample 36 may be added to the learning data. This makes it possible to prevent a training sample 36 that is extremely deviated from the true data from being added to the learning data. Based on the output result, it is possible to confirm the class to which the features included in the new training sample 36 belong, depending on the positional relationship in the feature space with the training sample used for generation. Furthermore, in the process of step S113, by accepting the designation of the class while confirming the class of the new training sample 36 depending on the positional relationship in the feature space, it is possible to assign a new correct answer label 37 indicating an appropriate class to the new training sample 36. This makes it possible to prevent an incorrect correct answer label 37 from being assigned to the new training sample 36. As a result, the obtained new training sample 36 can be appropriately used in the machine learning of the classification model 7, and the expectation of improvement in the performance of the generated trained classification model 7 can be increased.
[0190] Furthermore, when there are an excess of training samples (31, 36) in a nearby range in the space, some of the training samples (31, 36) may not contribute much to improving the performance of the classification model 7. In this embodiment, the processes of steps S115 to S117 can find an excess state of the training samples (31, 36) in the feature space and thin out some of the excess training samples. This can reduce the number of data sets (30, 35) used for machine learning of the classification model 7, and can reduce the amount of calculation for machine learning while suppressing a decrease in the performance of the generated trained classification model 7, thereby improving the efficiency of the computational resources (processor, memory) required for machine learning.
[0191] In addition, in this embodiment, the processes of steps S102 to S113 may be repeatedly executed by the process of step S114. When a new training sample 36 is added and the machine learning of the encoder 5 and the decoder 6 is further performed using the added new training sample 36, the feature space mapped by the trained encoder 5 may be changed. According to this embodiment, by repeating the machine learning of the encoder 5 and the decoder 6 by step S102 and the generation of the new training sample 36 by steps S104 to S111, the encoder 5 can be updated while adding the new training sample 36, thereby forming a state in which sample points are evenly present. This makes it possible to more appropriately collect comprehensive training samples (31, 36). By using the comprehensively obtained training samples (31, 36) for the machine learning of the classification model 7, it is possible to increase the expectation of improving the performance of the generated trained classification model 7.
[0192] In this embodiment, by using the trained encoder 5 and classification model 7 generated as described above in the classification device 2, it is possible to expect the classification task to be performed with high accuracy.
[0193] § 4 Variations Although the embodiment of the present invention has been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. In the following, the same reference numerals are used for the same components as in the above embodiment, and the description of the same points as in the above embodiment is omitted as appropriate. The following modifications can be combined as appropriate.
[0194] <4.1> The classification system 100 according to the above embodiment may be applied to any situation where an arbitrary classification task is performed on a predetermined type of data. The classification task may be, for example, identifying the type of object shown in image data, determining whether a product shown in image data has a defect, classifying the state of an object shown in sensing data, etc. The data types of the training samples 31 and the target samples 221 may be appropriately selected according to the classification task. Below, examples of modified examples with limited application situations are given.
[0195] (A) Image recognition scene 9 is a schematic diagram illustrating an example of an application scene of the classification system 100A according to the first modification. This modification is an example in which the above embodiment is applied to a scene in which an object captured in an image sample is identified. The classification system 100A according to this modification is an example of the above classification system 100, and includes a model generation device 1 and a classification device 2A. As in the above embodiment, the model generation device 1 and the classification device 2A may be connected to each other via a network.
[0196] The predetermined type of data (training sample 31 and target sample 221A) handled in this modification is an image sample (image data) in which an object RA appears. The image sample may be obtained by photographing the object RA with a camera SA. The classification task is to identify the type of the object RA appearing in the image sample. Identifying the type of the object RA may include segmentation (extracting the range in which the object appears). The object RA may be a person or any object. The range in the image sample may be a specific part of the object RA (e.g., a face, etc.) or the entire object RA. When the object RA is a person, the identification target may be, for example, a part of the body such as a face. Identifying the type of a person may be, for example, estimating an individual or a body part (face, arm, leg, joint, etc.). The same applies to any object. Except for these limitations, the classification system 100A according to this modification may be configured in the same manner as the classification system 100 according to the above embodiment.
[0197] (Model Generator) In this modification, the model generating device 1 acquires a plurality of first data sets 30 each composed of a combination of a training sample 31 and a correct answer label 32 in the above step S101. The training sample 31 is composed of an image sample in which an object RA appears. The correct answer label 32 is configured to indicate the type (correct answer) of the object RA appearing in the training sample 31. The model generating device 1 performs machine learning of the encoder 5 and the decoder 6 using the plurality of training samples 31 by the process of the above step S102. The model generating device 1 performs machine learning of the classification model 7 using the plurality of first data sets 30 by the process of the above step S103.
[0198] The model generating device 1 generates one or more second data sets 35 each composed of a combination of a new training sample 36 and a new correct label 37 by the above-mentioned processing of steps S104 to S113. The new training sample 36 is composed of an image sample in which the object RA appears. The new correct label 37 is configured to indicate the type (correct answer) of the object RA appearing in the corresponding new training sample 36. The model generating device 1 may repeat the processing of steps S102 to S113 by the processing of step S114, as in the above-mentioned embodiment. The model generating device 1 may calculate the density of the sample points by the above-mentioned processing of steps S115 to S117, and select targets to be thinned out from the one or more second data sets 35 when the calculated density exceeds a threshold value.
[0199] The model generation device 1 performs machine learning of the classification model 7 using the one or more second datasets 35 generated by the process of step S118 above. A trained classification model 7 that has acquired the ability to identify the type of object RA captured in the image sample is generated by the processes up to step S118. In step S119 above, the model generation device 1 appropriately stores information related to the trained encoder 5 and the classification model 7 in a predetermined storage area as learning result data 125. The generated learning result data 125 may be provided to the classification device 2A at any timing.
[0200] (classification device) The classification device 2A is an example of the classification device 2. The hardware configuration and software configuration of the classification device 2A may be the same as those of the classification device 2 according to the above embodiment. In this modification, the classification device 2A may be connected to the camera SA via a communication interface or an external interface. Alternatively, the camera SA may be connected to another computer, and the classification device 2A may be configured to be able to acquire image samples from the camera SA by connecting to the other computer. The camera SA may be, for example, a general RGB camera, a depth camera, an infrared camera, or the like. The camera SA may be appropriately disposed in a location where it is possible to photograph the object RA.
[0201] The classification device 2A executes information processing on the classification task for the object RA by the same processing procedure as the above-mentioned classification device 2. That is, in step S201, the classification device 2A acquires the object sample 221A from the camera SA. The object sample 221A is composed of an image sample in which the object RA to be identified appears. In step S202, the classification device 2A uses the trained encoder 5 and the classification model 7 to identify the type of the object RA appearing in the object sample 221A. Then, in step S203, the classification device 2A outputs information on the result of identifying the type of the object RA. In one example, the classification device 2A may output the result of identifying the object RA to the output device as it is. In another example, the classification device 2A may execute any information processing according to the result of identifying the object RA. For example, the classification device 2A may execute information processing such as unlocking when the identified target person (object RA) is a specific individual.
[0202] (Features) According to the first modification, when attempting to generate a trained classification model 7 that has acquired image recognition capabilities, it is possible to reduce the cost of comprehensively collecting training samples. In addition, by using the comprehensively obtained training samples (31, 36) for machine learning of the classification model 7, it is possible to improve the image recognition performance of the generated trained classification model 7. In the classification device 2A, by using the generated trained classification model 7, it is expected that accurate image recognition will be achieved.
[0203] (B) Visual inspection scene 10 is a schematic diagram illustrating an example of an application scene of an inspection system 100B according to a second modified example. This modified example is an example in which the above embodiment is applied to a scene in which an appearance inspection of a product shown in an image sample is performed. The inspection system 100B according to this modified example is an example of the above classification system 100, and includes a model generating device 1 and an inspection device 2B. As in the above embodiment, the model generating device 1 and the inspection device 2B may be connected to each other via a network.
[0204] The predetermined type of data (training sample 31 and target sample 221B) handled in this modification is an image sample (image data) showing the product RB. The image sample may be obtained by photographing the product RB with the camera SB. The classification task is to determine whether or not the product RB shown in the image sample has a defect. The product RB is an example of the above-mentioned object RA, and determining whether or not the product RB has a defect is an example of identifying the type of the object RA. That is, in this modification, the class of features is related to the type of defect included in the product RB. Determining whether or not there is a defect may be configured by identifying the presence or absence of a defect, estimating the probability that the product RB contains a defect, identifying the type of defect included in the product RB (which may include a type indicating "no defect"), extracting the range of defects included in the product RB, or a combination of these. Except for these limitations, the inspection system 100B according to this modification may be configured similarly to the classification system 100 according to the above-mentioned embodiment.
[0205] The product RB may be, for example, a product conveyed on a manufacturing line of electronic devices, electronic components, automobile components, medicines, food, etc. The electronic components may be, for example, a circuit board, a chip capacitor, a liquid crystal, a relay winding, etc. The automobile components may be, for example, a connecting rod, a shaft, an engine block, a power window switch, a panel, etc. The medicines may be, for example, packaged tablets, unpackaged tablets, etc. The product RB may be a final product generated after the manufacturing process is completed, an intermediate product generated during the manufacturing process, or an initial product prepared before the manufacturing process. The defects may be, for example, scratches, stains, cracks, dents, burrs, color unevenness, foreign matter contamination, etc.
[0206] (Model Generator) In this modification, the model generating device 1 acquires a plurality of first data sets 30 each composed of a combination of a training sample 31 and a correct answer label 32 in the above step S101. The training sample 31 is composed of an image sample showing the product RB. The correct answer label 32 is configured to indicate whether or not the product RB shown in the training sample 31 has a defect (correct answer). The model generating device 1 performs machine learning of the encoder 5 and the decoder 6 using the plurality of training samples 31 by the process of the above step S102. The model generating device 1 performs machine learning of the classification model 7 using the plurality of first data sets 30 by the process of the above step S103.
[0207] The model generating device 1 generates one or more second data sets 35 each composed of a combination of a new training sample 36 and a new correct answer label 37 by the above-mentioned processing of steps S104 to S113. The new training sample 36 is composed of an image sample in which the product RB appears. The new correct answer label 37 is configured to indicate whether or not the product RB appearing in the corresponding new training sample 36 has a defect (correct answer). The model generating device 1 may repeat the processing of steps S102 to S113 by the processing of step S114, as in the above-mentioned embodiment. The model generating device 1 may calculate the density of the sample points by the above-mentioned processing of steps S115 to S117, and select targets to be thinned out from the one or more second data sets 35 when the calculated density exceeds a threshold value.
[0208] The model generating device 1 performs machine learning of the classification model 7 using the one or more second data sets 35 generated by the process of step S118 above. By the processes up to step S118, a trained classification model 7 that has acquired the ability to determine whether or not the product RB shown in the image sample has a defect is generated. In step S119 above, the model generating device 1 appropriately stores information about the trained encoder 5 and the classification model 7 in a predetermined storage area as learning result data 125. The generated learning result data 125 may be provided to the inspection device 2B at any timing.
[0209] (Inspection equipment) The inspection device 2B is an example of the above-mentioned classification device 2. The hardware configuration and software configuration of the inspection device 2B may be the same as those of the classification device 2 according to the above-mentioned embodiment. In this modification, the inspection device 2B may be connected to the camera SB via a communication interface or an external interface. Alternatively, the camera SB may be connected to another computer, and the inspection device 2B may be configured to be able to acquire image samples from the camera SB by connecting to the other computer. The camera SB may be, for example, a general RGB camera, a depth camera, an infrared camera, or the like. The camera SB may be appropriately arranged in a location where it is possible to photograph the product RB. For example, the camera SB may be arranged near a conveyor device that transports the product RB.
[0210] The inspection device 2B executes information processing on the appearance inspection of the product RB by the same processing procedure as the classification device 2. That is, in step S201, the inspection device 2B acquires a target sample 221B from the camera SB. The target sample 221B is composed of an image sample in which the product RB to be inspected is captured. In step S202, the inspection device 2B uses the trained encoder 5 and classification model 7 to determine whether or not the product RB captured in the target sample 221B has a defect. Then, in step S203, the inspection device 2B outputs information on the result of determining whether or not the product RB has a defect. In one example, the inspection device 2B may directly output the result of determining whether or not the product RB has a defect to the output device. In another example, when the inspection device 2B determines that the product RB contains a defect, it may output a warning to the output device to notify the user of the fact. The output destination of this information is not limited to the output device of the inspection device 2B, and may be an output device of another computer. In yet another example, when inspection device 2B is connected to a conveyor device that transports product RB, inspection device 2B may control the conveyor device so that defective products and non-defective products are transported on separate lines based on the judgment results.
[0211] (Features) According to the second concrete example, when attempting to generate a trained classification model 7 that has acquired visual inspection capabilities, it is possible to reduce the cost of comprehensively collecting training samples. In addition, by using the comprehensively obtained training samples (31, 36) for machine learning of the classification model 7, it is possible to improve the visual inspection performance of the generated trained classification model 7. By using the generated classification model 7, the inspection device 2B can be expected to perform visual inspection with high accuracy.
[0212] (C) State identification scene 11 illustrates a schematic example of an application scene of a classification system 100C according to a third modified example. This modified example is an example in which the above embodiment is applied to a scene in which the state of an object RC appearing in sensing data is to be classified. The classification system 100C according to this modified example is an example of the above classification system 100, and includes a model generating device 1 and a classification device 2C. As in the above embodiment, the model generating device 1 and the classification device 2C may be connected to each other via a network.
[0213] The predetermined types of data (training sample 31 and target sample 221C) handled in this modification are samples of sensing data (hereinafter, also simply referred to as "data samples") generated by observing an object RC with a sensor SC. The sensor SC may be, for example, a camera, a microphone, an encoder, an environmental sensor, a vital sensor, a medical testing device, an in-vehicle sensor, a home security sensor, or the like. The classification task may be classifying the state of the object RC appearing in the sensing data. The classification may include prediction (i.e., classifying a future state from the time when the sensing data is obtained). Except for these limitations, the identification system 100C according to this modification may be configured in the same manner as the classification system 100 according to the above embodiment.
[0214] The type of the sensor SC may be appropriately selected depending on the classification task. As an example, the target object RC may be a person, and classifying the state of the target object RC may be classifying the state of the target person. In this case, the sensor SC may be composed of at least one of a camera, a microphone, a vital sensor, and a medical examination device, for example. Classifying the state of the target person may be classifying the health state, such as the probability of developing a certain disease or the probability of a change in physical condition. Alternatively, the target person may be, for example, a driver of a vehicle, and classifying the state of the target person may be classifying the driver's state (for example, drowsiness level, fatigue level, level of ease, etc.).
[0215] As another example, the object RC may be an industrial machine, and classifying the state of the object RC may be classifying (detecting or predicting) whether or not there is an abnormality in the industrial machine. In this case, the sensor SC may be composed of at least one of a microphone, an encoder, and an environmental sensor, for example. The data sample may be composed of an encoder value, temperature, operating sound, etc. of a motor.
[0216] As another example, the object RC may be an object existing outside the vehicle, and classifying the state of the object RC may be classifying the situation outside the vehicle. In this case, the sensor SC may be composed of, for example, at least one of a camera and an on-board sensor. Classifying the situation outside the vehicle may be, for example, classifying the attributes of an object existing outside the vehicle, classifying a congestion situation, classifying the risk of an accident, etc. The object existing outside the vehicle may be, for example, a road, a traffic light, an obstacle (a person, an object), etc. Classifying the attributes of an object existing outside the vehicle may include, for example, classifying the occurrence of an event such as a person or a vehicle jumping out, a sudden start, a sudden stop, a lane change, etc.
[0217] As another example, the object RC may be an object present in a specific location, for example, outdoors or in a specific indoor location (for example, in a vinyl greenhouse), and classifying the state of the object RC may be classifying the situation of the specific location. In this case, the sensor SC may be composed of at least one of a camera, a microphone, and an environmental sensor, for example. As a specific example, the object RC may be a plant, and classifying the situation of the specific location may be classifying the cultivation situation of the plant.
[0218] As another example, the object RC may be, for example, an object present in a house, and classifying the state of the object RC may be classifying the situation in the house. In this case, the sensor SC may be, for example, at least one of a camera, a microphone, an environmental sensor, and a home security sensor.
[0219] (Model Generator) In this modification, the model generation device 1 acquires a plurality of first data sets 30, each of which is composed of a combination of a training sample 31 and a correct answer label 32, in the above step S101. The training sample 31 is composed of a data sample obtained by the sensor SC. The correct answer label 32 is configured to indicate a type (correct answer) of a state of the object RC appearing in the training sample 31. The model generation device 1 performs machine learning of the encoder 5 and the decoder 6 using the plurality of training samples 31 by the process of the above step S102. The model generation device 1 performs machine learning of the classification model 7 using the plurality of first data sets 30 by the process of the above step S103.
[0220] The model generating device 1 generates one or more second data sets 35 each composed of a combination of a new training sample 36 and a new correct label 37 by the above-mentioned processing of steps S104 to S113. The new training sample 36 is composed of a sample of sensing data. The new correct label 37 is configured to indicate a type (correct answer) of a state of the object RC appearing in the corresponding new training sample 36. The model generating device 1 may repeat the processing of steps S102 to S113 by the processing of step S114, as in the above-mentioned embodiment. The model generating device 1 may calculate the density of the sample points by the above-mentioned processing of steps S115 to S117, and select targets to be thinned out from the one or more second data sets 35 when the calculated density exceeds a threshold value.
[0221] The model generation device 1 performs machine learning of the classification model 7 using the one or more second data sets 35 generated by the process of step S118 above. By the processes up to step S118, a trained classification model 7 that has acquired the ability to identify the state of the object RC appearing in the sensing data is generated. In step S119 above, the model generation device 1 appropriately stores information related to the trained encoder 5 and the classification model 7 in a predetermined storage area as learning result data 125. The generated learning result data 125 may be provided to the identification device 2C at any timing.
[0222] (Identification device) The identification device 2C is an example of the classification device 2. The hardware configuration and software configuration of the identification device 2C may be similar to those of the classification device 2 according to the above embodiment. In this modification, the identification device 2C may be connected to the sensor SC via a communication interface or an external interface. Alternatively, the sensor SC may be connected to another computer, and the identification device 2C may be configured to be able to acquire a sample of sensing data from the sensor SC by connecting to the other computer.
[0223] The identification device 2C executes information processing related to state identification of the object RC by the same processing procedure as the above-mentioned classification device 2. That is, in step S201, the identification device 2C acquires an object sample 221C from the sensor SC. The object sample 221C is composed of a sample of sensing data generated by observing the object RC, the state of which is to be identified, by the sensor SC. In step S202, the identification device 2C classifies the state of the object RC appearing in the object sample 221C by using the trained encoder 5 and classification model 7.
[0224] Then, in step S203, the identification device 2C outputs information on the result of classifying the state of the object RC. In one example, the identification device 2C may output the result of classifying the state of the object RC to the output device as it is. In another example, the identification device 2C may execute any information processing according to the result of classifying the state of the object RC. As a specific example, when classifying the state of the object RC is to classify the health state of the target person, the identification device 2C may output a warning to notify the target person when it is determined that there is an abnormality in the health state of the target person (for example, the target person is classified into a class with a high probability of developing a predetermined disease). As another specific example, when classifying the state of the object RC is to classify the state of the driver, the identification device 2C may execute an output such as notifying a message encouraging the driver to take a break from driving or prohibiting switching from automatic driving to manual driving when the driver's drowsiness level or fatigue level is classified into a high class. As another specific example, when classifying the state of the object RC is to classify whether or not there is an abnormality in industrial machinery, when it is determined that there is an abnormality or a sign of an abnormality in the industrial machinery, the identification device 2C may output a warning to notify the fact.As another specific example, when classifying the state of the object RC is to classify the situation outside the vehicle, the identification device 2C may determine an operation command for the vehicle according to the classified situation outside the vehicle, and control the vehicle according to the determined operation command (for example, executing a temporary stop of the vehicle when it detects a person jumping out).
[0225] (Features) According to the third concrete example, when attempting to generate a trained classification model 7 that has acquired the ability to identify the state of the object RC appearing in sensing data, it is possible to reduce the cost of comprehensively collecting training samples. In addition, by using the comprehensively obtained training samples (31, 36) for machine learning of the classification model 7, it is possible to improve the state identification performance of the generated trained classification model 7. By using the generated classification model 7, the identification device 2C can be expected to perform accurate state identification.
[0226] <4.2> In the above embodiment, the model generation device 1 is configured to execute all of the processes of machine learning of the encoder 5 and the decoder 6, generation of new training data (second data set 35), and machine learning of the classification model 7. However, the device configuration for executing each process is not limited to this example. In another example, each process may be executed by one or more independent computers.
[0227] 10 is a schematic diagram showing an example of the configuration of a model generation device 1 according to this modification. In this modification, the model generation device 1 is composed of a first model generation device 101, a data generation device 102, and a second model generation device 103. The first model generation device 101 is one or more computers configured to execute a process related to machine learning of the encoder 5 and the decoder 6. The data generation device 102 is one or more computers configured to execute a process of generating new training samples 36. The second model generation device 103 is one or more computers configured to execute a process related to machine learning of the classification model 7.
[0228] The hardware configuration of each of the devices (101, 102, 103) may be similar to the hardware configuration of the model generating device 1. The devices (101, 102, 103) may be directly connected. Alternatively, the devices (101, 102, 103) may be connected via a network. In one example, data may be exchanged between the devices (101, 102, 103) via these connections. In another example, data may be exchanged between the devices (101, 102, 103) via a storage medium or the like.
[0229] The first model generation device 101 operates as a computer having a data acquisition unit 1111, a first training processing unit 112, and a storage processing unit 1112 as software modules by executing a part of the model generation program 81 related to machine learning of the encoder 5 and the decoder 6. The data generation device 102 operates as a computer having a data acquisition unit 1113, a conversion unit 113, a point designation unit 114, a sample generation unit 115, an output unit 116, a label assignment unit 117, a thinning unit 118, and a storage processing unit 1114 as software modules by executing a part of the model generation program 81 related to machine learning of the classification model 7. The second model generation device 103 operates as a computer having a data acquisition unit 1115, a second training processing unit 119, and a storage processing unit 1116 as software modules by executing a part of the model generation program 81 related to machine learning of the classification model 7. As a result, in this modification, the first model generation device 101 executes the process of step S102 of the process procedure of the model generation device 1. The data generating device 102 executes the processes of the above steps S104 to S117. The second model generating device 103 executes the processes of the above steps S103 and S118.
[0230] Specifically, the control unit of the first model generation device 101 operates as a data acquisition unit 1111 and acquires a plurality of training samples 31. The acquisition of the plurality of training samples 31 may be executed in the same manner as in step S101 above. Then, the control unit of the first model generation device 101 operates as a first training processing unit 112 and executes the process of step S102 above. As a result, a trained encoder 5 and a decoder 6 are generated. Then, the control unit of the first model generation device 101 operates as a storage processing unit 1112 and generates information on the trained encoder 5 and the decoder 6 as learning result data, and stores the generated learning result data in a predetermined storage area. As in step S119 above, the predetermined storage area may be appropriately selected. The generated learning result data may be provided to the data generation device 102 at any timing. Also, the information on the trained encoder 5 may be provided to the second model generation device 103 at any timing.
[0231] The control unit of the second model generation device 103 operates as a data acquisition unit 1115 and acquires a plurality of first data sets 30. The acquisition of the plurality of first data sets 30 may be executed in the same manner as in step S101 above. Then, the control unit of the second model generation device 103 operates as a second training processing unit 119 and executes the process of step S103 above. Thereby, a provisional trained classification model 7 is generated. Then, the control unit of the second model generation device 103 operates as a storage processing unit 1116 and generates information on the provisional trained classification model 7 as learning result data, and stores the generated learning result data in a predetermined storage area. As in step S119 above, the predetermined storage area may be appropriately selected. The generated learning result data of the provisional trained classification model 7 may be provided to the data generation device 102 at any timing.
[0232] The control unit of the data generating device 102 operates as a data acquiring unit 1113 and acquires two or more training samples 31. The control unit of the data generating device 102 operates as a conversion unit 113-thinning unit 118 and executes the processes of steps S104-S117 on the acquired two or more training samples 31. This generates one or more second data sets 35. The control unit of the data generating device 102 operates as a storage processing unit 1114 and stores the generated one or more second data sets 35 in a predetermined storage area. The processing procedure of the data generating device 102 is an example of a data generation method.
[0233] When the process is repeated from step S102 by the process of step S114, the data generating device 102 may provide the first model generating device 101 with one or more generated training samples 36 and instruct the data generating device 102 to update the trained encoder 5 and the decoder 6. The control unit of the first model generating device 101 may operate as a data acquiring unit 1111 and acquire one or more training samples 36 generated by the data generating device 102. The control unit of the first model generating device 101 may operate as a first training processing unit 112 and execute the process of step S102 using the one or more generated training samples 36. Thereby, the first model generating device 101 may generate the updated trained encoder 5 and the decoder 6. The control unit of the first model generating device 101 may operate as a storage processing unit 1112 and generate information on the updated trained encoder 5 and the decoder 6 as learning result data and store the generated learning result data in a predetermined storage area. The learning result data on the updated trained encoder 5 and the decoder 6 may be provided to the data generating device 102 at any timing. Furthermore, the updated information about the trained encoder 5 may be provided to the second model generation device 103 at any time.
[0234] Moreover, the data generating device 102 may provide the generated one or more second data sets 35 to the second model generating device 103 and instruct the second model generating device 103 to update the provisional trained classification model 7. The control unit of the second model generating device 103 may operate as a data acquiring unit 1115 and acquire the generated one or more second data sets 35. The control unit of the second model generating device 103 may operate as a second training processing unit 119 and execute the process of step S103 using the generated one or more second data sets 35. Thereby, the second model generating device 103 may generate an updated provisional trained classification model 7. The control unit of the second model generating device 103 may operate as a storage processing unit 1116 and generate information on the updated provisional trained classification model 7 as learning result data and store the generated learning result data in a predetermined storage area. The learning result data on the updated provisional trained classification model 7 may be provided to the data generating device 102 at any timing. The data generating device 102 may execute the processes in and after step S104 after acquiring the learning result data updated by any method from each device (101, 103). The two or more training samples 31 acquired when executing the process in step S104 may include a new training sample 36 generated up to the previous cycle.
[0235] Then, before executing the process of step S118, the control unit of the second model generation device 103 operates as a data acquisition unit 1115 and acquires one or more second data sets 35 generated by the data generation device 102. The control unit of the second model generation device 103 operates as a second training processing unit 119 and executes the process of step S118 using the acquired one or more second data sets 35. Thereby, the second model generation device 103 generates a trained classification model 7. The control unit of the second model generation device 103 operates as a storage processing unit 1116 and may generate information about the trained classification model 7 as learning result data and store the generated learning result data in a predetermined storage area. Information about the trained encoder 5 generated by the first model generation device 101 and learning result data about the trained classification model 7 generated by the second model generation device 103 may be provided to the classification device 2 at any timing.
[0236] According to this modification, the load of each calculation process can be distributed to each device (101, 102, 103). Note that the data generating device 102 may execute the process up to the generation of new training samples 36, and the assignment of new correct labels 37 may be executed by another computer. In this case, the label assignment unit 117 may be omitted from the software configuration of the data generating device 102. The process of step S113 may be omitted from the processing procedure of the data generating device 102.
[0237] <4.3> In the above embodiment, the classification model 7 is configured to receive input of features obtained by the encoder 5, and output a result of classifying the input features into classes to which the features belong. However, the configuration of the classification model 7 is not limited to this example. In another example, the calculation process of the classification task may be configured not to include the calculation of the encoder 5.
[0238] 13 is a schematic diagram showing an example of the configuration of the classification model 70 according to this modification. The classification model 70 is configured to receive input of samples and output a result of classifying the input samples into classes to which features belong. The type and structure of the machine learning model constituting the classification model 70 may be appropriately determined depending on the embodiment. In one example, a neural network may be used for the classification model 70, as in the above embodiment.
[0239] In this modification, as in the above embodiment, the machine learning of the classification model 70 may be divided into a first machine learning (step S103) and a second machine learning (step S118). In the machine learning, the control unit 11 inputs the training samples (31, 36) included in each data set (30, 35) to the classification model 70 and executes a forward propagation arithmetic process of the classification model 70. Through this forward propagation arithmetic process, the control unit 11 acquires an output corresponding to a result of classifying the class of the features included in each training sample 31 from the classification model 70. The control unit 11 calculates an error (classification error) between the obtained classification result and a true value indicated by the corresponding correct label (32, 37). Except for these points, in the above steps S103 and S118, the model generating device 1 can execute the machine learning of the classification model 70 by a method similar to that of the above embodiment. In the above step S119, the control unit 11 generates information on the trained classification model 70 generated by the machine learning as learning result data 125.
[0240] Also in this modification, the classification model 70 may be configured to output, for each class, the posterior probability to which the feature included in the input sample belongs as a result of classifying the feature included in the sample. In response to this, the linear sum 312 (312_1) may be corrected based on entropy. In this case, in the above step S106, the control unit 11 uses the trained decoder 6 to decode the sample from the feature amount of the derived linear sum 312_1. That is, the control unit 11 inputs the derived linear sum 312_1 to the trained decoder 6 and executes the forward propagation calculation process of the trained decoder 6. Through the forward propagation calculation process of the decoder 6, the control unit 11 generates a decoded sample from the linear sum 312_1. Next, the control unit 11 inputs the obtained decoded sample to the provisional trained classification model 70 and executes the forward propagation calculation process of the provisional trained classification model 70. Through the forward propagation arithmetic process of this classification model 70, the control unit 11 obtains from the provisional trained classification model 70 an output corresponding to the result of calculating, for each class, the posterior probability that a feature included in a sample decoded by the trained decoder 6 belongs from the derived linear sum 312_1 (i.e., the classification result). Except for these points, the model generation device 1 can calculate the entropy and correct the derived linear sum 312_1 based on the value of the calculated entropy, as in the above embodiment.
[0241] In this modification, information about the trained encoder 5 may be omitted from the learning result data 125. In the classification device 2, the control unit 21 sets the trained classification model 70 by referring to the learning result data 125 in step S202. The control unit 21 inputs the acquired target sample 221 to the trained classification model 70, and executes a forward propagation arithmetic process of the trained classification model 70. As a result of this arithmetic process, the control unit 21 can acquire, from the classification model 70, an output corresponding to a result of classifying a class to which a feature included in the target sample 221 belongs.
[0242] According to this modification, the configuration of the classification model 70 does not depend on the configuration of the encoder 5. Therefore, the structure of the classification model 70 can be optimized for the classification task. As a result, it is expected that the calculation amount of the classification model 70 can be made more efficient.
[0243] <4.4> In the above embodiment, each of the machine learning models of the encoder 5, the decoder 6, the classification model 7 (70), and the estimator 600 is configured by a fully connected neural network. However, the type of the neural network configuring each machine learning model may not be limited to such an example. In another example, at least one of the machine learning models may be configured by a convolutional neural network, a recurrent neural network, or the like. At least one of the neural networks configuring each machine learning model may include other types of layers, such as a convolutional layer, a pooling layer, a normalization layer, a dropout layer, or the like.
[0244] In the above embodiment, the type of machine learning model constituting each of the encoder 5, the decoder 6, the classification model 7 (70), and the estimator 600 may not be limited to a neural network. As long as the above calculation process can be executed, the type of machine learning model may not be particularly limited and may be appropriately selected depending on the embodiment. As another example, the encoder 5 and the decoder 6 may be configured by a principal component vector obtained by principal component analysis. The classification model 7 (70) may be configured by a support vector machine. The machine learning method may be appropriately determined depending on the type of machine learning model adopted.
[0245] <4.5> In the above embodiment, the correction of the linear sum 312_1 based on the entropy may be omitted. In this case, the above steps S106 to S108 may be omitted from the processing procedure of the model generating device 1 (data generating device 102). In step S109, the control unit 11 may extract two sample points (311_1, 311_2) near the derived linear sum 312_1. Then, in step S110, the control unit 11 may acquire a projection point 313 corresponding to the derived linear sum 312_1 by projecting the derived linear sum 312_1 onto a straight line connecting the two extracted sample points (311_1, 311_2) in the feature space. The machine learning in the above step S103 may be integrated into the machine learning in the above step S118. That is, the processing of step S103 may be omitted, and the control unit 11 may perform machine learning of the classification model 7 in step S118 using multiple first datasets 30 and the generated one or more second datasets 35.
[0246] <4.6> In the above embodiment, the acquisition of the projection points 313 in the above steps S109 and S110 may be omitted from the processing procedure of the model generating device 1 (data generating device 102). In response to this, the control unit 11 may acquire, as a new training sample 36, a sample decoded from the corrected linear sum 312_2 (in the case where the processes of steps S106 to S108 are further omitted, the linear sum 312_1) by using the trained decoder 6 in the process of step S111.
[0247] Furthermore, in the above embodiment, the estimator 600 is provided corresponding to the encoder 5 and the decoder 6, and adversarial learning is performed between the encoder 5 and the decoder 6 and the estimator 600. However, the machine learning configuration of the encoder 5 and the decoder 6 may not be limited to such an example. In the process of step S102, the adversarial learning may be omitted. In this case, the estimator 600 may be omitted.
[0248] <4.7> In the above embodiment, the process of step S113 of assigning a new correct label 37 to the new training sample 36 may be executed at any timing after the new training sample 36 is generated. The process of generating the new correct label 37 may be automated by appropriately estimating the class to which the features contained in the new training sample 36 belong. The class of the new training sample 36 may be estimated from the training sample 31 (31, 36) used to generate the new training sample 36. The estimation method may be selected appropriately. A provisional trained classification model 7 (70) may be used to estimate the class.
[0249] Furthermore, after the control unit 11 assigns a new correct label 37 to a new training sample 36 based on the class estimation result, the control unit 11 may accept, at any timing, a correction to the new correct label 37 assigned to the training sample 36. In one example, the control unit 11 may accept a correction to the new correct label 37 assigned to the training sample 36 after outputting the training sample 36 in the above step S112. The process of accepting this correction is an example of the process of accepting the class designation in the above step S113.
[0250] In the above embodiment, the process of step S112 may be omitted from the process procedure of the model generating device 1 (data generating device 102). Accordingly, the output unit 116 may be omitted from the software configuration of the model generating device 1 (data generating device 102).
[0251] <4.8> In the above embodiment, the processes in steps S106 to S108 correct the derived linear sum 312_1 based on the entropy, and decode the sample from the corrected linear sum 312_2 (or its projection point 313) to generate a new training sample 36 that contributes to improving the performance of the classification model 7 (70). The index for evaluating whether or not the index contributes to such performance improvement is not limited to the entropy, and may be appropriately designed depending on the embodiment.
[0252] Another example of the evaluation index is reliability, which indicates the degree to which a data belongs to the distribution of a plurality of training samples 31 (i.e., the degree to which the data is close to the true data derived from the training samples 31). The model generating device 1 may correct the derived linear sum 312_1 based on reliability in addition to or instead of the entropy. The reliability may be calculated as appropriate. Two methods for calculating the reliability are exemplified below.
[0253] (I) First Method The first method uses a classifier for reliability evaluation. In this case, the first training processing unit 112 may be configured to further perform machine learning of the classifier using a plurality of training samples 31 included in a plurality of first data sets 30. The machine learning of the classifier may be configured by training the classifier to evaluate the degree to which an input sample belongs to the distribution of the plurality of training samples 31. Such training of the classifier may be appropriately selected depending on the embodiment.
[0254] FIG. 14 is a schematic diagram showing an example of a machine learning process of the classifier 650 according to this modification. In the example of FIG. 14, training the classifier 650 is configured by performing adversarial learning between the encoder 5 and the decoder 6 and the classifier 650. The adversarial learning includes a step of training the classifier 650 to identify the origin of the input sample by using a plurality of samples decoded by the decoder 6 and a plurality of training samples 31 as input samples, respectively, and a step of training the encoder 5 and the decoder 6 so that the discrimination performance of the classifier 650 decreases when a sample decoded by the decoder 6 is input to the classifier 650. This adversarial learning is an example of training the classifier to acquire the ability to evaluate the degree to which the input sample belongs to the distribution of the plurality of training samples 31. Note that, in FIG. 14, for convenience of explanation, the origin of the training sample 31 is expressed as "true" and the origin of the decoded sample is expressed as "false". The method of expressing the origin of each sample may be changed as appropriate.
[0255] As an example of the training process, the control unit 11 inputs each training sample 31 to the encoder 5 and executes the arithmetic process of the encoder 5 and the decoder 6. Through this arithmetic process, the control unit 11 generates a decoded sample corresponding to each training sample 31. The control unit 11 inputs each sample (training sample 31 / decoded sample) to the discriminator 650 and executes the arithmetic of the discriminator 650, thereby making it possible to obtain a result of discriminating the origin of each sample from the discriminator 650. The control unit 11 adjusts the parameter values of the discriminator 650 so that the error between this discrimination result and the true value of discrimination (true / false) becomes small. This makes it possible to train the discriminator 650 so as to acquire the ability to discriminate the origin of the input sample.
[0256] Furthermore, the control unit 11 inputs each training sample 31 to the encoder 5 and executes the arithmetic processing of the encoder 5 and the decoder 6. Through this arithmetic processing, the control unit 11 generates a decoded sample corresponding to each training sample 31. The control unit 11 inputs each obtained decoded sample to the discriminator 650 and executes the arithmetic processing of the discriminator 650, thereby obtaining a result of discriminating the origin of each decoded sample from the discriminator 650. The control unit 11 calculates an error so that this discrimination result is erroneous (i.e., the more the input sample is mistaken for the origin of the training sample 31, the smaller the error becomes), and adjusts the parameter values of the encoder 5 and the decoder 6 so that the calculated error becomes smaller. This makes it possible to train the encoder 5 and the decoder 6 so as to acquire the ability to generate a decoded sample that reduces the discrimination performance of the discriminator 650.
[0257] The type of the machine learning model constituting the discriminator 650 is not particularly limited as long as it is capable of performing an operation to discriminate the origin of the input sample, and may be appropriately selected according to the embodiment. In the example of FIG. 14, the discriminator 650 is constituted by a fully connected neural network, and includes an input layer 651, one or more intermediate layers 652, and an output layer 653. Each layer 651-653 may be configured similarly to each layer 51-53 of the encoder 5. In addition, the number of layers constituting the discriminator 650, the number of nodes of each layer, the connection relationship, and other structures may be appropriately determined according to the embodiment. In training the discriminator 650, similar to the encoder 5, the control unit 11 can adjust the value of each parameter by backpropagating the gradient of the calculated error by the error backpropagation method.
[0258] Similar to the adversarial learning with the estimator 600, a gradient inversion layer (not shown) may be provided between the decoder 6 and the discriminator 650. In this case, the training of the discriminator 650 in the adversarial learning and the training of the encoder 5 and the decoder 6 may be performed at once. Alternatively, the training of the discriminator 650 in the adversarial learning and the training of the encoder 5 and the decoder 6 may be alternately and repeatedly performed. In the training of the discriminator 650, the parameter values of the encoder 5 and the decoder 6 may be fixed, and in the training of the encoder 5 and the decoder 6, the parameter values of the discriminator 650 may be fixed. Even in this case, at least a part of each training process (e.g., a process for generating a decoded sample) may be commonly performed.
[0259] The type of neural network constituting the classifier 650 may not be limited to a fully connected neural network. As with the above-mentioned encoder 5, the classifier 650 may be composed of a neural network other than a fully connected type. The classifier 650 may include other types of layers. The type of machine learning model constituting the classifier 650 may not be limited to a neural network. As long as it is possible to execute an operation for identifying the origin of an input sample, the type of machine learning model constituting the classifier 650 may not be particularly limited and may be appropriately selected according to the embodiment. The machine learning method may be appropriately determined according to the type of machine learning model employed.
[0260] The discriminator 650 trained by adversarial learning acquires the ability to distinguish between the training sample 31 and a decoded sample (pseudo sample) derived from the decoder 6, depending on the degree of the adversarial learning. Thus, according to the trained discriminator 650, the reliability of the input sample can be evaluated according to the result of discriminating the origin of the input sample. That is, the more the discriminator 650 errs in determining that the input sample originates from the training sample 31 for the decoded sample, the more indistinguishable the decoded sample is from the true sample (training sample 31), and the higher the reliability of the decoded sample can be evaluated. On the other hand, the more the discriminator 650 correctly discriminates that the input sample originates from the decoder 6, the more easily the decoded sample can be distinguished from the true sample (training sample 31) (i.e., it deviates from the training sample 31), and the lower the reliability of the decoded sample can be evaluated. Thus, according to the trained discriminator 650 generated by adversarial learning, the reliability of the sample decoded by the trained decoder 6 can be appropriately evaluated.
[0261] In addition, the method of training the classifier 650 to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of a plurality of training samples 31 is not limited to the method by adversarial learning, and may be appropriately selected according to the embodiment. In another example, the trained classifier 650 may be configured by a principal component vector obtained by principal component analysis. In this case, training the classifier 650 to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of the training samples 31 may be configured by performing principal component analysis on the plurality of training samples 31. In order to evaluate the reliability of the decoded sample, the control unit 11 may use the principal component vector to project (i.e., compress) the decoded sample into a subspace, and then restore the decoded sample to the original space. The larger the difference between the original decoded sample before compression and the decoded sample after restoration, the more the decoded sample deviates from the distribution of the training samples 31, and the reliability of the decoded sample can be evaluated as low. On the other hand, the smaller the difference between the original decoded sample before compression and the decoded sample after restoration, the more closely the decoded sample follows the distribution of the training samples 31, and the higher the reliability of the decoded sample can be evaluated.
[0262] In yet another example, the classifier 650 may be configured by a one-class classifier (e.g., one-class support vector machine). In this case, training the classifier 650 to acquire the ability to evaluate the degree to which the input sample belongs to the distribution of the training samples 31 may be configured by, for example, solving an optimization problem of one-class classification using a plurality of training samples 31. The control unit 11 may evaluate the reliability of the decoded sample using the trained one-class classifier. The one-class classifier can identify whether the decoded sample belongs to the distribution of the training samples 31. In other words, it can be evaluated that the reliability of the decoded sample is high if the decoded sample is identified as belonging to the distribution of the training samples 31, and that the reliability of the decoded sample is low if the decoded sample is identified as not belonging to the distribution of the training samples 31.
[0263] The control unit 11 may operate as the first training processing unit 112 and perform machine learning of the classifier 650 at any timing before correcting the linear sum 312_1 by the process of step S108. When the machine learning of the classifier 650 is configured by the above-mentioned adversarial learning, the control unit 11 may perform machine learning of the classifier 650 together with other training in step S102. When the process is repeated from step S102 by the process of step S114, the control unit 11 may also perform machine learning of the classifier 650 again. In this case, the control unit 11 may further use one or more new training samples 36 obtained up to the previous cycle for the machine learning of the classifier 650. In the first method of this modification, the control unit 11 evaluates the reliability of the sample decoded from the linear sum 312 using the trained classifier 650 generated in this way.
[0264] FIG. 15 shows an example of a data generation process according to this modification. Before executing the process of step S108, the control unit 11 operates as the point designation unit 114 and uses the trained classifier 650 to evaluate the degree to which a sample decoded by the trained decoder 6 from the feature amount of the derived linear sum 312_1 belongs to the distribution of multiple training samples 31. When the trained classifier 650 is generated by the above-mentioned adversarial learning, evaluating the degree to which the decoded sample belongs to the distribution of multiple training samples 31 is configured by identifying the origin of the decoded sample using the trained classifier 650. That is, the control unit 11 uses the trained decoder 6 to generate a decoded sample from the derived linear sum 312_1. The control unit 11 uses the trained classifier 650 to identify the origin of the obtained decoded sample. The control unit 11 evaluates that the reliability is high when the discriminator 650 errs in determining that the input sample originates from the training sample 31 for the decoded sample, and that the reliability is low when the discriminator 650 correctly determines that the input sample originates from the decoder 6 for the decoded sample.
[0265] Next, the control unit 11 operates as the point designation unit 114, and calculates the reliability of the sample decoded by the trained decoder 6 based on the result of evaluating the degree to which the decoded sample belongs to the distribution of the training samples 31. The reliability calculation method may be appropriately determined according to the embodiment so that the higher the degree to which the decoded sample is evaluated to belong to the distribution of the training samples 31, the higher the reliability value will be, and the lower the degree to which the decoded sample is evaluated to belong to the distribution of the training samples 31, the lower the reliability value will be. In one example, the control unit 11 may calculate the evaluation value of the reliability R by the following Equation 9.
[0266]
number
[0267] Then, in step S108, the control unit 11 operates as the point designation unit 114 and calculates the gradient of the calculated reliability (∂R / ∂α i ) to correct the derived linear sum 312_1 so as to improve reliability. In this way, the control unit 11 obtains a corrected linear sum 312_5. As an example of the correction method, the control unit 11 may calculate a product S of each evaluation value by the following formula 10.
[0268]
number
[0269] 16 is a diagram for explaining an example of an optimization method for obtaining a corrected linear sum 312_5 in this modified example. As shown in FIG. 16, when the values of the distance (X), the entropy L, and the reliability R are obtained, the control unit 11 calculates the gradient (∂S / ∂α i ) to derive the maximum value 315 of the product S. The control unit 11 may obtain a point corresponding to the maximum value 315 as the corrected linear sum 312_5. As shown in FIG. 16, when there are multiple maximum values (two in the case of FIG. 16), the control unit 11 may adopt any of the maximum values as the corrected linear sum 312_5. When the correction based on the entropy L is omitted, the control unit 11 may obtain the corrected linear sum 312_5 based on the distance (X) and the reliability R.
[0270] After obtaining the corrected linear sum 312_5, the control unit 11 may obtain the projection point 313 corresponding to the corrected linear sum 312_5 by the processing of steps S109 to S110. Thereby, in the processing of step S111, the control unit 11 may generate a decoded sample corresponding to the feature of the projection point 323 of the corrected linear sum 312_5 as a new training sample 36 by using the trained decoder 6. On the other hand, when the processing of steps S109 to S110 is omitted, the control unit 11 may generate a decoded sample corresponding to the feature of the corrected linear sum 312_5 as a new training sample 36 by using the trained decoder 6 in the processing of step S111.
[0271] (II) Second Method 15 is a schematic diagram showing an example of a method for evaluating reliability R and a data generation process according to the second method of this modified example. The second method uses a trained encoder 5 and decoder 6 to evaluate reliability R. In the second method, before executing the process of step S108, the control unit 11 operates as a point designation unit 114 and evaluates the degree of difference between a sample decoded by the trained decoder 6 from the derived feature amount of the linear sum 312_1 and the training sample 31 included in the first datasets 30.
[0272] In this modification, the control unit 11 decodes the samples from the derived linear sum 312_1 using the trained decoder 6. Evaluating the degree of difference is configured by converting the samples to be decoded into features by the trained encoder 5, and calculating the difference between the obtained features of the decoded samples and the features of the linear sum 321_1. Alternatively, evaluating the degree of difference is configured by further using the trained decoder 6 to generate re-decoded samples from the features of the samples to be decoded, and calculating the reconstruction error between the samples to be decoded and the generated re-decoded samples.
[0273] The trained encoder 5 and decoder 6 are generated by machine learning using the training sample 31. Therefore, the more similar the sample decoded from the derived linear sum 312_1 is to the training sample 31, the smaller the above-mentioned differences (difference between features / reconstruction error) become, and the more deviated the decoded sample is from the training sample 31, the larger the above-mentioned differences become. Therefore, the above-mentioned method can evaluate the degree of difference between the sample decoded from the linear sum 312_1 and the training sample 31.
[0274] Based on the result of evaluating the degree of difference, the control unit 11 calculates the reliability R of the sample decoded by the trained decoder 6. The method of calculating the reliability R may be appropriately determined according to the embodiment so that the greater the degree of difference is evaluated to be, the lower the value of the reliability R is, and the smaller the degree of difference is evaluated to be, the higher the value of the reliability R is. In one example, the control unit 11 may calculate the evaluation value of the reliability R by the following formula 11.
[0275]
number
[0276] Then, in step S108, the control unit 11 operates as the point designation unit 114 and calculates the gradient of the calculated reliability (∂R / ∂α i ) to correct the derived linear sum 312_1 so as to improve reliability. Thus, the control unit 11 obtains a corrected linear sum 312_5. The correction method may be the same as the above-mentioned first method. As in the above-mentioned first method, the control unit 11 may obtain the corrected linear sum 312_5 based on the entropy L and the reliability R. In another example, the entropy L may be omitted, and the control unit 11 may obtain the corrected linear sum 312_5 based on the reliability R.
[0277] After obtaining the corrected linear sum 312_5, the control unit 11 may obtain the projection point 313 corresponding to the corrected linear sum 312_5 by the processing of steps S109 to S110. As a result, in the processing of step S111, the control unit 11 may generate a decoded sample corresponding to the feature of the projection point 323 of the corrected linear sum 312_5 as a new training sample 36 by using the trained decoder 6. On the other hand, when the processing of steps S109 to S110 is omitted, the control unit 11 may generate a decoded sample corresponding to the feature of the corrected linear sum 312_5 as a new training sample 36 by using the trained decoder 6 in the processing of step S111. A sample that can be appropriately encoded and decoded by the trained encoder 5 and decoder 6 is basically included in the distribution of the training sample, and the degree to which it is included in this distribution can be evaluated according to the reliability R of the above formula 11. Therefore, by using this reliability R as an index, a new training sample 36 can be generated within a desired distribution.
[0278] (summary) In the feature space, the sample decoded by the trained decoder 6 from the feature quantities of points other than the points corresponding to the training sample 31 is, so to speak, pseudo data imitating true data (training sample). In some cases, this pseudo data may be extremely different from the true data. According to the first method or the second method, the reliability of the decoded sample obtained from the derived linear sum 312_1 can be evaluated, and the linear sum 312_1 can be corrected so as to improve the reliability. This makes it possible to suppress the addition of a new training sample 36 that is extremely different from the true data to the learning data. As a result, by using the obtained new training sample 36 for machine learning of the classification model 7 (70), it is possible to increase the expectation of improving the performance of the generated trained classification model 7 (70). The model generating device 1 may simultaneously employ the first method and the second method. That is, the control unit 11 may calculate the reliability R by the evaluation by the discriminator 650 and the evaluation by each of the differences. This makes it possible to further prevent new training samples 36 that are significantly different from the true data from being added to the learning data.
[0279] <4.9> In the above embodiment, the above formula 1 and formula 2 are given as an example of the predetermined index for deriving the linear sum 312_1. However, the predetermined index may not be limited to these. In another example, when the classification model 7 is configured to output the posterior probability to which the feature included in the sample belongs for each class, the control unit 11 may use the provisional trained classification model 7 to derive the point X that maximizes the minimum value of the cross entropy H calculated by the following formula 12 as the linear sum 312 (312_1).
[0280]
number
[0281] In the above embodiment (variation), the processes of steps S115 to S117 may be omitted from the processing procedure of the model generating device 1 (data generating device 102). In this case, the thinning unit 118 may be omitted from the software configuration of the model generating device 1 (data generating device 102).
[0282] Furthermore, in the above embodiment, the model generation device 1 may store the generated second data set 35 in a predetermined storage area. The predetermined storage area may be appropriately selected depending on the embodiment. [Explanation of symbols]
[0283] 1...Model generation device, 11: control unit, 12: storage unit, 13: communication interface, 14...External interface, 15...input device, 16...output device, 17...drive, 81...model generation program, 91...storage medium, 111...data acquisition unit, 112...first training processing unit, 113: conversion unit; 114: point designation unit; 115... sample generation unit, 116... output unit, 117: labeling unit; 118: thinning unit; 119... second training processing unit, 1110... storage processing unit, 125...Learning outcome data, 2...classification device, 21: control unit, 22: storage unit, 23: communication interface, 24…External interface, 25...input device, 26...output device, 27...drive, 82... Classification program, 92... Storage medium, 211...acquisition section, 212...classification section, 213...output section, 221…Target sample, 30…First data set, 31...training sample, 32...correct label, 311...sample points, 312...linear sum, 313...projection points, 35…Second data set, 36...training sample, 37...correct label, 5…Encoder, 51...input layer, 52...intermediate (hidden) layer, 53...output layer, 6…Decoder, 61...input layer, 62...intermediate (hidden) layer, 63...output layer, 600...Estimator, 7…Classification model, 71...input layer, 72...intermediate (hidden) layer, 73...output layer
Claims
1. A data acquisition unit configured to acquire a plurality of first data sets each composed of a combination of a training sample and a ground truth label, The training samples include features, The ground truth label is configured to indicate a class to which the feature contained in the training sample belongs. A data acquisition unit; A first training processing unit configured to perform machine learning of an encoder and a decoder using a plurality of training samples included in the acquired plurality of first datasets, The encoder is configured to convert samples into features; the decoder is configured to decode the samples from the features; The machine learning of the encoder and the decoder includes converting each of the training samples into features by the encoder, and training the encoder and the decoder so that samples decoded by the decoder from each of the obtained features match each of the training samples. A first training processing unit; a conversion unit configured to convert each of two or more training samples among the plurality of training samples into a feature using the trained encoder, thereby obtaining two or more sample points in a feature space, each of which corresponds to each of the training samples; a point designation unit configured to derive a linear sum of the two or more sample points acquired on the feature space, the linear sum being a maximum sum of distances calculated from the respective sample points according to a predetermined index; a sample generation unit configured to generate decoded samples corresponding to the linear sum features as new training samples by using the trained decoder; a labeling unit configured to generate a second data set by assigning a new ground truth label to the new training sample, the ground truth label being configured to indicate a class to which a feature included in the new training sample belongs; a second training unit configured to perform machine learning of a classification model using the plurality of first datasets and the generated one or more second datasets, The machine learning of the classification model includes training the classification model for each of the data sets such that a result of classifying a class to which the feature included in the training sample belongs by the classification model matches the correct label. A second training processing unit; Equipped with Model generation device.
2. The classification model is configured to output a posterior probability that a feature included in the sample belongs to each class; The machine learning of the classification model is performed in a first machine learning process using the plurality of first datasets and a second machine learning process using the one or more second datasets; performing the first machine learning to generate a provisional trained classification model; The point designation unit is Using the provisional trained classification model, calculate a posterior probability for each class that a feature included in a sample decoded by the trained decoder belongs to from the derived linear sum feature; Calculating the entropy of the decoded sample from the calculated posterior probabilities for each class; and correcting the derived linear sum based on the calculated gradient of the entropy so that the entropy increases; The method further comprises: the sample generation unit is configured to generate, as the new training sample, a decoded sample corresponding to the corrected linear sum feature by using the trained decoder; The model generating device according to claim 1 .
3. The first training processing unit is configured to further perform machine learning of a classifier using a plurality of training samples included in the plurality of first datasets; The machine learning of the classifier is configured by training the classifier to evaluate the degree to which an input sample belongs to a distribution of the plurality of training samples; The point designation unit is using the trained classifier to evaluate the degree to which a sample decoded by the trained decoder belongs to a distribution of the plurality of training samples from the derived linear sum feature; Calculating the reliability of the samples decoded by the trained decoder based on the evaluation of the degree; and correcting the derived linear sum based on the calculated gradient of the reliability so that the reliability is improved; The method further comprises: the sample generation unit is configured to generate, as the new training sample, a decoded sample corresponding to the corrected linear sum feature by using the trained decoder; 3. The model generating device according to claim 1 or 2.
4. training the classifier comprises performing adversarial learning between the encoder and the decoder and the classifier; The adversarial learning includes: training a classifier to identify origins of the input samples using a plurality of samples decoded by the decoder and the plurality of training samples as input samples, respectively; and training the encoder and the decoder such that the discrimination performance of the discriminator decreases when a sample decoded by the decoder is input to the discriminator; Including, evaluating the degree to which the decoded sample belongs to a distribution of the plurality of training samples comprises identifying the origin of the decoded sample using the trained classifier.
4. The model generating device of claim 3.
5. The point designation unit is evaluating a degree of difference between samples decoded by the trained decoder and training samples included in the first plurality of data sets from the derived linear sum features; Calculating the reliability of the samples decoded by the trained decoder based on the evaluation of the degree; and correcting the derived linear sum based on the calculated gradient of the reliability so that the reliability is improved; The method further comprises: Evaluating the degree of difference includes: converting the decoded samples into features by the trained encoder and calculating the difference between the resulting decoded sample features and the linear sum features; or further using the trained decoder to generate re-decoded samples from features of the decoded samples, and to calculate a reconstruction error between the decoded samples and the generated re-decoded samples; Consists of:
3. The model generating device according to claim 1 or 2.
6. The machine learning of the encoder and decoder includes: obtaining, for each pair of training samples from the first data sets, two features corresponding to the two training samples constituting each pair by using the encoder; calculating a composite feature for each pair by combining the two feature values acquired for each pair at a predetermined internal division ratio; generating decoded samples for each of the pairs from the calculated composite features using the decoder; training an estimator such that, for each pair, a result of estimating the interior ratio from the generated decoded samples by the estimator matches the predetermined interior ratio; and training the encoder and decoder to generate decoded samples for each pair that degrade the performance of the estimator to estimate the interior ratio; Further comprising: A model generating device according to any one of claims 1 to 5.
7. The point designation unit is Extracting two sample points adjacent to the derived linear sum from the two or more sample points obtained; and projecting the linear sum onto a straight line connecting the two extracted sample points in the feature space to obtain a projection point corresponding to the linear sum; It is configured as follows: The sample generation unit is configured to generate, as the new training sample, a decoded sample corresponding to a feature of the projection point corresponding to the linear sum by using the trained decoder.
7. The model generating apparatus of claim 6.
8. The extracting of the two sample points is configured by extracting two sample points to which features included in the corresponding training samples belong that are different classes.
8. The model generating apparatus of claim 7.
9. and an output unit configured to output the two or more training samples and the new training sample after generating the new training sample, after arranging the two or more training samples and the new training sample according to a positional relationship in the feature space. A model generating device according to any one of claims 1 to 8.
10. The label assignment unit is After outputting the two or more training samples and the new training sample by the output unit, accepting a designation of a class to which a feature included in the new training sample belongs; and assigning the new ground truth labels to the new training samples that are configured to indicate the designated classes; It is configured as follows:
10. The model generating apparatus of claim 9.
11. a thinning unit configured to determine whether or not a density of sample points corresponding to each of the training samples in the plurality of first data sets and the one or more second data sets exceeds a threshold in the feature space when each of the training samples included in the plurality of first data sets and the one or more second data sets is converted into a feature by the trained encoder, and to select a target to be thinned out from the one or more second data sets when the density exceeds the threshold; The second training processing unit is configured to perform machine learning of the classification model, excluding the second dataset selected as the target for thinning. A model generating device according to any one of claims 1 to 10.
12. The classification model is configured to receive an input of the feature amount, and output a result of classifying the feature amount into a class to which the feature amount belongs. A model generating device according to any one of claims 1 to 11.
13. The classification model is configured to receive an input of the sample, and output a result of classifying the class to which the feature belongs from the input sample. A model generating device according to any one of claims 1 to 11.
14. updating the trained encoder and decoder by re-performing machine learning of the encoder and decoder using the one or more second datasets generated by the first training process; obtaining, by the transformer, two or more sample points corresponding to two or more training samples of a plurality of training samples included in the plurality of first data sets and the one or more second data sets using the updated trained encoder; deriving a linear sum of the two or more sample points acquired by the point designation unit; generating, by the sample generation unit, decoded samples corresponding to the linear sum features as new training samples using the updated trained decoder; and generating a new second dataset by assigning new correct labels to the new training samples generated by the label assignment unit; configured to repeatedly execute A model generating device according to any one of claims 1 to 13.
15. The predetermined index is composed of either of the following formula 1 and formula 2: A model generating device according to any one of claims 1 to 14. [0010] [0025] In addition, X represents the linear sum to be derived, and x i indicates each of the sample points.
16. The training samples are image samples showing an object; The feature class is related to a type of the object depicted in the image sample.
16. A model generating device according to any one of claims 1 to 15.
17. The object is a product, The class of features is related to a type of defect contained in the product.
17. The model generating apparatus of claim 16.
18. a data acquisition unit configured to acquire two or more training samples; a conversion unit configured to convert each of the two or more training samples into a feature using a trained encoder, thereby obtaining two or more sample points in a feature space, each of the two or more sample points corresponding to each of the training samples; a point designation unit configured to derive a linear sum of the two or more sample points acquired on the feature space, the linear sum being a maximum sum of distances calculated from the respective sample points according to a predetermined index; a sample generation unit configured to generate decoded samples corresponding to the linear sum features as new training samples by using a trained decoder; Equipped with the trained encoder and decoder are generated by machine learning using a plurality of training samples, including the two or more training samples; The machine learning includes converting each of the training samples into features by the encoder, and training the encoder and the decoder so that samples decoded by the decoder from each of the obtained features match each of the training samples. Data generation device.
19. The computer Obtaining a plurality of first data sets each composed of a combination of training samples and ground truth labels, The training samples include features, The ground truth label is configured to indicate a class to which the feature contained in the training sample belongs. Steps and performing machine learning of an encoder and a decoder using a plurality of training samples included in the obtained plurality of first datasets, The encoder is configured to convert samples into features; the decoder is configured to decode the samples from the features; The machine learning of the encoder and the decoder includes converting each of the training samples into features by the encoder, and training the encoder and the decoder so that samples decoded by the decoder from each of the obtained features match each of the training samples. Steps and using the trained encoder to convert each of two or more training samples among the plurality of training samples into a feature, thereby obtaining two or more sample points in a feature space, each of which corresponds to each of the training samples; deriving a linear sum of the two or more sample points acquired in the feature space, the linear sum being a maximum sum of distances calculated from the respective sample points according to a predetermined index; generating decoded samples corresponding to the linear sum features as new training samples by using the trained decoder; generating a second data set by assigning new ground truth labels to the new training samples, the ground truth labels being configured to indicate classes to which features contained in the new training samples belong; performing machine learning of a classification model using the plurality of first datasets and the generated one or more second datasets; The machine learning of the classification model includes training the classification model for each of the data sets such that a result of classifying a class to which the feature included in the training sample belongs by the classification model matches the correct label. Steps and Execute Model generation method.
20. On the computer, Obtaining a plurality of first data sets each composed of a combination of training samples and ground truth labels, The training samples include features, The ground truth label is configured to indicate a class to which the feature contained in the training sample belongs. Steps and performing machine learning of an encoder and a decoder using a plurality of training samples included in the obtained plurality of first datasets, The encoder is configured to convert samples into features; the decoder is configured to decode the samples from the features; The machine learning of the encoder and the decoder includes converting each of the training samples into features by the encoder, and training the encoder and the decoder so that samples decoded by the decoder from each of the obtained features match each of the training samples. Steps and using the trained encoder to convert each of two or more training samples among the plurality of training samples into a feature, thereby obtaining two or more sample points in a feature space, each of which corresponds to each of the training samples; deriving a linear sum of the two or more sample points acquired in the feature space, the linear sum being a maximum sum of distances calculated from the respective sample points according to a predetermined index; generating decoded samples corresponding to the linear sum features as new training samples by using the trained decoder; generating a second data set by assigning new ground truth labels to the new training samples, the ground truth labels being configured to indicate classes to which features contained in the new training samples belong; performing machine learning of a classification model using the plurality of first datasets and the generated one or more second datasets; The machine learning of the classification model includes training the classification model for each of the data sets such that a result of classifying a class to which the feature included in the training sample belongs by the classification model matches the correct label. Steps and In order to execute Model generator.
Citation Information
Patent Citations
Inspection device
JP2012026982A
Apparatus and method for online recognition and setting screen used therefor
JP2019074945A
Image generating method, image determining method, image generating apparatus, and image generating program
JP2020046984A
Data augmentation in transaction classification using a neural network
US20200210808A1
