Model generation device and method, classification device and medium

Abstract

This invention provides a model generation apparatus and method, a classification device, and a medium. One aspect of the model generation apparatus involves using a trained encoder to acquire two or more sample points in the feature space that correspond to two or more training samples from a plurality of training samples. Based on a predetermined index, the apparatus derives the sum of distances calculated from each acquired sample point as the maximum linear sum. Then, using a trained decoder, it generates decoded samples corresponding to the feature values ​​of the derived linear sum as new training samples.

Description

Technical Field

[0001] This invention relates to a model generation apparatus and method, a classification device and a medium. Background Technology

[0002] In the past, in manufacturing scenarios such as manufacturing lines, the following technology has been developed: using an imaging device to photograph the manufactured products and analyzing the obtained image data to inspect the quality of the products. For example, Patent Document 1 proposes an inspection device that uses a trained first neural network to determine whether an object mapped in an image is normal or abnormal. If it is determined to be abnormal, a trained second neural network is used to classify the type of abnormality.

[0003] By utilizing machine learning models such as neural networks, product appearance inspection can be performed based on the computational results of a trained machine learning model, even without detailed manual specification of image processing for the acquired images. This simplifies information processing for appearance inspection and reduces the time required to create inspection procedures. On the other hand, generating a trained machine learning model (i.e., implementing machine learning) requires preparing training samples corresponding to the capabilities the machine learning model learns (in this example, the ability to inspect product appearance). If a sufficient number of training samples comprehensively cover all possible scenarios in which the trained machine learning model can be applied, the generation of a trained machine learning model capable of performing inference tasks with high accuracy can be expected. However, preparing such comprehensive and sufficient training samples is costly.

[0004] Therefore, in recent years, research has been conducted on techniques such as using training samples obtained through data augmentation, generative models, and other methods to generate new training samples, thereby increasing the number of training samples. For example, Non-Patent Literature 1 and Patent Literature 2 propose methods that use a trained conditional generative model (AC-GAN in Non-Patent Literature 1) to generate training samples for object categories. According to these methods, new training samples that exhibit different states of affairs from the original training samples can be generated, thereby reducing the cost of preparing a sufficient number of training samples. Furthermore, by using a sufficient number of training samples obtained in machine learning, the performance of the trained machine learning model can be expected to improve.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2012-026982

[0008] Patent Document 2: Japanese Patent Application Publication No. 2019-074945

[0009] Non-patent literature

[0010] Non-Patent Literature 1: Quan Kong, Bin Tong, Martin Klinkigt, Yuki Watanabe, Naoto Akira, Tomokazu Murakami, “Active Generative Adversarial Network for Image Classification”, arXiv:1906.07133v1 [cs.LG], June 17, 2019 Summary of the Invention

[0011] The problem that the invention aims to solve

[0012] The inventors of this application have discovered that the conventional methods for generating training samples have the following problems. Specifically, in data augmentation methods, new training images (training samples) are generated, for example, by applying image processing techniques such as translation, scaling, rotation, and noise addition to the original training image. Therefore, while it is possible to generate training samples that represent situations near the original training samples, it is difficult to generate training samples that represent other situations (e.g., intermediate situations). Consequently, it is difficult to obtain comprehensive training samples in data augmentation methods. Furthermore, in methods using generative models, new training samples are generated primarily based on random numbers, making it difficult to control the generated training samples. Therefore, it is difficult to efficiently obtain comprehensive training samples.

[0013] Furthermore, this problem is not unique to generating a fully trained model that can be used for visual inspection. Moreover, it is not unique to situations where image data is used as training samples. For example, the same problem may arise in all scenarios where training samples are collected to generate a fully trained classification model capable of performing classification tasks—such as classifying the types of objects mapped in an image, classifying the states of objects represented by sensor data, etc.—that classify features contained in the data. Training samples may include, in addition to image data, audio data, numerical data, text data, and measurement data obtained through sensors.

[0014] The present invention was made in view of this situation, and its purpose is to reduce the cost of collecting training samples in a comprehensive manner, or to improve the performance of the generated trained classification model by using the training samples obtained therefrom in machine learning.

[0015] Technical means to solve the problem

[0016] To solve the aforementioned problems, the present invention adopts the following structure.

[0017] That is, one aspect of the model generation apparatus of the present invention includes a data acquisition unit, a first training processing unit, a conversion unit, a point assignment unit, a sample generation unit, a label assignment unit, and a second training processing unit. The data acquisition unit is configured to acquire multiple first datasets, each of which contains a combination of training samples and positive solution labels. The training samples contain features, and the positive solution labels are configured to represent the category to which the features contained in the training samples belong. The first training processing unit is configured to perform machine learning on an encoder and a decoder using the multiple training samples contained in the acquired multiple first datasets. The encoder is configured to convert samples into feature quantities. The decoder is configured to decode the samples based on the feature quantities. The machine learning of the encoder and decoder includes the following process: training the encoder and decoder such that the encoder converts each training sample into a feature quantity, and the decoder decodes the samples based on the obtained feature quantities to conform to each training sample. The conversion unit is configured to use the trained encoder to convert two or more training samples from the multiple training samples into feature quantities, thereby obtaining two or more sample points in the feature space corresponding to each training sample. The point designation unit is configured to derive a linear sum in the feature space according to a predetermined index. This linear sum is the linear sum of the acquired two or more sample points, and the sum of the distances calculated from each sample point is the largest possible linear sum. The sample generation unit is configured to generate decoded samples corresponding to the feature values ​​of the linear sum as new training samples using the trained decoder. The label assignment unit is configured to assign new positive solution labels to the new training samples, thereby generating a second dataset. The new positive solution labels represent the categories to which the features contained in the generated new training samples belong. The second training processing unit is configured to perform machine learning of the classification model using the plurality of first datasets and one or more generated second datasets. The machine learning of the classification model includes the following process: training the classification model with respect to each dataset such that the classification model's classification of the categories to which the features contained in the training samples belong conforms to the positive solution labels.

[0018] In this structure, multiple original datasets (first dataset) are used to implement machine learning for the encoder and decoder. The result of machine learning is the generation of a trained encoder and decoder. Based on the trained encoder, each training sample can be mapped to a feature space (latent space) while maintaining information related to the contained features. Based on the trained decoder, samples can be decoded based on the values ​​(feature quantities) in the feature space. In this structure, by using the trained encoder to convert two or more training samples from multiple training samples into feature quantities, two or more sample points corresponding to each training sample are obtained in the feature space. The sum of the distances calculated from each sample point is derived to be the maximum linear sum based on a specified benchmark. Furthermore, by using the trained decoder, a decoded sample corresponding to the feature quantity of the linear sum is generated as a new training sample. That is, according to this structure, in the feature space, the point where the sum of the distances from each sample point is the maximum linear sum is obtained as a new sample point, thereby forming a state where sample points exist everywhere. Therefore, by using the trained decoder, a new dataset (second dataset) containing samples decoded based on linear summation of features is appended to the learning data, thereby obtaining learning data with high comprehensiveness and evaluable in the feature space. Furthermore, in this structure, by using the trained encoder and decoder, at least a portion of the process of generating such new training samples (the dataset containing the new training samples) can be automated. Therefore, according to this structure, the cost of comprehensively collecting training samples can be reduced. In addition, in this structure, besides the original dataset (first dataset), the new dataset (second dataset) generated through the processing is used for machine learning of the classification model. Thus, according to this structure, by using the comprehensively obtained training samples for machine learning, the performance of the generated trained classification model can be improved.

[0019] In the model generation apparatus of one aspect, the classification model may be configured to output the posterior probability of the features contained in the sample for each category. The machine learning of the classification model may be implemented using a first machine learning using the plurality of first datasets and a second machine learning using one or more second datasets. Through the implementation of the first machine learning, a temporarily trained classification model can be generated. The point designation unit may further be configured to: using the temporarily trained classification model, calculate for each category the posterior probability of the features contained in the sample decoded by the trained decoder based on the feature quantity of the derived linear sum; calculate the entropy of the decoded sample based on the calculated posterior probability of each category; and correct the derived linear sum based on the gradient of the calculated entropy to increase the entropy. The sample generation unit may be configured to generate a decoded sample corresponding to the feature quantity of the corrected linear sum as the new training sample using the trained decoder.

[0020] In this structure, the entropy of samples decoded from a linear sum of features is evaluated using a temporarily trained classification model. Entropy represents the degree to which the features contained in the decoded sample can be classified (in other words, uncertainty). Higher uncertainty indicates that classifying the features contained in the sample is more difficult, meaning that machine learning of the sample is less sufficient. In this structure, the derived linear sum is modified to increase the entropy, and a new training sample is generated using the trained decoder, based on the modified linear sum of features. Thus, in the temporarily trained classification model, training samples with insufficient machine learning can be obtained. By using these training samples for machine learning in the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0021] In the model generation apparatus of one aspect, the first training processing unit may be configured to further implement machine learning of the recognizer using multiple training samples contained in the multiple first datasets. The machine learning of the recognizer may be configured by training the recognizer to evaluate the degree to which an input sample belongs to the distribution of the multiple training samples. The point designation unit may further be configured to: use the trained recognizer to evaluate the degree to which a sample decoded by the trained decoder based on the feature quantity of the derived linear sum belongs to the distribution of the multiple training samples; calculate the reliability of the sample decoded by the trained decoder based on the result of the evaluation of the degree; and correct the derived linear sum based on the gradient of the calculated reliability to improve the reliability. The sample generation unit may be configured to generate a decoded sample corresponding to the feature quantity of the corrected linear sum as the new training sample using the trained decoder.

[0022] The samples decoded by the trained decoder based on the feature values ​​of points other than those corresponding to the training samples in the feature space can be considered pseudo-data that mimics the real data (training samples). Depending on the situation, this pseudo-data may deviate drastically from the real data. Therefore, in this structure, the reliability of the samples decoded by the trained decoder is evaluated using a trained recognizer. For example, the higher the degree to which the decoded sample belongs to the distribution of the training samples, the higher the reliability evaluation of the sample; the lower the degree to which the decoded sample belongs to the distribution of the training samples, the lower the reliability evaluation of the sample. Furthermore, based on the evaluation result, the linear sum is corrected to improve reliability, and the trained decoder is used to generate samples decoded based on the feature values ​​of the corrected linear sum as new training samples. This prevents training samples that deviate drastically from the real data from being added to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0023] In the model generation apparatus of one aspect, training the recognizer can be configured by implementing adversarial learning between the encoder and the decoder and the recognizer. Furthermore, the adversarial learning may include the following steps: training the recognizer using multiple samples decoded by the decoder and the multiple training samples as input samples to identify the source of the input samples; and training the encoder and the decoder such that the recognition performance of the recognizer decreases when a sample decoded by the decoder is input to the recognizer. The degree to which the decoded samples belong to the distribution of the multiple training samples can be evaluated by using the trained recognizer to identify the source of the decoded samples.

[0024] A recognizer trained using adversarial learning acquires the ability to distinguish between training samples and samples (pseudo-samples) originating from the decoder, depending on the degree of adversarial learning. Therefore, with this trained recognizer, the reliability of input samples can be evaluated based on the result of identifying the source of the input sample. That is, when the source of the input sample is a training sample, the more errors the recognizer makes, the higher the reliability can be evaluated; when the source of the input sample is the decoder, the more accurately the recognizer identifies it, the lower the reliability can be evaluated. Therefore, according to this structure, the reliability of the decoded samples can be appropriately evaluated using a recognized recognizer generated with the aid of adversarial learning. This helps to prevent training samples that deviate significantly from real data from being added to the training data, thereby improving the expected performance of the generated trained classification model.

[0025] In the model generation apparatus of one aspect, the point designation unit may further be configured to: evaluate the degree of difference between the sample decoded by the trained decoder based on the feature quantity of the derived linear sum and the training samples contained in the plurality of first datasets; calculate the reliability of the sample decoded by the trained decoder based on the result of the evaluation of the degree; and correct the derived linear sum based on the gradient of the calculated reliability to improve the reliability. The evaluation of the degree of difference may be configured by: converting the decoded sample into a feature quantity by the trained encoder and calculating the difference between the feature quantity of the decoded sample and the feature quantity of the linear sum; or further using the trained decoder to generate a re-decoded sample from the feature quantity of the decoded sample and calculating the reconstruction error between the decoded sample and the generated re-decoded sample.

[0026] The reliability of the decoded samples can also be evaluated by the differences between the decoded samples and the training samples in the feature space or the original space (the dimensional space of the samples). In this structure, as the difference between the decoded samples and the training samples, the trained encoder converts the samples decoded by the trained decoder from the derived linear sum into feature quantities, and calculates the difference between the obtained feature quantities and the linear sum. Alternatively, the trained decoder is further used to generate re-decoded samples from the feature quantities obtained from the decoded samples, and the reconstruction error between the decoded samples and the re-decoded samples is calculated. The trained encoder and decoder are generated by machine learning using the training samples contained in the first dataset. Therefore, the more similar the samples decoded by the trained decoder from the derived linear sum are to the training samples, the smaller the differences (differences between feature quantities / reconstruction errors) are; the more the decoded samples deviate from the training samples, the larger the differences are. Therefore, the degree of difference between the samples decoded from the linear sum and the training samples can be appropriately evaluated by these differences. In this structure, the linear sum is corrected based on the evaluation results to improve reliability, and the trained decoder is used to generate samples decoded from the features of the corrected linear sum as new training samples. This prevents training samples that deviate significantly from the real data from being added to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0027] In the model generation apparatus of one aspect, the machine learning of the encoder and decoder may further include: for a pair of training samples obtained from the plurality of first datasets, using the encoder to obtain two feature quantities corresponding to the two training samples constituting each pair; synthesizing the two feature quantities obtained for each pair with a predetermined inner fraction, thereby calculating a synthesized feature quantity for each pair; using the decoder to generate a decoded sample for each pair based on the calculated synthesized feature quantity; training an inferr with respect to each pair such that the result of the inferr inferring the inner fraction based on the generated decoded sample conforms to the predetermined inner fraction; and training the encoder and decoder with respect to each pair to generate decoded samples that degrade the performance of the inferr inferring the inner fraction.

[0028] In this structure, adversarial learning is implemented between the encoder / decoder and the inferr. Through adversarial learning, the more the inferr's ability to discern the inner fraction increases, the more the encoder and decoder can generate samples whose inner fraction is difficult to infer. That is, regarding points between sample points, it is possible to suppress the decoding (generation) of samples that deviate from the real data whose inner fraction is easily determined (in other words, the decoder decodes samples whose inner fraction is difficult to determine and indistinguishable from the real data). Therefore, according to this structure, a trained encoder and decoder can be generated that possess the ability to generate decoded samples similar to real data (training samples) from linearly summed feature quantities. Thus, when using the trained encoder and decoder to generate new training samples, it is possible to suppress the addition of training samples that are extremely different from the real data to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0029] In the model generation apparatus of one aspect, the point designation unit may be configured to: extract two sample points from the two or more acquired sample points that are near the derived linear sum, and project the derived linear sum onto the straight line connecting the two extracted sample points in the feature space, thereby obtaining a projection point corresponding to the linear sum. The sample generation unit may be configured to generate a decoded sample corresponding to the feature quantity of the projection point corresponding to the linear sum as the new training sample using the trained decoder. According to this structure, by converting the linear sum into the inner ratio (projection point) of two sample points, the value of the linear sum can be corrected to a value within or near the training range of the adversarial learning. This improves the certainty of suppressing training samples that deviate significantly from the real data from the training data. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0030] In the model generation apparatus described above, the extraction of the two sample points can be achieved by extracting two sample points from each corresponding training sample whose features belong to different categories. According to this structure, it is possible to suppress the generation of new training samples that deviate from real data between training samples of different categories. That is, with the trained decoder, decoded samples similar to real data can be generated as new training samples within a range closer to the category recognition boundary than each sample point. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be enhanced.

[0031] The model generation apparatus of one aspect may further include an output unit, which is configured to, after generating the new training sample, configure the two or more training samples and the new training sample according to their positional relationship in the feature space and then output them. According to this structure, it is possible to determine whether the decoded sample (new training sample) can be added to the learning data based on the output result. This prevents training samples that deviate significantly from the real data from being added to the learning data. As a result, by using the obtained training samples for machine learning of the classification model, the expected performance improvement of the generated trained classification model can be increased. Furthermore, it is possible to determine the category to which the features contained in the decoded sample (new training sample) belong based on the output result and according to their positional relationship with the original training sample in the feature space.

[0032] In the model generation apparatus of one aspect, the label assignment unit may be configured to: after outputting the two or more training samples and the new training sample from the output unit, accept the assignment of the category to which the features contained in the new training sample belong, and assign the new training sample a new correct label representing the assigned category. According to this structure, while determining the category to which the features contained in the decoded sample (new training sample) belong based on the output result and the positional relationship with the original training sample in the feature space, and accepting the category assignment, it is possible to assign a new correct label representing the appropriate category to the new training sample. This prevents the assignment of incorrect correct labels to the new training sample. As a result, the obtained training samples can be appropriately used for machine learning of the classification model, thereby improving the expected performance improvement of the generated trained classification model.

[0033] The model generation apparatus of one aspect may further include a thinning unit, which is configured to, when the trained samples contained in the plurality of first datasets and the one or more second datasets are converted into feature quantities by the trained encoder, determine whether the density of sample points corresponding to each training sample in the feature space exceeds a threshold, and if the density exceeds the threshold, select objects to be thinned from the one or more second datasets. The second training processing unit may be configured to perform machine learning of the classification model except for the second datasets selected as objects to be thinned. When there are excessive training samples in the vicinity of the space, some of the training samples may not contribute much to improving the performance of the classification model. According to the above structure, such an excess of training samples can be detected based on the density in the feature space, and a portion of the excess training samples can be thinned. Thus, the number of datasets used in the machine learning of the classification model is reduced, thereby both suppressing the performance degradation of the generated trained classification model and reducing the computational load of machine learning, achieving efficiency in the computational resources (processors, memory) consumed by machine learning.

[0034] In the model generation apparatus of one aspect, the classification model can be configured to accept the input of the feature quantity and output a classification result based on the category to which the feature belongs according to the input feature quantity. According to this structure, in a configuration where the classification task computation and encoder processing are shared, the performance of the generated, trained classification model can be improved. In this configuration, the classification model can be trained simultaneously with the encoder training, thus also improving the efficiency of the machine learning processing of the classification model.

[0035] In the model generation apparatus of one aspect, the classification model can be configured to accept the input of the sample and output a classification result based on the category to which the feature belongs according to the input sample. According to this structure, in a configuration where the classification model, encoder, and decoder are independently configured, the performance of the generated, trained classification model can be improved. In this configuration, the structure of the classification model is independent of the encoder, thus the structure of the classification model can be optimized for the classification task. Therefore, the computational cost of the classification model can also be reduced.

[0036] The model generation apparatus of one aspect can be configured to repeatedly perform: the first training processing unit uses the generated one or more second datasets to re-implement the machine learning of the encoder and decoder, thereby updating the trained encoder and decoder; the conversion unit uses the updated trained encoder to obtain two or more sample points corresponding to two or more training samples from the plurality of training samples contained in the plurality of first datasets and the one or more second datasets; the point assignment unit derives the linear sum of the obtained two or more sample points; the sample generation unit uses the updated trained decoder to generate decoded samples corresponding to the feature values ​​of the linear sum as new training samples; and the label assignment unit assigns new positive solution labels to the generated new training samples, thereby generating a new second dataset. When new training samples are added and the machine learning of the encoder and decoder is further implemented using the added new training samples, the feature space mapped by the encoder may change. According to this structure, by repeatedly performing machine learning of the encoder and decoder and generating new training samples, it is possible to add new training samples while forming a state where sample points are present everywhere. As a result, it is possible to collect a more appropriate range of training samples.

[0037] In the model generation apparatus of the aforementioned aspect, the specified index may include any one of the following formulas 1 and 2.

[0038] [Number 1]

[0039]

[0040] [Number 2]

[0041]

[0042] Additionally, X represents the derived linear sum, x i Let each of the sample points be represented. According to the structure, the centroid can be obtained according to Equation 1. According to Equation 2, the Voronoi point can be obtained. Based on these, a linear sum maximizing the sum of distances from each sample point can be derived in the inner region of each sample point. Thus, an appropriate linear sum can be obtained, and sample points can be added to regions sparse in the feature space. Furthermore, by repeatedly performing this operation, a state where sample points are ubiquitous can be formed. Therefore, according to the structure, a comprehensive collection of training samples can be appropriately gathered. Moreover, by using the comprehensively gathered training samples for machine learning, the performance of the generated, trained classification model can be improved.

[0043] In the model generation apparatus described above, the type of training samples is not particularly limited as long as they can be used as objects in the classification task, and can be appropriately selected according to the implementation method. Training samples can be, for example, image data, sound data, numerical data, text data, and sensor data obtained through sensors. Similarly, the object samples obtained when performing a classification task using the trained classification model are also included.

[0044] For example, in the model generation apparatus of one aspect, the training samples may be image samples that map objects, and the category of the features may be related to the type of the object mapped in the image samples. According to this structure, in scenarios where a trained classification model for object recognition is generated, the cost of comprehensively collecting training samples can be reduced. Furthermore, by using the training samples obtained thereby for machine learning, the performance of the generated trained classification model can be improved.

[0045] Furthermore, in the model generation apparatus of the aforementioned aspect, the object can be a product, and the category of the feature can be related to the type of defect contained in the product. According to this structure, in a scenario where a trained classification model is generated for appearance inspection as an embodiment of object recognition, the cost of extensively collecting training samples can be reduced. Moreover, by using the training samples obtained thereby for machine learning, the performance of the generated trained classification model can be improved.

[0046] The present invention is not limited to a model generation apparatus that performs all of the aforementioned series of processes. One aspect of the apparatus can be configured by extracting, for example, a portion for generating a trained encoder and decoder, a portion for generating new training samples (a second dataset) using the trained encoder and decoder, and a portion for performing machine learning on a classification model using learning data containing the new training samples, from the model generation apparatuses of the various embodiments. In one example, the apparatus corresponding to the portion for generating the trained encoder and decoder can be called a first model generation apparatus. The portion for generating new training samples using the trained encoder and decoder can be called a data generation apparatus. The portion for performing machine learning on a classification model using learning data containing the new training samples can be called a second model generation apparatus. Furthermore, one aspect of the present invention can also be a classification apparatus that uses a trained classification model generated by the model generation apparatus of any of the aforementioned embodiments. The classification apparatus can be appropriately modified (e.g., a checking device, etc.) depending on the type of classification task.

[0047] For example, a classification apparatus according to one aspect of the present invention includes: an acquisition unit configured to acquire object samples; a classification unit including a trained classification model generated by the model generation apparatus of any of the embodiments, and the classification unit being configured to classify the categories to which features contained in the acquired object samples belong by using the trained classification model; and an output unit configured to output information related to the result of classifying the categories. According to this structure, by using the trained classification model generated by the processing, high-precision classification tasks can be expected to be performed.

[0048] Furthermore, for example, a data generation apparatus according to one aspect of the present invention includes: a data acquisition unit configured to acquire two or more training samples; a conversion unit configured to convert the two or more training samples into feature quantities using a trained encoder, thereby acquiring two or more sample points in a feature space corresponding to each training sample; a point designation unit configured to derive a linear sum in the feature space according to a predetermined index, the linear sum being the linear sum of the acquired two or more sample points, and the sum of the distances calculated from each sample point being the largest linear sum; and a sample generation unit configured to generate a decoded sample corresponding to the feature quantity of the linear sum as a new training sample using a trained decoder. The trained encoder and decoder are generated by machine learning using multiple training samples containing the two or more training samples. The machine learning includes the process of training the encoder and decoder such that the encoder converts each training sample into a feature quantity, and the decoder decodes a sample based on the acquired feature quantity that conforms to each training sample. According to this structure, the cost of comprehensively collecting training samples can be reduced.

[0049] Furthermore, as other forms of the various devices described above, one aspect of the present invention can also be an information processing method that implements all or part of the above structures, or it can be a program, or it can be a storage medium readable by a computer or other device, machine, etc., storing such a program. Here, a storage medium readable by a computer or other device refers to a medium that stores information such as programs through electrical, magnetic, optical, mechanical, or chemical action. Moreover, one aspect of the present invention can also be a classification system that includes a model generation apparatus (a first model generation apparatus, a data generation apparatus, and a second model generation apparatus) and a classification apparatus from any of the embodiments described above.

[0050] For example, one aspect of the model generation method of the present invention is an information processing method in which a computer performs the following steps: acquiring a plurality of first datasets, each of the plurality of first datasets containing a combination of training samples and positive solution labels; using the plurality of training samples contained in the acquired plurality of first datasets to perform machine learning on the encoder and decoder; using the trained encoder, converting two or more training samples from the plurality of training samples into feature quantities, thereby obtaining two or more sample points in the feature space corresponding to each training sample; deriving a linear sum in the feature space according to a predetermined index, the linear sum being the linear sum of the two or more sample points obtained, and the sum of the distances calculated from each sample point being the largest linear sum; using the trained decoder, generating a decoded sample corresponding to the feature quantity of the linear sum as a new training sample; assigning a new positive solution label to the new training sample, thereby generating a second dataset, the new positive solution label constituting a category to which the features contained in the generated new training sample belong; and using the plurality of first datasets and one or more generated second datasets to perform machine learning on the classification model.

[0051] Furthermore, for example, one aspect of the model generation program of the present invention is a computer-readable storage medium storing a program for causing a computer to perform the following steps: acquiring a plurality of first datasets, each of the plurality of first datasets containing a combination of training samples and positive solution labels; using the plurality of training samples contained in the acquired plurality of first datasets to perform machine learning on the encoder and decoder; using the trained encoder to convert two or more training samples from the plurality of training samples into feature quantities, thereby obtaining two or more sample points in the feature space corresponding to each training sample; deriving a linear sum in the feature space according to a predetermined index, the linear sum being the linear sum of the acquired two or more sample points, and the sum of the distances calculated from each sample point being the largest linear sum; using the trained decoder to generate decoded samples corresponding to the feature quantities of the linear sum as new training samples; assigning new positive solution labels to the new training samples, thereby generating a second dataset, the new positive solution labels constituting a category to which the features contained in the generated new training samples belong; and using the plurality of first datasets and one or more generated second datasets to perform machine learning on the classification model.

[0052] The effects of the invention

[0053] According to the present invention, the cost of extensively collecting training samples can be reduced. Moreover, by using the training samples obtained thereby for machine learning, the performance of the generated trained classification model can be improved. Attached Figure Description

[0054] Figure 1 An example of a scenario in which the present invention is applicable is illustrated.

[0055] Figure 2 An example of the hardware structure of the model generation apparatus of the embodiment is illustrated schematically.

[0056] Figure 3 An example of the hardware structure of the classification device in the embodiment is illustrated schematically.

[0057] Figure 4 An example of the software structure of the model generation apparatus of the embodiment is illustrated schematically.

[0058] Figure 5A This is an example of a machine learning process involving an encoder, decoder, and classification model.

[0059] Figure 5B This is an illustrative example of a machine learning process involving an encoder and decoder.

[0060] Figure 5C This is an illustrative example of a data generation process.

[0061] Figure 5D This is an illustrative example of a data generation process.

[0062] Figure 5E This illustrative example illustrates an example of the output format of a new training sample.

[0063] Figure 6 An example of the software structure of the classification device in the embodiment is illustrated schematically.

[0064] Figure 7A This is a flowchart illustrating an example of the processing flow of a model generation apparatus for an implementation method.

[0065] Figure 7B This is a flowchart illustrating an example of the processing flow of a model generation apparatus for an implementation method.

[0066] Figure 8 This is a flowchart illustrating an example of the processing flow of the classification device in an embodiment.

[0067] Figure 9 An example of another scenario in which the invention is applicable is illustrated.

[0068] Figure 10An example of another scenario in which the invention is applicable is illustrated.

[0069] Figure 11 An example of another scenario in which the invention is applicable is illustrated.

[0070] Figure 12 An example of the structure of a model generation apparatus according to another embodiment is illustrated schematically.

[0071] Figure 13 An example of the structure of a classification model of another embodiment is illustrated schematically.

[0072] Figure 14 An example of the machine learning (adversarial learning) process of a recognizer in another embodiment is illustrated schematically.

[0073] Figure 15 An example of a data generation process in another embodiment is illustrated schematically.

[0074] Figure 16 This is a diagram illustrating an example of an optimization method for obtaining a modified linear sum in another embodiment.

[0075] Figure 17 An example of a data generation process in another embodiment is illustrated schematically.

[0076] Explanation of symbols

[0077] 1: Model generation device

[0078] 2, 2A: Sorting device

[0079] 2B: Inspection device

[0080] 2C: Identification device

[0081] 5: Encoder

[0082] 6: Decoder

[0083] 7, 70: Classification Model

[0084] 11, 21: Control Department

[0085] 12, 22: Storage Department

[0086] 13, 23: Communication Interface

[0087] 14, 24: External Interfaces

[0088] 15, 25: Input devices

[0089] 16, 26: Output devices

[0090] 17, 27: Drivers

[0091] 30: First Dataset

[0092] 31, 31_1, 31_2, 36: Training samples

[0093] 32, 37: Correct answer labels

[0094] 35: Second Dataset

[0095] 51, 61, 71, 601, 651: Input layer

[0096] 52, 62, 72, 602, 652: Middle (hidden) layers

[0097] 53, 63, 73, 603, 653: Output layers

[0098] 81: Model Generation Program

[0099] 82: Classification Procedure

[0100] 91, 92: Storage media

[0101] 100, 100A: Classification system

[0102] 100B: Inspection System

[0103] 100C: Identification System

[0104] 101: First Model Generation Device

[0105] 102: Data generation device

[0106] 103: Second model generation device

[0107] 111, 1111, 1113, 1115: Data Acquisition Department

[0108] 112: First Training Processing Department

[0109] 113: Conversion Section

[0110] 114: Point to designated section

[0111] 115: Sample Generation Department

[0112] 116, 213: Output Section

[0113] 117: Labeling Department

[0114] 118: Intermittent section

[0115] 119: Second Training Processing Department

[0116] 125: Learning Outcome Data

[0117] 211: Acquisition Department

[0118] 212: Classification Department

[0119] 221, 221A, 221B, 221C: Object Samples

[0120] 311, 311_1, 311_2: Sample points

[0121] 312, 312_1, 312_2, 313_5: Linear sums

[0122] 313: Projection point

[0123] 315: Maximum value

[0124] 600: Predictor

[0125] 650: Recognizer

[0126] 1110, 1112, 1114, 1116: Storage and Processing Department

[0127] RA, RC: Object

[0128] RB: Product

[0129] S101~S111, S112~S119, S201~S203: Steps

[0130] SA, SB: Camera

[0131] SC: Sensor Detailed Implementation

[0132] Hereinafter, an embodiment of one aspect of the present invention (hereinafter also referred to as "this embodiment") will be described based on the accompanying drawings. However, the embodiment described below is merely an example of the present invention in all respects. Various modifications or variations can be made without departing from the scope of the present invention. That is, in the implementation of the present invention, specific structures corresponding to the embodiments may also be appropriately adopted. In addition, the data appearing in this embodiment is described using natural language, but more specifically, it is specified using computer-recognizable pseudo-language, commands, parameters, machine language, etc.

[0133] §1 Applicable Examples

[0134] Figure 1 An example of a scenario in which the present invention is applied is illustrated. For example... Figure 1 As shown, the classification system 100 of this embodiment includes a model generation device 1 and a classification device 2.

[0135] The model generation apparatus 1 of this embodiment is a computer configured to perform the following processes: machine learning of the encoder 5 and decoder 6, generation of new training samples 36 using the trained encoder 5 and decoder 6, and machine learning of the classification model 7. Specifically, the model generation apparatus 1 of this embodiment acquires a plurality of first datasets 30. Each first dataset 30 contains a combination of training samples 31 and positive solution labels 32. The training samples 31 contain features. The positive solution labels 32 are configured to indicate the category to which the features contained in the corresponding training sample 31 belong (i.e., the positive solution for the feature classification task).

[0136] The data type of each training sample 31 is not particularly limited as long as it can be used as an object in the classification task, and can be appropriately selected according to the implementation method. Each training sample 31 can be, for example, image data, sound data, numerical data, text data, and sensor data obtained through sensors. In one example, each training sample 31 can be sensor data obtained by observing any object using a sensor. Sensors can be, for example, image sensors (cameras), infrared sensors, sound sensors (microphones), ultrasonic sensors, light sensors, pressure sensors, barometric pressure sensors, temperature sensors, etc. Furthermore, sensors can be, for example, environmental sensors, life sensors, medical examination devices, vehicle sensors, home security sensors, etc. Environmental sensors can be, for example, barometers, thermometers, hygrometers, sound barometers, sound sensors, ultraviolet sensors, lux meters, rain gauges, gas sensors, etc. Vital sensors include, for example, blood pressure monitors, pulse monitors, heart rate monitors, electrocardiogram (ECG) recorders, electromyography (EMG) recorders, thermometers, skin conductance meters, microwave sensors, brainwave analyzers, magnetoencephalograms (MEGs), activity measuring devices, blood glucose meters, potential ocular sensors, and eye movement measuring devices. Medical examination devices include, for example, computed tomography (CT) scanners and magnetic resonance imaging (MRI) devices. Vehicle sensors include, for example, image sensors, light detection and ranging (Lidar) sensors, millimeter-wave radar, ultrasonic sensors, and accelerometers. Home security sensors include, for example, image sensors, infrared sensors, motion (sound) sensors, gas (CO2, etc.) sensors, current sensors, and smart meters (sensors that measure the power consumption of household appliances, lighting, etc.).

[0137] The content of the classification task can be appropriately determined according to the implementation method. In one example, each training sample 31 can be an image sample mapping an object. In this case, the classification task can be to identify the type of object. The category of the feature can be related to the type of object mapped in the image sample. The correct label 32 can be configured to represent the type of object mapped in the corresponding training sample 31. In another example, each training sample 31 can be a sample of sensor data reflecting the state of the subject (e.g., vital data obtained through a vital sensor, image data obtained through an image sensor, etc.). In this case, the classification task can be to identify the state of the subject (e.g., determining whether they are healthy, whether there are signs of disease, whether they are normal, etc.). The category of the feature can be related to the type of state of the subject. The correct label 32 can be configured to represent the state of the subject (examinee) represented by the corresponding training sample 31.

[0138] In this embodiment, the model generation apparatus 1 uses multiple training samples 31 contained in the acquired multiple first datasets 30 to perform machine learning on the encoder 5 and decoder 6. The encoder 5 is configured to convert the input samples into feature quantities (i.e., latent representations). The decoder 6 is configured to decode the samples based on the feature quantities. The machine learning of the encoder 5 and decoder 6 includes the following process: training the encoder 5 and decoder 6 such that the encoder 5 converts each training sample 31 into a feature quantity, and the decoder 6 decodes the samples based on the obtained feature quantities, which conform to each training sample 31. The result of the machine learning is the generation of a trained encoder 5 and decoder 6. Based on the trained encoder 5, each training sample 31 can be mapped to a feature space (latent space) while maintaining information related to the contained features. Based on the trained decoder 6, samples can be decoded based on the values ​​(feature quantities) of the feature space.

[0139] In this embodiment, the model generation device 1 uses the trained encoder 5 to convert two or more training samples 31 from a plurality of training samples 31 into feature quantities, thereby obtaining two or more sample points 311 corresponding to the two or more training samples 31 in the feature space. The model generation device 1 derives a linear sum 312 in the feature space according to a predetermined index. The linear sum 312 is the linear sum of the two or more sample points 311 obtained, and the sum of the distances calculated from the two or more sample points 311 is the largest linear sum. The model generation device 1 uses the trained decoder 6 to generate a decoded sample (i.e., a sample decoded based on the linear sum 312) corresponding to the feature quantity of the linear sum 312 as a new training sample 36.

[0140] In this embodiment, the model generation device 1 assigns a new positive solution label 37 to the new training sample 36. The new positive solution label 37 is configured to represent the category to which the features contained in the newly generated training sample 36 belong. Thus, the model generation device 1 generates a second dataset 35, which contains a combination of the new training sample 36 and the positive solution label 37. Alternatively, the first dataset 30 can be referred to as the original dataset, and the second dataset 35 as the new dataset. However, "original" and "new" simply refer to the relationship in a cycle of processing: generating the trained encoder 5 and decoder 6 through machine learning, and using the trained encoder 5 and decoder 6 to generate new training samples 36. Among the multiple first datasets 30, at least one of the previously generated second dataset 35 and datasets generated using other methods may be included.

[0141] The model generation device 1 uses multiple first datasets 30 and one or more second datasets 35 to implement machine learning for the classification model 7. The machine learning for the classification model 7 includes the following process: training the classification model 7 on each dataset (30, 35) so that the classification result of the classification model 7 on the categories to which the features contained in the training samples (31, 36) belong matches the correct label (32, 37). The result of the machine learning is that a trained classification model 7 is generated that has acquired the ability to perform the task of classifying the features contained in data samples of the same type as the training samples (32, 37). The generated trained classification model 7 can be provided to the classification device 2 at any time.

[0142] The classification device 2 is a computer configured to perform a classification task using a trained classification model 7. Specifically, the classification device 2 of this embodiment acquires object samples 221. Object samples 221 are data of the same type as training samples (31, 36). The classification device 2 includes a trained classification model 7 generated by the model generation device 1, and classifies the categories to which the features contained in the acquired object samples 221 belong by using the trained classification model 7. The classification device 2 outputs information related to the result of classifying the categories. In addition, the classification device 2 can be appropriately modified (e.g., a checking device, etc.) depending on the type of classification task.

[0143] As described above, according to the model generation apparatus 1 of this embodiment, the linear sum 312, which is the largest sum of distances from each sample point 311, is obtained as a new sample point, thereby enabling the sample points (sample point 311, linear sum 312) to exist everywhere in the feature space. Therefore, using the trained decoder 6, a new dataset (second dataset 35) containing the samples (training samples 36) decoded based on the feature quantity of linear sum 312 is added to the learning data, thereby obtaining learning data with high coverage and evaluable in the feature space. Moreover, according to the model generation apparatus 1 of this embodiment, at least a part of the process of generating such new training samples (dataset containing new training samples) can be automated by using the trained encoder 5 and decoder 6. Therefore, the cost of collecting training samples in a comprehensive manner can be reduced. Furthermore, in the model generation apparatus 1 of this embodiment, in addition to the original dataset (first dataset 30), the newly generated dataset (second dataset 35) is also used for machine learning of the classification model 7. Therefore, according to the model generation apparatus 1 of this embodiment, by using the training samples (31, 36) obtained in a comprehensive manner for machine learning of the classification model 7, the performance of the generated trained classification model 7 can be improved. In the classification apparatus 2 of this embodiment, by using the trained classification model 7 generated in this way, high-precision classification of the object sample 221 can be expected.

[0144] in addition, Figure 1 In this example, the model generation device 1 and the classification device 2 are interconnected via a network. The type of network can be appropriately selected from, for example, the Internet, wireless communication networks, mobile communication networks, telephone networks, private networks, etc. However, the method for exchanging data between the model generation device 1 and the classification device 2 is not limited to this example and can be appropriately selected depending on the implementation method. For example, data can be exchanged between the model generation device 1 and the classification device 2 using a storage medium.

[0145] and, Figure 1 In the example, the model generation device 1 and the classification device 2 each include a separate computer. However, the structure of the classification system 100 in this embodiment is not limited to this example and can be appropriately determined according to the embodiment. For example, the model generation device 1 and the classification device 2 may also be an integrated computer. Moreover, for example, at least one of the model generation device 1 and the classification device 2 may include multiple computers.

[0146] §2 Structural Examples

[0147] [Hardware Structure]

[0148] <Model Generation Device>

[0149] Figure 2 An example of the hardware structure of the model generation apparatus 1 in this embodiment is illustrated schematically. For example... Figure 2 As shown, the model generation apparatus 1 of this embodiment is a computer electrically connected together, comprising a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a driver 17. Furthermore, Figure 2 In this context, the communication interface and the external interface are referred to as "Communication I / F" and "External I / F".

[0150] The control unit 11 includes a central processing unit (CPU), random access memory (RAM), read-only memory (ROM), etc., which are hardware processors (processor resources), and is configured to perform information processing based on programs and various data. The storage unit 12 is an example of a memory (memory resource), such as a hard disk drive, solid-state drive, etc. In this embodiment, the storage unit 12 stores various information such as the model generation program 81, multiple first datasets 30, one or more second datasets 35, learning result data 125, etc.

[0151] The model generation program 81 is a program for causing the model generation device 1 to perform the following processes: machine learning of the encoder 5 and decoder 6, generation of new training samples 36 using the trained encoder 5 and decoder 6, and information processing related to the machine learning of the classification model 7 (described later). Figure 7A as well as Figure 7B The model generation program 81 contains a series of commands for the information processing. Additionally, the parts related to at least one of the information processing operations can be configured as independent programs. In this case, the subroutines related to machine learning of encoder 5 and decoder 6, and the subroutines related to machine learning of classification model 7, can be respectively referred to as model generation program, learning program, etc. The subroutines related to generating new training samples 36 using the trained encoder 5 and decoder 6 can be referred to as data generation program.

[0152] Multiple first datasets 30 are used in the machine learning of encoder 5 and decoder 6 and classification model 7. Furthermore, the multiple first datasets 30 used in the machine learning of encoder 5 and decoder 6 and the multiple first datasets 30 used in the machine learning of classification model 7 do not necessarily need to be exactly the same. Second dataset 35 is generated as a result of information processing related to the generation of new training samples 36 using the trained encoder 5 and decoder 6. Second dataset 35 is used in the machine learning of classification model 7. Learning result data 125 represents information related to the trained classification model 7 generated through machine learning. In this embodiment, learning result data 125 is generated as a result of executing model generation procedure 81. Details will be described later.

[0153] Communication interface 13, such as a wired local area network (LAN) module or a wireless LAN module, is an interface used for wired or wireless communication via a network. The model generation device 1 can utilize communication interface 13 to perform data communication via a network with other information processing devices. External interface 14, such as a Universal Serial Bus (USB) port or a dedicated port, is an interface used for connecting to external devices. The type and number of external interfaces 14 can be arbitrarily selected. In one example, training samples 31 can be obtained through a sensor. In another example, training samples 31 can be generated by another computer. In these cases, the model generation device 1 can be connected to the sensor or other computer via at least one of communication interface 13 and external interface 14.

[0154] The input device 15 is, for example, a mouse, keyboard, or other device for input. The output device 16 is, for example, a display, speaker, or other device for output. Users or operators can operate the model generation device 1 by utilizing the input device 15 and the output device 16.

[0155] The drive 17, such as a compact disc (CD) drive or a digital versatile disc (DVD) drive, is a drive device used to read various information, such as programs, stored in the storage medium 91. The storage medium 91 is a medium that preserves the stored programs and other information through electrical, magnetic, optical, mechanical, or chemical means, in a manner that allows a computer or other device or machine to read them. At least one of the model generation program 81, the first dataset 30, and the second dataset 35 may also be stored in the storage medium 91. The model generation apparatus 1 can also read at least one of the model generation program 81, the first dataset 30, and the second dataset 35 from the storage medium 91. Furthermore, Figure 2In this example, storage medium 91 is illustrated with disc-type storage media such as CDs and DVDs. However, the type of storage medium 91 is not limited to disc-type and may be other types. Examples of storage media other than disc-type include semiconductor memories such as flash memory. The type of drive 17 can be arbitrarily selected depending on the type of storage medium 91.

[0156] Furthermore, the specific hardware structure of the model generation apparatus 1 can be appropriately modified by omitting, replacing, or adding components depending on the implementation method. For example, the control unit 11 may include multiple hardware processors. These hardware processors may include microprocessors, field-programmable gate arrays (FPGAs), digital signal processors (DSPs), etc. The storage unit 12 may also include the RAM and ROM contained in the control unit 11. At least one of the communication interface 13, external interface 14, input device 15, output device 16, and driver 17 may be omitted. The model generation apparatus 1 may also include multiple computers. In this case, the hardware structures of each computer may be identical or different. Besides information processing devices specifically designed for the services provided, the model generation apparatus 1 may also be a general-purpose server device, a personal computer (PC), etc.

[0157] <Classification Device>

[0158] Figure 3 An example of the hardware structure of the sorting device 2 in this embodiment is illustrated schematically. For example... Figure 3 As shown, the classification device 2 in this embodiment is a computer electrically connected to a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a driver 27.

[0159] The control unit 21, driver 27, and storage medium 92 of the classification device 2 can be configured similarly to the control unit 11, driver 17, and storage medium 91 of the model generation device 1. The control unit 21 includes a CPU, RAM, ROM, etc., which are hardware processors, and is configured to perform various information processing based on programs and data. The storage unit 22 includes, for example, a hard disk drive, a solid-state drive, etc. In this embodiment, the storage unit 22 stores various information such as the classification program 82 and learning result data 125.

[0160] Classification procedure 82 is used to enable classification device 2 to perform classification tasks using the trained classification model 7, as described later. Figure 8The classification program 82 contains a series of commands for the information processing. At least one of the classification program 82 and the learning result data 125 may also be stored in the storage medium 92. The classification device 2 may also read at least one of the classification program 82 and the learning result data 125 from the storage medium 92.

[0161] Similar to the training sample 31, in one example, the object sample 221 can also be obtained by a sensor. In another example, the object sample 221 can be generated by another computer. In these cases, the classification device 2 can be connected to the sensor or other computer via at least one of the communication interface 23 and the external interface 24.

[0162] Furthermore, the specific hardware structure of the sorting device 2 can be appropriately modified by omitting, replacing, or adding components depending on the implementation method. For example, the control unit 21 may include multiple hardware processors. The hardware processors may include microprocessors, FPGAs, DSPs, etc. The storage unit 22 may also include the RAM and ROM contained in the control unit 21. At least one of the communication interface 23, external interface 24, input device 25, output device 26, and driver 27 may be omitted. The sorting device 2 may also include multiple computers. In this case, the hardware structure of each computer may be the same or different. Moreover, in addition to information processing devices specifically designed for the services provided, the sorting device 2 may also be a general-purpose server device, a general-purpose PC, an industrial PC, a tablet PC, a terminal device (e.g., a smartphone), a programmable logic controller (PLC), etc.

[0163] [Software Structure]

[0164] <Model Generation Device>

[0165] Figure 4 An example of the software structure of the model generation apparatus 1 according to this embodiment is illustrated schematically. The control unit 11 of the model generation apparatus 1 expands the model generation program 81 stored in the storage unit 12 into RAM. Furthermore, the control unit 11 uses a CPU to interpret and execute the commands contained in the model generation program 81 expanded in RAM to control each component. Thus, as... Figure 4 As shown, the model generation apparatus 1 of this embodiment operates as a computer comprising a data acquisition unit 111, a first training processing unit 112, a conversion unit 113, a point assignment unit 114, a sample generation unit 115, an output unit 116, a label assignment unit 117, a gap removal unit 118, a second training processing unit 119, and a storage processing unit 1110 as software modules. That is, in this embodiment, each software module of the model generation apparatus 1 is implemented by the control unit 11 (CPU).

[0166] The data acquisition unit 111 is configured to acquire multiple first datasets 30, each of which contains a combination of training samples 31 and positive solution labels 32. The training samples 31 contain features. The positive solution labels 32 are configured to indicate the category to which the features contained in the corresponding training sample 31 belong.

[0167] The first training processing unit 112 is configured to perform machine learning on the encoder 5 and decoder 6 using multiple training samples 31 contained in the acquired multiple first datasets 30. The encoder 5 is configured to accept input samples and convert the input samples into feature quantities. The decoder 6 is configured to accept input feature quantities and decode samples based on the input feature quantities. The machine learning on the encoder 5 and decoder 6 includes the following process: training the encoder 5 and decoder 6 such that the encoder 5 converts each training sample 31 into a feature quantity, and the decoder 6 decodes samples based on the obtained feature quantities that conform to each training sample 31.

[0168] The conversion unit 113 is configured to use the trained encoder 5 to convert two or more training samples 31 from a plurality of training samples 31 into feature quantities, thereby obtaining two or more sample points 311 in the feature space that correspond to each training sample 31. The point designation unit 114 is configured to derive a linear sum 312 in the feature space according to a predetermined index, wherein the linear sum 312 is the linear sum 312 of the two or more obtained sample points 311, and the sum of the distances calculated from each sample point 311 is the largest linear sum 312.

[0169] The sample generation unit 115 is configured to generate decoded samples corresponding to the feature quantities of the linear sum 312 as new training samples 36 using the trained decoder 6. The output unit 116 is configured to, after generating the new training sample 36, configure two or more training samples 31 and the new training sample 36 according to their positional relationship in the feature space and then output them. The label assignment unit 117 is configured to assign a new positive solution label 37 to the new training sample 36, thereby generating a second dataset 35, wherein the new positive solution label 37 represents the category to which the features contained in the generated new training sample 36 belong.

[0170] The interleaving unit 118 is configured to, when the trained encoder 5 converts each training sample (31, 36) contained in multiple first datasets 30 and one or more second datasets 35 into feature quantities, determine whether the density of sample points corresponding to each training sample (31, 36) in the feature space exceeds a threshold. When the density exceeds the threshold, the interleaving object is selected from one or more second datasets 35.

[0171] The second training processing unit 119 is configured to perform machine learning on the classification model 7 using multiple first datasets 30 and one or more generated second datasets 35. When the selection unit 118 selects a selection object, the second training processing unit 119 is configured to perform machine learning on the classification model 7, excluding the second dataset 35 selected as the selection object. The machine learning on the classification model 7 includes the following process: training the classification model 7 on each dataset (30, 35) such that the classification result of the classification model 7 on the categories to which the features contained in the training samples (31, 36) belong matches the correct label (32, 37). The storage processing unit 1110 generates information related to the trained classification model 7 generated through machine learning as learning result data 125. Furthermore, the storage processing unit 1110 saves the generated learning result data 125 to a designated storage area.

[0172] (An example of a machine learning process)

[0173] Figure 5A This is an example illustrating the training process of machine learning for encoder 5 and decoder 6, and machine learning for classification model 7. Figure 5B This is an illustrative example of a further training process for machine learning in encoder 5 and decoder 6.

[0174] like Figure 5A As shown, training samples 31 are used in the training of encoder 5 and decoder 6. First, the first training processing unit 112 inputs each training sample 31 into encoder 5 and performs computational processing on encoder 5. Based on the computational results of encoder 5, the first training processing unit 112 obtains the feature quantities corresponding to each training sample 31. Then, the first training processing unit 112 inputs the obtained feature quantities into decoder 6 and performs computational processing on decoder 6. Based on the computational results of decoder 6, the first training processing unit 112 obtains the samples decoded based on each feature quantity. Furthermore, the first training processing unit 112 calculates the error (reconstruction error) between each decoded sample and the corresponding training sample 31, and adjusts the values ​​of the parameters of encoder 5 and decoder 6 to reduce the calculated error. Thus, encoder 5 and decoder 6 can be trained so that encoder 5 converts each training sample 31 into feature quantities, and the samples decoded by decoder 6 based on the obtained feature quantities conform to each training sample 31.

[0175] In addition, in this embodiment, machine learning in encoder 5 and decoder 6 can also be implemented. Figure 5BThe illustrated adversarial learning. In this case, an inferrer 600 can be provided corresponding to the encoder 5 and the decoder 6. The inferrer 600 can be appropriately configured to infer the inner division ratio based on the decoded sample when the decoder 6 generates a decoded sample based on a synthetic feature quantity obtained by synthesizing the feature quantities corresponding to the two samples respectively with an arbitrary inner division ratio.

[0176] As an example of adversarial learning processing, firstly, the first training processing unit 112, for pairs (combinations) of training samples 31 obtained from multiple first datasets 30, acquires two feature quantities corresponding to the two training samples (31_1, 31_2) constituting each pair using the encoder 5. Next, the first training processing unit 112 synthesizes the two feature quantities acquired for each pair with a predetermined inner ratio (α: 1-α), thereby calculating a synthesized feature quantity for each pair. The predetermined inner ratio (the value of α) can be appropriately determined or changed for each pair / each iteration of training. Then, the first training processing unit 112 generates decoded samples for each pair using the decoder 6 based on the calculated synthesized feature quantity.

[0177] Furthermore, the first training processing unit 112 inputs the decoded samples generated for each pair into the specifier 600 and performs calculations on the specifier 600. As the result of the calculations of the specifier 600, the first training processing unit 112 obtains the result of speculating the inner ratio from the decoded samples for each pair. The first training processing unit 112 calculates the error (speculation error) between the obtained speculated inner ratio and the true value (α / 1-α) of the corresponding inner ratio, and adjusts the values ​​of the parameters of the specifier 600 to reduce the calculated error. Thus, the first training processing unit 112 trains the specifier 600 for each pair so that the result of specifier 600 speculating the inner ratio from the generated decoded samples conforms to the specified inner ratio.

[0178] Furthermore, the first training processing unit 112 inputs the decoded samples generated for each pair to the inferrer 600 and performs the calculation processing of the inferrer 600. As the result of the calculation of the inferrer 600, the first training processing unit 112 obtains the result of inferring the inner ratio from the decoded samples for each pair. The first training processing unit 112 adjusts the values ​​of the parameters of the encoder 5 and the decoder 6 to increase the error between the obtained inner ratio prediction result and the corresponding true value of the inner ratio. In one example, the first training processing unit 112 calculates the obtained inner ratio prediction result and a fixed value ( Figure 5BThe error between 0 and 0 is calculated, and the values ​​of the parameters of encoder 5 and decoder 6 are adjusted to reduce the calculated error. Thus, the first training processing unit 112 trains encoder 5 and decoder 6 for each pair to generate decoded samples that degrade the performance of the inferr 600 that infers the inner ratio (i.e., degrade the performance of the inferr 600 that infers the inner ratio from the generated decoded samples).

[0179] On the other hand, such as Figure 5A As shown, datasets (30, 35) are used in the training of classification model 7. In this embodiment, classification model 7 is configured to take the input of feature quantities obtained by encoder 5 and output the result of classifying the category to which the feature belongs based on the input feature quantities. That is, the operation process of the classification task is configured to include the operation of encoder 5. Therefore, firstly, the second training processing unit 119 inputs each training sample (31, 36) contained in each dataset (30, 35) to encoder 5 and performs the operation processing of encoder 5. Through the operation result of encoder 5, the second training processing unit 119 obtains the feature quantities corresponding to each training sample (31, 36). Then, the second training processing unit 119 inputs the obtained feature quantities to classification model 7 and performs the operation result of classification model 7. Through the operation result of classification model 7, the second training processing unit 119 obtains the result of classifying the category to which the feature contained in each training sample (31, 36) belongs. Furthermore, the second training processing unit 119 calculates the error (classification error) between the obtained classification result and the true value (correct classification) represented by the corresponding correct solution labels (32, 37), and adjusts the values ​​of the parameters of the classification model 7 to reduce the calculated error. Thus, the classification model 7 can be trained on each dataset (30, 35) so that the classification result of the classification model 7 on the categories to which the features contained in the training samples (31, 36) belong conforms to the correct solution labels (32, 37).

[0180] The learning result data 125 is configured to include information used for performing the classification task. In this embodiment, as described above, the classification model 7 is configured to utilize the computation results of the encoder 5, and correspondingly, the storage processing unit 1110 generates information for reproducing the trained encoder 5 and the trained classification model 7 as the learning result data 125. Furthermore, information related to other trained machine learning models (in this embodiment, the decoder 6 and the inferrer 600) may or may not be included in the learning result data 125.

[0181] A portion of the machine learning process for classification model 7 (specifically, a portion using the first dataset 30) can be performed concurrently with the machine learning processes of encoder 5 and decoder 6. At this time, the values ​​of the parameters of encoder 5 can also be adjusted during the machine learning process to reduce the classification error. Based on this structure, at least a portion of the training of classification model 7 can be performed concurrently with the training of encoder 5, thus improving the efficiency of the machine learning process for classification model 7.

[0182] However, the timing of machine learning in classification model 7 is not limited to this example. Machine learning in classification model 7 can also be performed independently of machine learning in encoder 5 and decoder 6. Furthermore, the structure of classification model 7 is not limited to the example described above. In another example, the computational steps of the classification task can be configured to not include computation in encoder 5. As a specific example, classification model 7 can be configured to accept input samples and output the result of classifying the category to which the features belong based on the input sample. In this case, information related to the trained encoder 5 can be omitted from the learning result data 125. Additionally, the output format of classification model 7 is not particularly limited as long as it can determine the result of classifying the category to which the features contained in the sample belong, and can be appropriately determined according to the implementation method. In one example, classification model 7 can be configured to output the posterior probability of the features contained in the sample for each category, as the result of classifying the features contained in the input sample.

[0183] (An example of the process of generating new training samples)

[0184] Figure 5C as well as Figure 5D This schematically illustrates an example of the process of generating new training samples 36. For example... Figure 5C As shown, firstly, the conversion unit 113 uses the trained encoder 5 to convert two or more training samples 31 from the multiple training samples 31 into feature values. Thus, the conversion unit 113 obtains two or more sample points 311 in the feature space that correspond to each training sample 31.

[0185] in addition, Figure 5C The example shown illustrates obtaining three sample points 311, but the number of sample points 311 obtained is not limited to three as long as there are two or more, and can be appropriately determined according to the implementation method. Furthermore, the two or more training samples 31 from which sample points 311 are obtained can be appropriately selected from multiple training samples 31. In one example, to improve the classification accuracy of the classification model 7, it is preferable to generate new training samples 36 within the range where there are no samples near the recognition boundaries of different categories. Therefore, it is preferable that the two or more training samples 31 are selected in a manner that includes training samples 31 associated with positive solution labels 32 representing different categories.

[0186] Next, the point designation unit 114 derives a linear sum 312_1 in the feature space according to a predetermined index. This linear sum 312_1 is the linear sum 312 of two or more acquired sample points 311, and the sum of the distances calculated from each sample point 311 is the largest linear sum 312_1. In this embodiment, this directly derived linear sum 312_1 can also be used to generate new training samples 36. That is, the sample generation unit 115 inputs the derived linear sum 312_1 to the trained decoder 6 and performs the computational processing of the trained decoder 6. Thus, the sample generation unit 115 can also generate decoded samples corresponding to the linear sum 312_1 as new training samples 36 by using the trained decoder 6.

[0187] In this embodiment, the classification model 7 can be configured to output the posterior probability of the features contained in the input sample for each category. The machine learning of the classification model 7 can then be implemented using a first machine learning dataset 30 and a second machine learning dataset 35. The first machine learning can be referred to as prior learning relative to the second machine learning. The second machine learning can be referred to as relearning or additional learning relative to the first machine learning. In the second machine learning, at least any one of the multiple first datasets 30 can also be used again.

[0188] like Figure 5C As shown, in this embodiment, the point designation unit 114 may also use the generated temporary, fully trained classification model 7 to evaluate the entropy of the decoded sample, and correct the linear sum 312_1 by increasing the entropy. Specifically, the point designation unit 114 uses the temporary, fully trained classification model 7 to calculate, for each category, the posterior probability of the features contained in the sample decoded by the trained decoder 6 based on the feature quantity of the derived linear sum 312_1. In this embodiment, the point designation unit 114 inputs the derived linear sum 312_1 into the temporary, fully trained classification model 7 and performs the operation processing of the temporary, fully trained classification model 7. Through the operation processing, the point designation unit 114 can obtain the result of calculating the posterior probability of the features contained in the decoded sample for each category. Next, the point designation unit 114 calculates the entropy of the decoded sample based on the calculated posterior probability of each category.

[0189] Entropy represents the degree to which the features contained in the decoded sample can be classified (in other words, uncertainty). Higher uncertainty indicates greater difficulty in classifying the features contained in the sample, meaning the machine learning of the sample in the temporarily trained classification model 7 is less sufficient. The point designation unit 114 can also correct the derived linear sum 312_1 based on the calculated gradient of the entropy to increase the entropy. Through this process, the point designation unit 114 can also obtain the corrected linear sum 312_2. In this embodiment, the corrected linear sum 312_2 can be used in the generation of new training samples 36. That is, the sample generation unit 115 can also input the corrected linear sum 312_2 into the trained decoder 6 and perform the computational processing of the trained decoder 6. Thus, the sample generation unit 115 can also generate decoded samples corresponding to the corrected linear sum 312_2 as new training samples 36 by using the trained decoder 6.

[0190] Furthermore, such as Figure 5D As shown, corresponding to the execution of the adversarial learning, the point designation unit 114 can also extract two sample points (311_1, 311_2) from the two or more sample points 311 acquired, which are near the linear sum 312_2 derived through the operation. Furthermore, the point designation unit 114 can also project the derived linear sum 312_2 onto the line connecting the two extracted sample points (311_1, 311_2) in the feature space, thereby obtaining a projection point 313 corresponding to the linear sum 312_2. Without performing the entropy-based correction, the point designation unit 114 can also extract two sample points (311_1, 311_2) near the linear sum 312_1, obtaining a projection point 313 corresponding to the linear sum 312_1.

[0191] Alternatively, the extraction of two sample points (311_1, 311_2) can be achieved by extracting two sample points from each of the corresponding training samples 31 whose features belong to different categories. That is, the point designation unit 114 can also extract two sample points (311_1, 311_2) from the sample points 311 located near the linear sum 312_2 (or the linear sum 312_1), which are training samples 31 whose ground values ​​are different from the categories represented by the associated positive solution label 32.

[0192] In this embodiment, the acquired projection point 313 can be used in the generation of new training samples 36. That is, the sample generation unit 115 can also input the value of the projection point 313 into the trained decoder 6 and perform the computational processing of the trained decoder 6. Thus, the sample generation unit 115 can also generate decoded samples corresponding to the feature values ​​of the projection point 313 as new training samples 36 by using the trained decoder 6.

[0193] (An example of output)

[0194] Figure 5E An example of the output format of the new training sample 36 is illustrated schematically. After the new training sample 36 is generated through the operation, the output unit 116 outputs the two or more training samples 31 and the new training sample 36 while maintaining their positional relationship in the feature space. Correspondingly, the label assignment unit 117 may be configured to, after the output unit 116 outputs two or more training samples 31 and the new training sample 36, accept the assignment of the category to which the features contained in the new training sample 36 belong (i.e., the new correct answer label 37), and assign the new training sample 36 a new correct answer label 37 that represents the assigned category.

[0195] in addition, Figure 5E In the example, three training samples 31 are shown. In one example, all training samples 31 used in the generation of the new training sample 36 could also be output. However, if a large number of training samples 31 are used in the generation of the new training sample 36, the output of training samples 31 may occupy most of the output resources (e.g., the display area of ​​the monitor), potentially limiting the area for outputting the new training sample 36. Therefore, in another example, the output unit 116 may select, for each category, one training sample 31 that is closest to the new training sample 36 in the feature space among the training samples 31 used in the generation of the new training sample 36, and output only the selected training samples 31 of each category along with the new training sample 36. This achieves efficiency in output resources. As a result, when comparing the training samples 31 of each category with the new training sample 36, each training sample (31, 36) can be easily viewed.

[0196] (An example of the structure of each model)

[0197] In this embodiment, encoder 5, decoder 6, classification model 7, and inferrer 600 are used as machine learning models (models that adjust parameter values ​​through machine learning). As long as the computational processing of encoder 5, decoder 6, classification model 7, and inferrer 600 can be performed, the types of machine learning models constituting each of them are not particularly limited and can be appropriately selected according to the implementation method.

[0198] In one example, such as Figure 5A as well as Figure 5B As shown, a neural network can be used for the machine learning model that constitutes encoder 5, decoder 6, classification model 7 and inferrer 600, respectively. Figure 5A as well as Figure 5BIn the example, encoder 5, decoder 6, classification model 7, and inferrer 600 include input layers (51, 61, 71, 601), one or more intermediate (hidden) layers (52, 62, 72, 602), and output layers (53, 63, 73, 603). The number of intermediate layers (52, 62, 72, 602) can be appropriately selected according to the implementation method. The number of intermediate layers (52, 62, 72, 602) can be different among encoder 5, decoder 6, classification model 7, and inferrer 600, or it can be the same among at least some of them. Intermediate layers (52, 62, 72, 602) can also be omitted. The number of neural networks constituting encoder 5, decoder 6, classification model 7, and inferrer 600 can be appropriately determined according to the implementation method.

[0199] Each layer (51-53, 61-63, 71-73, 601-603) includes one or more nodes (neurons). The number of nodes in each layer (51-53, 61-63, 71-73, 601-603) is not particularly limited and can be appropriately selected according to the implementation method. Each node in each layer (51-53, 61-63, 71-73, 601-603) can be combined with all nodes in adjacent layers. Thus, the encoder 5, decoder 6, classification model 7, and inferrer 600 can contain a fully associative neural network. However, the combination relationship of the nodes is not limited to this example and can be appropriately determined according to the implementation method. For example, each node can also be connected to specific nodes in adjacent layers, or to nodes in layers other than adjacent layers.

[0200] For each combination in each layer (51-53, 61-63, 71-73, 601-603), weights (combination loads) are set. For each node, a threshold is set; essentially, the output of each node is determined by whether the sum of the products of each input and each weight exceeds the threshold. The threshold can also be expressed using an activation function. In this case, the sum of the products of each input and each weight is input to the activation function, and the activation function operation is performed to determine the output of each node. The type of activation function can be arbitrarily chosen. The weights of the combinations between nodes in each layer (51-53, 61-63, 71-73, 601-603) and the thresholds of each node are examples of parameters used in the respective computational processing of encoder 5, decoder 6, classification model 7, and inferrer 600.

[0201] When each machine learning model (encoder 5, decoder 6, classification model 7, and inferrer 600) contains a neural network, the computational processing of each machine learning model includes forward propagation computational processing. That is, in the computational processing of each machine learning model, for the input data of each input layer (51, 61, 71, 601), the activation determination and other computational processing of the neurons contained in each layer (51-53, 61-63, 71-73, 601-603) are executed sequentially from the input side. The result of the forward propagation computational processing is to obtain the execution result of the task of each machine learning model from the output layer (53, 63, 73, 603). In addition, in this embodiment, the input data for encoder 5 and inferrer 600 is samples, and the input data for decoder 6 and classification model 7 is features. Moreover, the task of encoder 5 is to convert samples into features. The task of decoder 6 is to decode samples based on features. The task of classification model 7 is to classify the categories of features contained in samples. The inferrer 600 infers the internal ratio of the decoded samples. In each of the machine learning processes, the parameters of each machine learning model are adjusted using backpropagation to minimize errors.

[0202] In this embodiment, the storage processing unit 1110 generates learning result data 125 for reproducing the trained encoder 5 and classification model 7 generated through machine learning. The structure of the learning result data 125 is not particularly limited as long as the trained encoder 5 and classification model 7 can be reproduced, and can be appropriately determined according to the embodiment. For example, the learning result data 125 may include information representing the values ​​of each parameter of the encoder 5 and classification model 7 obtained through the machine learning adjustments. Depending on the situation, the learning result data 125 may further include information representing the structure of the encoder 5 and classification model 7. The structure can be determined, for example, based on the number of layers in the neural network from the input layer to the output layer, the types of each layer, the number of nodes in each layer, and the connection relationships between nodes in adjacent layers. When the structure of each machine learning model is shared between devices, the information representing the structure of the encoder 5 and classification model 7 can be omitted.

[0203] <Classification Device>

[0204] Figure 6 An example of the software structure of the sorting device 2 in this embodiment is illustrated schematically. The control unit 21 of the sorting device 2 expands the sorting program 82 stored in the storage unit 22 into RAM. Furthermore, the control unit 21 interprets and executes the commands contained in the sorting program 82 expanded in RAM via the CPU, thereby controlling each component element. Thus, as Figure 6As shown, the classification device 2 in this embodiment operates as a computer comprising an acquisition unit 211, a classification unit 212, and an output unit 213 as software modules. That is, in this embodiment, each software module of the classification device 2 is implemented by the control unit 21 (CPU), just like the model generation device 1.

[0205] The acquisition unit 211 is configured to acquire object sample 221. The classification unit 212 includes the trained classification model 7 generated by the model generation device 1 by maintaining the learning result data 125. The classification unit 212 is configured to classify the categories to which the features contained in the acquired object sample 221 belong by using the trained classification model 7. In this embodiment, the classification unit 212 includes the trained encoder 5 and the classification model 7 by maintaining the learning result data 125. The classification unit 212 is configured to perform a classification task for the object sample 221 using the trained encoder 5 and the classification model 7. The output unit 213 is configured to output information related to the result of classifying the categories (i.e., performing the classification task).

[0206] <Other>

[0207] The software modules of the model generation device 1 and the classification device 2 will be described in detail in the operational examples described later. Furthermore, in this embodiment, an example has been described where each software module of the model generation device 1 and the classification device 2 is implemented using a general-purpose CPU. However, it is also possible that some or all of the software modules are implemented using one or more dedicated processors (e.g., graphics processors). Each module can also be implemented as a hardware module. Moreover, regarding the software structure of the model generation device 1 and the classification device 2, the software modules can be appropriately omitted, replaced, or added depending on the implementation method.

[0208] §3 Action Examples

[0209] [Model Generation Device]

[0210] Figure 7A and Figure 7B This is a flowchart illustrating an example of the processing flow related to machine learning and data generation performed by the model generation apparatus 1 of this embodiment. The processing flow of the model generation apparatus 1 described below is an example of a model generation method. However, the processing flow of the model generation apparatus 1 described below is only an example, and each step can be changed as much as possible. Moreover, for the following processing flow, steps can be appropriately omitted, replaced, or added according to the embodiment.

[0211] (Step S101)

[0212] In step S101, the control unit 11 operates as a data acquisition unit 111 to acquire multiple first datasets 30, each of which contains a combination of training samples 31 and correct answer labels 32.

[0213] Each first dataset 30 can be appropriately generated. For example, an environment for performing the classification task can be envisioned in real or virtual space, and each training sample 31 can be obtained by observing arbitrary objects in said environment. The observed objects can be appropriately selected according to the classification task. Any sensor can be used to obtain each training sample 31. As an example, if the classification task is to identify the type of objects reflected in an image, the training samples 31 can be obtained by using a camera to photograph the objects. Next, the result of performing the classification task on each training sample 31 is obtained, that is, the information representing the correct answer (true value) of the category to which the features contained in each training sample 31 belong is used as the correct answer label 32. In this scenario, the classification task can be performed manually by an operator or the like. And, the obtained correct answer label 32 is associated with the corresponding training sample 31. Thus, each first dataset 30 can be generated.

[0214] Each first dataset 30 can be generated automatically by computer operation or manually by operation involving at least a portion of the operator. Furthermore, the generation of each first dataset 30 can be performed by the model generation device 1 or by a computer other than the model generation device 1. When each first dataset 30 is generated by the model generation device 1, the control unit 11 automatically, or manually by an operator via the input device 15, executes the series of generation processes to obtain multiple first datasets 30. Conversely, when each first dataset 30 is generated by another computer, the control unit 11 obtains the multiple first datasets 30 generated by the other computer, for example, via a network, storage medium 91, etc. Alternatively, some first datasets 30 may be generated by the model generation device 1, while others may be generated by one or more other computers.

[0215] The number of items in the first dataset 30 to be acquired can be arbitrarily selected. When multiple first datasets 30 are acquired, the control unit 11 will proceed to the next step S102.

[0216] (Step S102)

[0217] In step S102, the control unit 11 operates as a first training processing unit 112, using multiple training samples 31 contained in the acquired multiple first datasets 30 to perform machine learning on the encoder 5 and decoder 6. In the machine learning of the encoder 5 and decoder 6, at least a portion of the training samples 31 acquired in step S101 are used.

[0218] As an example of machine learning processing, the control unit 11 first performs initial settings for the machine learning models that are the objects of machine learning processing. In this embodiment, the control unit 11 performs initial settings for the encoder 5, decoder 6, and inferrer 600. The initial values ​​of the structure and parameters of each machine learning model can be provided by a template or determined by operator input. Furthermore, in the case of relearning or additional learning, the control unit 11 can also perform initial settings for each machine learning model based on the learning result data obtained through past machine learning.

[0219] Next, the control unit 11 trains the encoder 5 and decoder 6 so that the encoder 5 converts each training sample 31 into a feature value, and the decoder 6 decodes the sample based on the obtained feature values ​​to match each training sample 31. Furthermore, in this embodiment, as adversarial learning, the control unit 11 trains the inferrer 600 so that the result of the inferrer 600 inferring the inner fraction ratio in the synthesized feature value from the decoded sample generated by the decoder 6 matches a predetermined inner fraction ratio. Furthermore, the control unit 11 trains the encoder 5 and decoder 6 to reduce the performance of the inferrer 600, which infers the inner fraction ratio from the generated decoded sample. In each training process, probabilistic gradient descent, mini-batch gradient descent, etc., can be used.

[0220] As an example of the training process, firstly, the control unit 11 inputs each training sample 31 to the input layer 51 of the encoder 5 and performs forward propagation operations on the encoder 5. Through the forward propagation operations of the encoder 5, the control unit 11 obtains an output from the output layer 53 corresponding to the result of converting each training sample 31 into a feature quantity. Next, the control unit 11 inputs the obtained feature quantities to the input layer 61 of the decoder 6 and performs forward propagation operations on the decoder 6. Through the forward propagation operations of the decoder 6, the control unit 11 obtains an output from the output layer 63 corresponding to the result of decoding samples based on each feature quantity. Next, the control unit 11 calculates the error (reconstruction error) between each decoded sample and its corresponding training sample 31. The control unit 11 then calculates the gradient of the calculated error and backpropagates the calculated error gradient using the error backpropagation method. Thus, starting from the output layer 63, the errors in the parameter values ​​of the encoder 5 and decoder 6 are calculated sequentially. Furthermore, the control unit 11 updates the values ​​of each parameter of the encoder 5 and the decoder 6 based on the calculated errors of each parameter, so as to reduce the sum of the calculated errors.

[0221] Furthermore, as adversarial learning, the control unit 11 inputs the two training samples (31_1, 31_2) constituting each pair to the input layer 51 of the encoder 5, and performs forward propagation operations on the encoder 5. Through these forward propagation operations, the control unit 11 obtains from the output layer 53 an output corresponding to the result of converting the two training samples (31_1, 31_2) constituting each pair into feature quantities. Preferably, each pair contains two training samples (31_1, 31_2) whose contained features belong to different categories (i.e., the ground truth values ​​represented by the forward label 32 are different). However, the structure of the pair is not limited to this example. Each pair may also contain two training samples (31_1, 31_2) whose contained features belong to the same category. Moreover, the pair creation can be performed either at the training sample 31 stage or at the feature quantity stage. Pair creation can be performed at any time until synthesis at the inner ratio.

[0222] The control unit 11 synthesizes the two feature quantities acquired for each pair using a predetermined inner ratio (α: 1-α), thereby calculating a synthesized feature quantity for each pair. The predetermined inner ratio (the value of α) can be appropriately determined or changed for each pair / each training iteration. In one example, the predetermined inner ratio can be determined randomly or according to predetermined rules. In another example, the predetermined inner ratio can be determined by the operator's specification. For example, the operator can specify multiple inner ratios, and any one of the specified inner ratios can be used as the predetermined inner ratio. The control unit 11 inputs the synthesized feature quantity to the input layer 61 of the decoder 6 and performs the forward propagation operation processing of the decoder 6. Through the forward propagation operation processing, the control unit 11 obtains from the output layer 63 an output corresponding to the result of decoding the sample based on the synthesized feature quantity of each pair.

[0223] The control unit 11 inputs the decoded samples generated for each pair to the input layer 601 of the specifier 600 and performs forward propagation processing of the specifier 600. Through the forward propagation processing, the control unit 11 obtains an output from the output layer 603 corresponding to the result of speculating the inner ratio from the decoded samples of each pair. The control unit 11 calculates the error (speculation error) between the obtained inner ratio speculation result and the true value (α / 1-α) of the corresponding predetermined inner ratio. The control unit 11 then calculates the gradient of the calculated error and backpropagates the calculated error gradient using the error backpropagation method, thereby sequentially calculating the error values ​​of the specifier 600 parameters from the output layer 603. Furthermore, the control unit 11 updates the values ​​of the specifier 600 parameters based on the calculated parameter errors to reduce the sum of the calculated errors.

[0224] Furthermore, the control unit 11 inputs the decoded samples generated for each pair to the input layer 601 of the predictor 600, and performs forward propagation processing of the predictor 600. Through the forward propagation processing, the control unit 11 obtains an output from the output layer 603 corresponding to the result of predicting the inner fraction ratio from the decoded samples of each pair. The control unit 11 calculates the error based on the obtained inner fraction ratio prediction result in a manner that reduces the prediction accuracy of the predictor 600. In one example, the control unit 11 calculates the difference between the obtained inner fraction ratio prediction result and a fixed value ( Figure 5B The error is between 0 and 0. The control unit 11 then calculates the gradient of the calculated error and propagates the calculated error gradient back to the encoder 5 and decoder 6 via the estimator 600 using the error backpropagation method. Thus, the errors of the values ​​of the encoder 5 and decoder 6 are calculated sequentially from the output layer 63 of the decoder 6. Furthermore, the control unit 11 updates the values ​​of each parameter of the encoder 5 and decoder 6 based on the calculated errors of each parameter, so that the predicted result of the inner ratio deviates from the true value (in the case of calculating the error between the value and the fixed value, the sum of the calculated errors is reduced). In another example, the control unit 11 can also calculate the error between the obtained predicted result of the inner ratio and the corresponding predetermined true value of the inner ratio. Furthermore, the control unit 11 can also update the values ​​of each parameter of the encoder 5 and decoder 6 using the error backpropagation method, so that the sum of the calculated errors is increased.

[0225] In the calculation of the reconstruction error, the inference error, and the error that degrades the performance of the inferrer 600, a loss function can be used. The loss function can be appropriately set according to each task, the format of the correct solution, etc. Moreover, the degree to which the values ​​of each parameter are updated can be adjusted according to the learning rate. The learning rate can be given by the operator or as a predetermined value within the program. The control unit 11 can repeatedly adjust the values ​​of each parameter by means of each of the series of update processes until predetermined conditions are met, such as executing a predetermined number of times or the sum of the calculated errors being below a threshold.

[0226] By adjusting parameters based on reconstruction error, encoder 5 and decoder 6 are trained to acquire the ability to convert each training sample 31 into a feature value by encoder 5, and to decode samples that conform to each training sample 31 based on the acquired feature values ​​by decoder 6. By adjusting the parameters of the inferr 600 in adversarial learning, inferrer 600 is trained for each pair corresponding to the capabilities of encoder 5 and decoder 6 to acquire the ability to infer a specified inner fraction from the generated decoded samples. Furthermore, by adjusting the parameters of encoder 5 and decoder 6 in adversarial learning, encoder 5 and decoder 6 are trained for each pair corresponding to the capabilities of inferrer 600 to acquire the ability to generate decoded samples that degrade the performance of inferrer 600 in inferring the inner fraction (i.e., make it difficult to infer the inner fraction using inferrer 600).

[0227] The training of the inferrer 600 in adversarial learning can be performed alternately and repeatedly with the training of the encoder 5 and decoder 6. During this time, the computational processing to obtain the inference result of the inner ratio through the inferrer 600 can be performed for each training iteration. The series of processes for generating decoded samples from the two training samples 31 can be performed separately between training iterations or jointly between training iterations. The series of processes for generating the decoded samples can be performed for each iteration of training. During the training of the inferrer 600, the values ​​of the parameters of the encoder 5 and decoder 6 can be fixed; during the training of the encoder 5 and decoder 6, the values ​​of the parameters of the inferrer 600 can be fixed.

[0228] Alternatively, a gradient inversion layer (not shown) may be provided between decoder 6 and inferrer 600. The gradient inversion layer may be appropriately configured to allow values ​​to pass directly during forward propagation and to invert values ​​during backward propagation. In this case, the training of inferrer 600 can be performed concurrently with the training of encoder 5 and decoder 6. That is, the computational processing up to obtaining the inference result of the inner ratio based on the two training samples 31 can be performed jointly between each training iteration. The series of processes for generating the decoded samples can be performed for each iteration of training.

[0229] Moreover, in one case, training for adversarial learning ( Figure 5B It can be compared with training based on reconstruction error. Figure 5A Simultaneously. In this case, the computational processing for calculating the feature quantities of each pair can be performed concurrently with the computational processing of encoder 5 during training based on reconstruction error. Alternatively, adversarial learning training can be performed separately from training based on reconstruction error.

[0230] Furthermore, the machine learning of encoder 5 and decoder 6 is not limited to training based on the aforementioned errors. In one example, in addition to updating based on the aforementioned errors, control unit 11 can also calculate the error between the feature quantity obtained from encoder 5 and the value obtained according to a predetermined probability distribution (e.g., Gaussian distribution), and further adjust the values ​​of the encoder 5's parameters to reduce the sum of the calculated errors. Thus, control unit 11 can also normalize the output value of encoder 5.

[0231] The result of the machine learning is that the control unit 11 can generate the trained encoder 5 and decoder 6. When the machine learning process is complete, the control unit 11 will proceed to the next step S103.

[0232] (Step S103)

[0233] In step S103, the control unit 11 operates as a second training processing unit 119, using multiple first datasets 30 to implement machine learning for the classification model 7. This machine learning in step S103 is an example of the first machine learning of the classification model 7. Furthermore, the multiple first datasets 30 used in the machine learning of the classification model 7 may not be the same as the multiple first datasets 30 (training samples 31) used in the machine learning of the encoder 5 and decoder 6. A portion of the first datasets 30 used in the machine learning of the classification model 7 may not be used in the machine learning of the encoder 5 and decoder 6.

[0234] Aside from the calculation of errors and the different aspects of the adjustment objects, the machine learning of classification model 7 can be performed in the same way as the machine learning of encoder 5 and decoder 6. For example, firstly, control unit 11 performs initial settings for classification model 7, which is the object of machine learning processing. The initial values ​​of the structure and parameters of classification model 7 can be provided by a template or determined by operator input. Furthermore, in the case of relearning or additional learning, control unit 11 can perform initial settings for classification model 7 based on learning result data obtained through past machine learning.

[0235] Next, the control unit 11 trains the classification model 7 on each of the first datasets 30, so that the encoder 5 converts the training samples 31 into feature values, and the classification model 7 classifies the categories of the features contained in the training samples 31 according to the obtained feature values, which conforms to the true value represented by the corresponding positive solution label 32. In the training process, probabilistic gradient descent, mini-batch gradient descent, etc., can be used.

[0236] As an example of training processing, firstly, the control unit 11 inputs the training samples 31 contained in each first dataset 30 to the input layer 51 of the encoder 5, and performs forward propagation operation processing of the encoder 5. Through the forward propagation operation of the encoder 5, the control unit 11 obtains an output from the output layer 53 corresponding to the result of converting each training sample 31 into a feature quantity. Next, the control unit 11 inputs the obtained feature quantities to the input layer 71 of the classification model 7, and performs forward propagation operation processing of the classification model 7. Through the forward propagation operation processing of the classification model 7, the control unit 11 obtains an output from the output layer 73 corresponding to the result of classifying the categories of the features contained in each training sample 31. Next, the control unit 11 calculates the error (classification error) between the obtained classification result and the true value represented by the corresponding positive solution label 32. A loss function can be used for error calculation. The loss function can be appropriately set. The control unit 11 then calculates the gradient of the calculated error and backpropagates the gradient using the error backpropagation method. This allows the error in the parameter values ​​of the classification model 7 to be calculated sequentially from the output layer 73. Furthermore, the control unit 11 updates the parameter values ​​of the classification model 7 based on the calculated parameter errors. The degree of parameter value updating can be adjusted according to the learning rate. The learning rate can be specified by the operator or set as a predetermined value within the program.

[0237] The control unit 11 adjusts the values ​​of the parameters of the classification model 7 for each of the first datasets 30 through a series of update processes, so as to reduce the sum of the calculated errors. The control unit 11 may repeatedly adjust the values ​​of each parameter through the series of update processes until predetermined conditions are met, such as executing a predetermined number of times or the sum of the calculated errors being below a threshold. The result of the machine learning is that a trained classification model 7 is generated that has the ability to classify the categories of features contained in the samples within the scope of the first datasets 30 used. That is, the control unit 11 can generate a temporarily trained classification model 7. When the machine learning process is completed, the control unit 11 advances the process to the next step S104.

[0238] Furthermore, the machine learning processing of the classification model 7 in step S103 can be performed simultaneously with the machine learning processing of the encoder 5 and decoder 6 in step S102. At this time, the process of converting each training sample 31 into feature values ​​by the encoder 5 can be performed concurrently between the machine learning processes in steps S102 and S103. Moreover, in the machine learning process of step S103, the control unit 11 can backpropagate the calculated error gradient to the encoder 5 to calculate the error in the values ​​of the encoder 5's parameters. Furthermore, the control unit 11 can also update the values ​​of the encoder 5's parameters based on the calculated parameter error. Thus, the control unit 11 can adjust the values ​​of the encoder 5's parameters together with the classification model 7 based on the classification error.

[0239] (Steps S104 to S105)

[0240] In step S104, the control unit 11 operates as a conversion unit 113, using the encoder 5 trained through the machine learning method to convert two or more training samples 31 from the plurality of training samples 31 into feature values. In this embodiment, the control unit 11 inputs two or more training samples 31 into the input layer 51 of the trained encoder 5 and performs forward propagation operations on the trained encoder 5. As a result, the control unit 11 acquires two or more sample points 311 in the feature space corresponding to each training sample 31. When two or more sample points 311 are acquired, the control unit 11 proceeds the processing to the next step S105.

[0241] Furthermore, the two or more training samples 31 used can be appropriately selected from multiple first datasets 30. Preferably, at least a portion of the two or more training samples 31 are selected in a manner where the categories to which their contained features belong are different. However, the method of selecting the training samples 31 used is not limited to this example. The two or more training samples 31 can also be selected in a manner where all the categories to which their contained features belong are the same.

[0242] In step S105, the control unit 11 operates as the point designation unit 114, deriving a linear sum 312(312_1) in the feature space according to a predetermined index. This linear sum 312(312_1) is the linear sum 312(312_1) of two or more acquired sample points 311, and the maximum linear sum 312(312_1) is the sum of the distances calculated from each sample point 311. The linear sum 312_1(X) can be derived from each sample point 311(x). i It is calculated in a manner that satisfies Equations 3 to 5 below.

[0243] [Number 3]

[0244] X=∑ i αi x i …(Equation 3)

[0245] [Number 4]

[0246] ∑ i α i =1…(Equation 4)

[0247] [Number 5]

[0248] α i ≥1…(Equation 5)

[0249] In addition, α i This represents the relationship between each sample point 311(x) i The coefficient vector of the values ​​of ).

[0250] Furthermore, the specified index can be appropriately set to derive a linear sum 312 (312_1) of sample points that moderately deviates from each sample point 311 and helps to form a state where the sample points exist everywhere in the feature space. In one example, the specified index may include either Equation 1 or Equation 2. When Equation 1 is used for the specified index, the control unit 11 calculates the centroid of each sample point 311 as a linear sum 312_1. When Equation 2 is used for the specified index, the control unit 11 calculates the Volonoi points of each sample point 311 as a linear sum 312_1. When the derivation of the linear sum 312 (312_1) is completed, the control unit 11 proceeds the process to the next step S106.

[0251] (Steps S106 to S108)

[0252] In step S106, the control unit 11 operates as the point designation unit 114, using the temporarily trained classification model 7 to calculate, for each category, the posterior probability of the features contained in the sample decoded by the trained decoder 6 based on the derived linear sum 312_1. In this embodiment, the control unit 11 inputs the derived linear sum 312_1 to the input layer 71 of the temporarily trained classification model 7 and performs forward propagation processing of the temporarily trained classification model 7. Through the forward propagation processing, the control unit 11 obtains from the output layer 73 an output corresponding to the result of calculating the posterior probability of the features contained in the sample decoded by the trained decoder 6 based on the derived linear sum 312_1 for each category (i.e., the classification result). When the posterior probability calculation is complete, the control unit 11 proceeds the processing to the next step S107.

[0253] In step S107, the control unit 11 operates as the point designation unit 114, and calculates the entropy of the decoded sample based on the calculated posterior probability of each category. The method for calculating the entropy is not particularly limited as long as a value representing the uncertainty of classification can be derived, and can be appropriately determined according to the implementation method. For example, the control unit 11 may also calculate the entropy L according to the following Equation 6.

[0254] [Number 6]

[0255] L=∑ j p j log p j …(Equation 6)

[0256] In addition, p j This represents the posterior probability of category j calculated by the temporarily trained classification model 7. The correspondence between the number of categories and the types can be appropriately set according to the implementation method. When the calculation of entropy L is completed, the control unit 11 will proceed to the next step S108.

[0257] In step S108, the control unit 11 operates as the point designation unit 114, and further calculates the gradient of the calculated entropy L. The control unit 11 calculates the gradient of the entropy L. The derived linear sum 312_1 is corrected to increase the entropy L. The amount of correction can be appropriately determined according to the implementation method. In one example, the correction amount of the linear sum 312_1 can be proportional to the value of the gradient of the entropy L and the correction rate. The correction rate can be given by the operator or as a predetermined value in the program. In another example, the control unit 11 can also correct the linear sum 312_1 to maximize the entropy L. As a result of the correction process, the control unit 11 can obtain the corrected linear sum 312_2. In addition, the control unit 11 can also derive the linear sum 312_2 in such a way that the relationship of Equations 4 and 5 also holds after correction. When the corrected linear sum 312_2 is obtained, the control unit 11 proceeds the process to the next step S109.

[0258] (Steps S109 to S110)

[0259] In step S109, the control unit 11 operates as the point designation unit 114, extracting two sample points (311_1, 311_2) from the two or more sample points 311 that are near the derived linear sum 312 (312_2).

[0260] In one example, when at least a portion of two or more training samples 31 are selected in a manner where the categories of their contained features are different, the control unit 11 may also extract two sample points 311 in each corresponding training sample 31 where the categories of their contained features are different, as two sample points (311_1, 311_2) near linearity 312 (312_2). In another example, the control unit 11 may also extract two sample points 311 in each corresponding training sample 31 where the categories of their contained features are the same.

[0261] Furthermore, the two nearby sample points (311_1, 311_2) may include the two sample points 311 that are closest to the derived linear sum 312 (312_2). Alternatively, the two nearby sample points (311_1, 311_2) may include any two sample points 311 located within a proximity range of the linear sum 312 (312_2) defined by a predetermined distance. The predetermined distance may be appropriately set according to the implementation method.

[0262] When the extraction of two sample points (311_1, 311_2) is completed, the control unit 11 will advance the processing to the next step S110.

[0263] In step S110, the control unit 11 operates as a point designation unit 114. In the feature space, it projects the derived linear sum 312 (312_2) onto the straight line connecting the two extracted sample points (311_1, 311_2), thereby obtaining the projection point 313 corresponding to the linear sum 312 (312_2). The projection point 313 is the foot of the perpendicular. That is, a perpendicular line is extended from the linear sum 312 (312_2) to the straight line connecting the two sample points (311_1, 311_2), and the intersection of the perpendicular line and the straight line is found, thereby calculating the projection point 313. Known methods can be used to calculate this projection point 313. When the calculation of the projection point 313 is completed, the control unit 11 proceeds to the next step S111.

[0264] (Step S111)

[0265] In step S111, the control unit 11 uses the trained decoder 6 to generate a decoded sample corresponding to the linear sum 312 (feature quantity of projection point 313) as a new training sample 36. In this embodiment, the control unit 11 inputs the value (feature quantity) of projection point 313 to the input layer 61 of the trained decoder 6 and performs forward propagation operation processing of the trained decoder 6. Through the forward propagation operation processing, the control unit 11 obtains the output corresponding to the result of decoding the sample based on the feature quantity of projection point 313 from the output layer 63. Thus, a decoded sample can be generated as a new training sample 36. When the generation of the new training sample 36 is completed, the control unit 11 advances the process to the next step S112.

[0266] (Step S112)

[0267] In step S112, the control unit 11 operates as the output unit 116, and outputs two or more training samples 31 and a new training sample 36 according to their positional relationship in the feature space.

[0268] As long as the positional relationship of each sample (31, 36) can be indicated, the output form and output target are not particularly limited and can be appropriately selected according to the implementation method. In one example, the output target can be a display. The display can be at least any of the output device 16 and other computer output devices. In this case, the control unit 11 can appropriately express the feature space in coordinate space in the display area of ​​the display, reflect the positional relationship on the feature space as coordinates to arrange each sample (31, 36), and draw the position of each sample (31, 36) at its coordinates. In another example, the output target can be a printing medium (e.g., paper). In this case, the control unit 11 can also output printing instructions to a printer (not shown), which reflect the positional relationship on the feature space as coordinates to arrange each sample (31, 36) and draw the position of each sample (31, 36) at its coordinates.

[0269] Furthermore, the output training samples 31 can be appropriately selected. In one example, the control unit 11 may output all training samples 31 used in generating the new training sample 36. In another example, the output of a portion of the training samples 31 used in generating the new training sample 36 may be omitted. For example, the control unit 11 may select, for each different category, one training sample 31 that is closest to the new training sample 36 in the feature space among the training samples 31 used in generating the new training sample 36. Moreover, the control unit 11 may output only the selected training samples 31 for each category along with the new training sample 36. According to this method, the number of output training samples 31 can be reduced, thus achieving efficiency in output resources. As a result, when comparing the training samples 31 for each category with the new training sample 36, each training sample (31, 36) can be easily viewed, thereby making it easier to determine the category to which the features contained in the training sample 36 belong.

[0270] When the output of each sample (31, 36) is completed, the control unit 11 will proceed to the next step S113.

[0271] (Step S113)

[0272] In step S113, the control unit 11 operates as a label assignment unit 117, generating a new positive solution label 37. This new positive solution label 37 is configured to represent the category to which the features contained in the newly generated training sample 36 belong. Furthermore, the control unit 11 assigns the generated new positive solution label 37 to the new training sample 36. Thus, the control unit 11 generates the second dataset 35.

[0273] In this embodiment, through the output of step S112, the operator can visually confirm the new training sample 36 and training sample 31, and confirm the category to which the features contained in the new training sample 36 belong. The control unit 11 can also handle the assignment of the category to which the features contained in the new training sample 36 belong, corresponding to the output of step S112. The method for assigning the category can be appropriately selected according to the implementation. In one example, the category can be directly assigned by the operator via the operation of the input device 15. In another example, the category can be selected from a list. In yet another example, the category can be assigned by selecting a training sample from the output training sample 31 that belongs to the same category as the assigned category. Furthermore, the control unit 11 can also generate a new correct answer label 37 configured to represent the assigned category, and assign (associate) the generated new correct answer label 37 to the new training sample 36. Thus, a second dataset 35 can be generated.

[0274] In this embodiment, the control unit 11 can generate one or more second datasets 35 through a cycle of steps S104 to S113. The number of second datasets 35 generated in one cycle is not particularly limited and can be appropriately set according to the embodiment. When the generation of one or more second datasets 35 is completed, the control unit 11 proceeds to the next step, S114.

[0275] (Step S114)

[0276] In step S114, the control unit 11 determines whether to repeat the series of processes from step S102 to step S113.

[0277] The repetition criteria can be appropriately set according to the implementation method. For example, whether to repeat a series of processes can be selected by the operator's specification. In this case, the control unit 11 can also determine whether to repeat the series of processes based on the operator's selection. As another example, the number of repetitions can also be set. The number of repetitions can be given arbitrarily, for example, by the operator's specification, a setting within the program, or other methods. In this case, the control unit 11 can also determine whether to repeat the series of processes based on whether the set number of times the series of processes has been performed has been reached. As yet another example, the number of items in the generated second dataset can also be set. The set number of items in the second dataset can be given arbitrarily, for example, by the operator's specification, a setting within the program, or other methods. In this case, the control unit 11 can also determine whether to repeat the series of processes based on whether the total number of items in the second dataset generated through the series of processes has reached the set number.

[0278] If it is determined that the series of processes will not be repeated, the control unit 11 will proceed to the next step S115. On the other hand, if it is determined that the series of processes will be repeated, the control unit 11 will return to step S102 and repeatedly execute the series of processes.

[0279] In the case of repeated processing, in step S102, the control unit 11 may also use one or more generated second datasets 35 (training samples 36) to perform machine learning on the encoder 5 and decoder 6 again, thereby updating the trained encoder 5 and decoder 6. In step S103, the control unit 11 may also use one or more generated second datasets 35 to perform machine learning on the classification model 7 again, thereby updating the temporarily trained classification model 7. Each retraining may be performed using only one or more generated second datasets 35, or it may be performed using at least a portion of multiple first datasets 30 together with one or more generated second datasets 35.

[0280] In step S104, the control unit 11 can use the updated, fully trained encoder 5 to acquire two or more sample points 311 corresponding to two or more training samples (31, 36) contained in multiple first datasets 30 and one or more second datasets 35. In step S105, the control unit 11 can also derive the linear sum 312 (312_1) of the acquired two or more sample points 311. In step S106, the control unit 11 can use the updated, temporarily trained classification model 7 to calculate, for each category, the posterior probability of the features contained in the sample decoded by the trained decoder 6 based on the derived linear sum 312_1. The processing in steps S107 to S110 can be performed in the same manner. In step S111, the control unit 11 can also use the updated, fully trained decoder 6 to generate a decoded sample corresponding to the linear sum 312 (feature quantity of projection point 313) as a new training sample 36. In step S112, the control unit 11 may also configure two or more training samples selected from multiple training samples (31, 36) and the new training sample 36 generated in this loop according to their positional relationship in the feature space and then output them. In step S113, the control unit 11 may also assign a new positive solution label 37 to the newly generated training sample 36, thereby generating a new second dataset 35.

[0281] Furthermore, the repetitive processing is not limited to this example. In the case of a series of repeated processes, at least one of the retraining of encoder 5 and decoder 6 in step S102 and the retraining of classification model 7 in step S103 can be omitted. If the retraining of both is omitted, the control unit 11 can start executing the repeated processing from step S104. In this case, in subsequent processes, the trained encoder 5, trained decoder 6, and temporarily trained classification model 7 generated in the first process can be used.

[0282] (Steps S115 to S116)

[0283] In step S115, the control unit 11 operates as an interpolation unit 118, using the trained encoder 5 to convert each training sample (31, 36) contained in the multiple first datasets 30 and the generated second datasets 35 into feature quantities. The process of converting each training sample (31, 36) into feature quantities can be performed in the same way as in step S104. Moreover, at least a portion of the feature quantity conversion result can reuse the calculation result of step S104. Based on the conversion result, the control unit 11 calculates the density of sample points in the feature space corresponding to each training sample (31, 36). When the density calculation is completed, the control unit 11 advances the process to the next step S116.

[0284] In step S116, the control unit 11 determines the branch target of the processing based on the density calculation result of the processing in step S115. If the density exceeds the threshold, the control unit 11 proceeds the processing to the next step S117. On the other hand, if the density is less than the threshold, the control unit 11 skips the processing in step S117 and proceeds the processing to the next step S118. If the density equals the threshold, either branch target can be used.

[0285] Furthermore, the density calculation method performed in step S115 is not particularly limited as long as it can evaluate the number of sample points contained in any range, and can be appropriately determined according to the implementation method. For example, in step S115, the control unit 11 may also count the number of sample points in the feature space within a region of a predetermined size containing at least one sample point corresponding to the new training sample 36, thereby calculating the density. In step S116, the control unit 11 may also determine whether the counted number of sample points exceeds a threshold.

[0286] In another example, in step S115, the control unit 11 may also extract any two sample points in the feature space from the sample points obtained using the trained encoder 5, wherein at least one of them is a sample point corresponding to the new training sample 36. Furthermore, the control unit 11 may also calculate the distance D and cross-entropy H between the two extracted sample points according to the following equations 7 and 8 as density evaluation indicators.

[0287] [Number 7]

[0288] D = ||x1-x2|| < first threshold... (Equation 7)

[0289] [Number 8]

[0290] H = Σ j p 1j log p 2j <Second threshold...(Equation 8)

[0291] In addition, x1 and x2 represent the values ​​(features) of the two sample points respectively. 1j and p 2j This represents the posterior probability that the features contained in a sample, decoded by the trained decoder 6 from the values ​​of the two sample points, belong to class j. Posterior probability (p) 1j p 2j The result can be calculated using a temporary, trained classification model 7, similar to step S106.

[0292] At this point, in step S116, the control unit 11 can also determine that the density exceeds the threshold when the calculated distance D is less than the first threshold and the cross-entropy H is less than the second threshold. On the other hand, the control unit 11 can also determine that the density is less than the threshold when the calculated distance D exceeds the first threshold or the cross-entropy H exceeds the second threshold. As described above, when each indicator is equal to each threshold, either branch target can be processed. According to this evaluation method, the second dataset 35 can be excluded from dense regions using distance D and cross-entropy H as indicators.

[0293] (Step S117)

[0294] In step S117, the control unit 11 operates as a selection unit 118, selecting selection objects from those whose density exceeds a threshold. Here, the training samples 36 of the second dataset 35 are pseudo-samples generated by the trained decoder 6. Therefore, it is preferable that the selection objects are chosen from the second dataset 35. That is, it is preferable that the control unit 11 selects selection objects from one or more of the generated second datasets 35 within the range where the density exceeds the threshold.

[0295] The second dataset 35, which can be selected as the object of thinning, can be arbitrarily chosen. When the density is calculated within a region of the specified size, the control unit 11 can arbitrarily select the object of thinning from the second dataset 35 located within that region. When the distance D and cross-entropy H between the two sample points are calculated as density evaluation metrics, the control unit 11 can select the second dataset 35 from the two datasets corresponding to the two sample points as the object of thinning. When both corresponding datasets are second datasets 35, the control unit 11 can select either of the two second datasets 35 as the object of thinning.

[0296] However, the objects to be interleaved are not limited to the second dataset 35. In this step S117, the control unit 11 may also select the first dataset 30 as the object to be interleaved. In this case, in step S115, the control unit 11 can calculate the density of objects that do not contain new training samples 36. Moreover, the objects to be interleaved can be appropriately selected with a density less than a threshold. When an object to be interleaved is selected, the control unit 11 will proceed to the next step S118.

[0297] (Step S118)

[0298] In step S118, the control unit 11 operates as a second training processing unit 119, using one or more generated second datasets 35 to perform machine learning on the classification model 7. This machine learning in step S118 is an example of the second machine learning of the classification model 7.

[0299] The machine learning in step S118 can be performed using only one or more of the final generated second datasets 35, or it can be performed using at least a portion of multiple first datasets 30 together with one or more of the final generated second datasets 35. Furthermore, as part of the machine learning in step S118, the control unit 11 can also retrain the temporarily trained classification model 7. Alternatively, as part of the machine learning in step S118, the control unit 11 can also retrain the initialized classification model 7. In the case of retraining the initialized classification model 7, the control unit 11 uses at least a portion of the multiple first datasets 30 together with one or more of the final generated second datasets 35 for the machine learning in step S118.

[0300] If an object to be interpolated is selected through the processing in step S117, the control unit 11 performs machine learning on the classification model 7, excluding the second dataset 35 selected as the object to be interpolated. Except for including the finally generated second dataset 35 as learning data for machine learning, the machine learning process in step S118 is the same as in step S103. The result of the machine learning is that a trained classification model 7 can be generated. When the trained classification model 7 is generated, the control unit 11 proceeds to the next step, S119.

[0301] (Step S119)

[0302] In step S119, the control unit 11 operates as a storage processing unit 1110, generating information related to the trained classification model 7 generated through machine learning as learning result data 125. In this embodiment, the control unit 11 generates information related to the trained encoder 5 generated in step S102 and the classification model 7 generated in the processes up to step S118 as learning result data 125. Furthermore, the control unit 11 saves the generated learning result data 125 to a predetermined storage area.

[0303] The designated storage area may be, for example, RAM within the control unit 11, storage unit 12, external storage device, storage medium, or a combination thereof. Storage medium may be, for example, a CD, DVD, etc., and the control unit 11 may also save the learning result data 125 to the storage medium via drive 17. External storage device may be, for example, a data server such as Network Attached Storage (NAS). In this case, the control unit 11 may also use communication interface 13 to save the learning result data 125 to the data server via the network. Furthermore, the external storage device may be, for example, an external storage device connected to the model generation device 1 via external interface 14. Additionally, information related to the trained encoder 5 and information related to the trained classification model 7 may be saved as separate files. Each file may be saved to a different storage location.

[0304] When the learning result data 125 is saved, the control unit 11 ends the processing flow of the model generation device 1 related to this action example.

[0305] Furthermore, the generated learning result data 125 can be provided to the classification device 2 at any time. For example, the control unit 11 can transmit the learning result data 125 to the classification device 2 as part of or independently of step S103. The classification device 2 can also acquire the learning result data 125 by receiving the transmission. Moreover, for example, the classification device 2 can access the model generation device 1 or the data server via a network using the communication interface 23 to acquire the learning result data 125. Furthermore, for example, the classification device 2 can also acquire the learning result data 125 via the storage medium 92. Moreover, for example, the learning result data 125 can also be pre-loaded into the classification device 2.

[0306] Furthermore, the control unit 11 can also update or generate new learning result data 125 by periodically or irregularly repeating the processing of steps S101 to S119. During the repetition, at least a portion of the datasets (30, 35) used for machine learning can be modified, corrected, added, or deleted as appropriate. Moreover, the control unit 11 can also update the learning result data 125 held by the classification device 2 by providing the updated or newly generated learning result data 125 to the classification device 2 in any way.

[0307] [Sorting device]

[0308] Figure 8This is a flowchart illustrating an example of the processing flow related to the classification device 2 of this embodiment performing a classification task. The processing flow of the classification device 2 described below is an example of a classification method. However, the processing flow of the classification device 2 described below is only an example, and each step can be changed as much as possible. Moreover, for the following processing flow, steps can be appropriately omitted, replaced, or added according to the embodiment.

[0309] (Step S201)

[0310] In step S201, the control unit 21 operates as an acquisition unit 211 to acquire object samples 221. Object samples 221 are samples of data of a specified type that are the objects of a classification task. The method for acquiring object samples 221 can be appropriately determined depending on the type of data. For example, object samples 221 can be generated by observing objects through a sensor (e.g., a camera). Furthermore, the method for acquiring object samples 221 is not particularly limited and can be appropriately selected depending on the implementation method. In one example, the control unit 21 may directly acquire object samples 221 from a sensor or other device. In another example, the control unit 21 may indirectly acquire object samples 221 via other computers, storage media 92, etc. Any preprocessing can be applied to object samples 221. When object samples 221 are acquired, the control unit 21 proceeds the processing to the next step S202.

[0311] (Step S202)

[0312] In step S202, the control unit 21 operates as the classification unit 212. In this embodiment, the control unit 21 sets the trained encoder 5 and classification model 7 based on the learning result data 125. Furthermore, the control unit 21 uses the trained encoder 5 and classification model 7 to classify the categories to which the features contained in the object sample 221 belong. Specifically, the control unit 21 inputs the object sample 221 into the input layer 51 of the trained encoder 5 and performs forward propagation operations on the trained encoder 5 and classification model 7. As a result of this operation, the control unit 21 obtains an output from the output layer 73 of the trained classification model 7 corresponding to the classification result of the categories to which the features contained in the object sample 221 belong. The classification result can be appropriately obtained according to the output format of the classification model 7. In this embodiment, the control unit 21 extracts the category with the highest calculated posterior probability, thereby obtaining the classification result for the category of the object sample 221. When the classification process is complete, the control unit 21 proceeds to the next step S203.

[0313] (Step S203)

[0314] In step S203, the control unit 21 operates as the output unit 213, outputting information related to the results of classifying the categories.

[0315] The output target and the content of the information to be output can be appropriately determined according to the implementation method. For example, the control unit 21 may directly output the classification result obtained through step S202 to the output device 26. Moreover, the control unit 21 may also perform arbitrary information processing based on the obtained classification result. Furthermore, the control unit 21 may output the result of the information processing as information related to the classification result. The output of the result of the information processing may include controlling the operation of the controlled object device according to the classification result. The output target may be, for example, the output device 26, the output device of other computers, the controlled object device, etc.

[0316] When the output of information related to the classification result is completed, the control unit 21 terminates the processing flow of the classification device 2 associated with this action example. Alternatively, the control unit 21 may continuously and repeatedly execute the series of information processing steps S201 to S203. The timing of this repetition can be appropriately determined according to the implementation method. Thus, the classification device 2 can be configured to continuously and repeatedly perform the classification task.

[0317] [feature]

[0318] As described above, according to this embodiment, in the model generation apparatus 1, through the processing of step S105, a new sample point is obtained by obtaining the linear sum 312, which is the largest sum of distances from each sample point 311. This allows the sample points (sample point 311, linear sum 312) to exist everywhere in the feature space. Therefore, through the processing of steps S111 and S113, using the trained decoder 6, a new dataset (second dataset 35) containing samples (training samples 36) decoded based on the feature quantity of linear sum 312 is appended to the learning data, thereby obtaining learning data with high coverage and evaluable in the feature space. Furthermore, according to this embodiment, by using the trained encoder 5 and decoder 6 through a series of processes from steps S104 to S113, at least a portion of the process of generating such new training samples (the dataset containing the new training samples) can be automated. Therefore, the cost of comprehensively collecting training samples can be reduced. Furthermore, in this embodiment, in the processing of steps S103 and S118, in addition to the original dataset (first dataset 30), a new dataset (second dataset 35) generated through the processing is also used for machine learning of the classification model 7. Thus, by using the extensively obtained training samples (31, 36) for machine learning of the classification model 7, the performance of the trained classification model 7 can be improved.

[0319] Furthermore, in this embodiment, in the processing of step S103, the calculation method of Equation 1 or Equation 2 can be used as a method to calculate the linear sum 312 (312_1). According to these calculation methods, the sum of the distances from each sample point 311 to the inner region of each sample point 311 can be derived to become the maximum linear sum 312 (312_1). Thus, an appropriate linear sum 312 (312_1) can be obtained, and sample points can be added to regions where sample points are sparse in the feature space. Moreover, by repeating this operation, a state in which sample points are present everywhere can be formed. Therefore, a comprehensive collection of training samples can be appropriately collected. Furthermore, in the processing of steps S103 and S118, by using the comprehensive training samples (31, 36) for machine learning, the performance of the generated trained classification model can be improved.

[0320] Furthermore, in this embodiment, a temporary, fully trained classification model 7 is generated through the processing in step S103. Through the processing in steps S106 to S108, the derived linear sum 312_1 is corrected using the temporary, fully trained classification model 7 to increase entropy. And through the processing in step S111, samples decoded based on features derived from the corrected linear sum 312_2 are generated as new training samples 36 using the fully trained decoder 6. Entropy is calculated in a way that represents the uncertainty of whether classification is possible; therefore, by performing entropy-based correction as described above, training samples that are insufficiently machine learning-based can be generated in the temporary, fully trained classification model 7. Therefore, in the processing in step S118 (and also in the processing in step S103 if training samples 36 are used repeatedly), by using the obtained new training samples 36, the expected performance improvement of the generated fully trained classification model 7 can be enhanced.

[0321] Furthermore, in this embodiment, adversarial learning is performed between the encoder 5 and decoder 6 and the inferr 600 during the processing in step S102. Based on the training of the inferr 600, the ability to distinguish the inner fraction ratio is obtained corresponding to the capabilities of the encoder 5 and decoder 6 (i.e., the accuracy of distinguishing the inner fraction ratio is improved). On the other hand, based on the training of the encoder 5 and decoder 6, the ability to generate decoded samples that are difficult to predict the inner fraction ratio is obtained corresponding to the capabilities of the inferr 600. Through adversarial learning, the more the ability of the inferr 600 to distinguish the inner fraction ratio is improved, the more the ability of the encoder 5 and decoder 6 to generate samples that are difficult to predict the inner fraction ratio is improved. That is, regarding the points between each sample point, it is possible to suppress the decoding of samples that deviate from the real data where the inner fraction ratio is easily determined. Therefore, according to this embodiment, it is possible to generate a trained encoder 5 and decoder 6 that has obtained the ability to generate decoded samples similar to real data (training sample 31) from the linear sum 312. Thus, in the processing in step S111, it is possible to suppress the generation of training samples 36 that are extremely different from the real data. As a result, in the machine learning of classification model 7, by using the new training samples 36 obtained, the expected performance improvement of the generated trained classification model 7 can be enhanced.

[0322] Furthermore, in this embodiment, through steps S109 and S110, the linear sum 312(312_2) is converted into the inner ratio (projection point 313) of two sample points (311_1, 311_2), thereby correcting the linear sum 312(312_2) to a value within or near the training range of the adversarial learning. Thus, in the processing of step S111, the certainty of suppressing the addition of training samples 36 that deviate significantly from the real data to the learning data can be improved. As a result, in the machine learning of the classification model 7, by using the newly obtained training samples 36, the expected performance improvement of the generated, trained classification model 7 can be further enhanced.

[0323] Furthermore, in this embodiment, during the processing of step S109, two sample points 311 of different categories can be extracted as two sample points (311_1, 311_2). Therefore, during the processing of step S111, decoded samples similar to real data can be generated as new training samples 36 within a range closer to the category recognition boundary than each sample point 311. As a result, in the machine learning of the classification model 7, by using the obtained new training samples 36, the expected performance improvement of the generated, trained classification model 7 can be further enhanced.

[0324] Furthermore, in this embodiment, it is possible to determine whether a new training sample 36 can be added to the learning data based on the output obtained through the processing in step S112. This prevents training samples 36 that deviate significantly from the real data from being added to the learning data. Moreover, the category to which the features contained in the new training sample 36 belong can be determined based on the output and its positional relationship in the feature space with the training samples used during generation. Furthermore, in the processing of step S113, while determining the category of the new training sample 36 based on its positional relationship in the feature space, the category is assigned, thereby assigning a new correct label 37 representing the appropriate category to the new training sample 36. This prevents the assignment of an incorrect correct label 37 to the new training sample 36. These results mean that the obtained new training sample 36 can be appropriately used in the machine learning of the classification model 7, thereby increasing the expectation of improved performance of the generated, trained classification model 7.

[0325] Furthermore, when there is an excess of training samples (31, 36) in the vicinity of the space, a portion of these training samples (31, 36) may not contribute significantly to improving the performance of the classification model 7. In this embodiment, through the processing in steps S115 to S117, the excess state of training samples (31, 36) in the feature space can be detected, and a portion of the excess training samples can be removed. This reduces the number of datasets (30, 35) used in the machine learning of the classification model 7, suppresses the performance degradation of the generated trained classification model 7, reduces the computational load of machine learning, and optimizes the efficiency of computational resources (processors, memory) consumed by machine learning.

[0326] Furthermore, in this embodiment, the processes of steps S102 to S113 can be repeatedly executed through the processing of step S114. When new training samples 36 are added and the new training samples 36 are further used to implement machine learning of encoder 5 and decoder 6, the feature space mapped by the trained encoder 5 may change. According to this embodiment, by repeatedly performing machine learning of encoder 5 and decoder 6 in step S102 and generating new training samples 36 in steps S104 to S111, it is possible to update encoder 5 while adding new training samples 36, thus forming a state where sample points are present everywhere. As a result, it is possible to collect more comprehensive training samples (31, 36). By using the comprehensive training samples (31, 36) in the machine learning of classification model 7, it is possible to improve the expected performance of the generated trained classification model 7.

[0327] In this embodiment, the classification device 2 uses the trained encoder 5 and classification model 7 generated through the above processing, thereby enabling the execution of a high-precision classification task.

[0328] §4 Variations

[0329] The embodiments of the present invention have been described in detail above, but the description up to the foregoing is merely illustrative in all respects. Various modifications or variations can be made without departing from the scope of the invention. For example, the following changes can be made. Furthermore, the same reference numerals are used below for the same constituent elements as in the described embodiments, and descriptions of the same points as in the described embodiments are appropriately omitted. The following variations can be appropriately combined.

[0330] <4.1>

[0331] The classification system 100 of the described embodiment is applicable to all scenarios where arbitrary classification tasks are performed on data of a specified type. Classification tasks may include, for example, identifying the type of object mapped in image data, determining whether a product mapped in image data has defects, or classifying the state of an object represented by sensor data. The types of data for training samples 31 and object samples 221 can be appropriately selected according to the classification task. The following examples illustrate variations that limit the applicable scenarios.

[0332] (A) Image recognition scenario

[0333] Figure 9 This illustration schematically demonstrates an example of an applicable scenario for the classification system 100A of the first variation. This variation is an example of applying the described embodiment to a scenario of recognizing objects mapped in image samples. The classification system 100A of this variation is an example of the classification system 100, including a model generation device 1 and a classification device 2A. Similar to the described embodiment, the model generation device 1 and the classification device 2A can be interconnected via a network.

[0334] The data of the specified type processed in this variation (training sample 31 and object sample 221A) are image samples (image data) that map to an object RA. Image samples can be obtained by capturing the object RA using a camera SA. The classification task is to identify the type of object RA mapped in the image samples. Identifying the type of object RA may include segmentation (extracting the area mapped to the object). The object RA can be a person or any object. The area mapped in the image sample can be a specific part of the object RA (e.g., a face) or the entire object RA. In the case of an object RA being a person, the identified object may be, for example, a part of the body such as a face. The type of person identified may be, for example, a presumed individual or a presumed body part (face, arm, foot, joint, etc.). The same applies to any object. Except for these limitations, the classification system 100A of this variation can be configured similarly to the classification system 100 of the described embodiment.

[0335] (Model generation device)

[0336] In this variant, the model generation device 1 acquires multiple first datasets 30 in step S101. Each of the multiple first datasets 30 contains a combination of training samples 31 and positive resolution labels 32. The training samples 31 contain image samples mapped to objects RA. The positive resolution labels 32 are configured to represent the type (positive resolution) of the object RA mapped in the training samples 31. Through the processing in step S102, the model generation device 1 uses the multiple training samples 31 to implement machine learning for the encoder 5 and decoder 6. Through the processing in step S103, the model generation device 1 uses the multiple first datasets 30 to implement machine learning for the classification model 7.

[0337] The model generation device 1 generates one or more second datasets 35 through the processes described in steps S104 to S113. Each of the second datasets 35 contains a combination of new training samples 36 and new correct answer labels 37. The new training samples 36 contain image samples mapped to objects RA. The new correct answer labels 37 are configured to represent the type (correct answer) of the object RA mapped to the corresponding new training sample 36. The model generation device 1 can repeat the processes described in steps S102 to S113 in the same manner as in the previous embodiment, through the process described in step S114. The model generation device 1 calculates the density of sample points through the processes described in steps S115 to S117. When the calculated density exceeds a threshold, selective objects can be chosen from one or more second datasets 35.

[0338] The model generation device 1 performs machine learning on the classification model 7 using one or more generated second datasets 35 through the processing in step S118. Through the processing up to step S118, a trained classification model 7 is generated, gaining the ability to identify the types of objects RA mapped in the image samples. In step S119, the model generation device 1 appropriately saves the information related to the trained encoder 5 and the classification model 7 as learning result data 125 to a designated storage area. The generated learning result data 125 can be provided to the classification device 2A at any time.

[0339] (Sorting device)

[0340] Classification device 2A is an example of classification device 2. The hardware and software structures of classification device 2A can be the same as those of classification device 2 in the described embodiment. In this variation, classification device 2A can be connected to camera SA via a communication interface or an external interface. Alternatively, camera SA can be connected to another computer, and classification device 2A can be connected to that other computer, thereby acquiring image samples from camera SA. Camera SA can be, for example, a general RGB camera, depth camera, infrared camera, etc. Camera SA can be appropriately configured in a location capable of capturing the object RA.

[0341] Classification device 2A performs information processing related to the classification task for the object RA through the same processing flow as classification device 2. Specifically, in step S201, classification device 2A acquires object sample 221A from camera SA. Object sample 221A contains image samples that map the object RA, which is the object to be identified. In step S202, classification device 2A uses the trained encoder 5 and classification model 7 to identify the type of object RA mapped in object sample 221A. Next, in step S203, classification device 2A outputs information related to the result of identifying the type of object RA. In one example, classification device 2A can directly output the result of identifying object RA to an output device. In another example, classification device 2A can perform arbitrary information processing based on the result of identifying object RA. For example, classification device 2A can also perform information processing such as unlocking if the identified object (object RA) is a specific individual.

[0342] (feature)

[0343] According to the first variation, in a scenario where a trained classification model 7 with image recognition capabilities is to be generated, the cost of extensively collecting training samples can be reduced. Furthermore, by using the extensively collected training samples (31, 36) for machine learning of the classification model 7, the image recognition performance of the generated trained classification model 7 can be improved. In the classification apparatus 2A, by using the generated trained classification model 7, high-precision image recognition can be expected.

[0344] (B) Visual inspection scenario

[0345] Figure 10 This illustration schematically demonstrates an example of an applicable scenario for the inspection system 100B of the second variation. This variation is an example of applying the described embodiment to a scenario where an appearance inspection of a product mapped onto an image sample is performed. The inspection system 100B of this variation is an example of the classification system 100, including a model generation device 1 and an inspection device 2B. Similar to the described embodiment, the model generation device 1 and the inspection device 2B can be interconnected via a network.

[0346] The data of the specified type processed in this variation (training sample 31 and object sample 221B) are image samples (image data) that map onto the product RB. The image samples can be obtained by taking pictures of the product RB using a camera SB. The classification task is to determine whether the product RB mapped onto the image sample has a defect. The product RB is an example of the object RA, and determining whether the product RB has a defect is an example of identifying the type of object RA. That is, in this variation, the category of features is related to the type of defect contained in the product RB. Determining whether a defect exists includes: identifying the presence or absence of a defect, estimating the probability that the product RB contains a defect, identifying the type of defect contained in the product RB (which may also include a type indicating "no defect"), extracting the range of defects contained in the product RB, or a combination thereof. Apart from these limitations, the inspection system 100B of this variation can be configured in the same way as the classification system 100 of the described embodiment.

[0347] In addition, Product RB can be, for example, a product that is transported on a manufacturing line, such as electronic machinery, electronic components, automotive parts, pharmaceuticals, and food. Electronic components can include, for example, substrates, surface-mount capacitors, liquid crystal displays, and relay windings. Automotive parts can include, for example, connecting rods, shafts, engine blocks, power window switches, and dashboards. Pharmaceuticals can include, for example, packaged tablets and unpackaged tablets. Product RB can be a final product generated after the manufacturing process is completed, an intermediate product generated during the manufacturing process, or an initial product prepared before the manufacturing process. Defects can include, for example, scratches, dirt, cracks, dents, burrs, color differences, and foreign matter contamination.

[0348] (Model generation device)

[0349] In this variant, the model generation device 1 acquires multiple first datasets 30 in step S101. Each of the multiple first datasets 30A contains a combination of training samples 31 and positive solution labels 32. The training samples 31 contain image samples mapped to products RB. The positive solution labels 32 are configured to indicate whether the product RB mapped to the training samples 31 has a defect (positive solution). Through the processing in step S102, the model generation device 1 uses the multiple training samples 31 to implement machine learning for the encoder 5 and decoder 6. Through the processing in step S103, the model generation device 1 uses the multiple first datasets 30 to implement machine learning for the classification model 7.

[0350] The model generation device 1 generates one or more second datasets 35 through the processes described in steps S104 to S113. Each of the second datasets 35 contains a combination of new training samples 36 and new positive solution labels 37. The new training samples 36 contain image samples mapped to products RB. The new positive solution labels 37 are configured to indicate whether the product RB mapped to the corresponding new training sample 36 has a defect (positive solution). The model generation device 1 can repeat the processes described in steps S102 to S113 in the same manner as in the previous embodiment, through the process described in step S114. The model generation device 1 calculates the density of sample points through the processes described in steps S115 to S117. When the calculated density exceeds a threshold, it can select isolated objects from one or more second datasets 35.

[0351] The model generation device 1, through the processing in step S118, uses one or more generated second datasets 35 to perform machine learning on the classification model 7. Through the processing up to step S118, a trained classification model 7 is generated, capable of determining whether a product RB mapped in an image sample has a defect. In step S119, the model generation device 1 appropriately saves the information related to the trained encoder 5 and the classification model 7 as learning result data 125 to a designated storage area. The generated learning result data 125 can be provided to the inspection device 2B at any time.

[0352] (Inspection device)

[0353] The inspection device 2B is an example of the sorting device 2. The hardware and software structures of the inspection device 2B can be the same as those of the sorting device 2 in the described embodiment. In this variation, the inspection device 2B can be connected to the camera SB via a communication interface or an external interface. Alternatively, the camera SB can be connected to another computer, and the inspection device 2B can be connected to that other computer, thereby acquiring image samples from the camera SB. The camera SB can be, for example, a general RGB camera, a depth camera, an infrared camera, etc. The camera SB can be appropriately positioned in a location capable of capturing images of the product RB. For example, the camera SB can be positioned near a conveyor device that transports the product RB.

[0354] Inspection device 2B performs information processing related to the appearance inspection of product RB through the same processing flow as classification device 2. Specifically, in step S201, inspection device 2B acquires object sample 221B from camera SB. Object sample 221B contains an image sample that maps to product RB, which is the object of inspection. In step S202, inspection device 2B uses the trained encoder 5 and classification model 7 to determine whether product RB mapped in object sample 221B has a defect. Next, in step S203, inspection device 2B outputs information related to the result of determining whether product RB has a defect. In one example, inspection device 2B may directly output the result of determining whether product RB has a defect to an output device. In another example, inspection device 2B may also output a warning to an output device if product RB is determined to contain a defect. The output target of this information is not limited to the output device of inspection device 2B, but may also be the output device of other computers. In another example, when the inspection device 2B is connected to the conveyor device that transports the product RB, the inspection device 2B can also control the conveyor device based on the judgment result to transport defective products and non-defective products on different lines.

[0355] (feature)

[0356] According to the second specific example, in a scenario where a trained classification model 7 capable of visual inspection is to be generated, the cost of extensively collecting training samples can be reduced. Furthermore, by using the extensively collected training samples (31, 36) for machine learning of the classification model 7, the performance of the generated trained classification model 7 in visual inspection can be improved. In the inspection apparatus 2B, by using the generated classification model 7, high-precision visual inspection can be expected.

[0357] (C) Scenarios for State Recognition

[0358] Figure 11This illustration schematically demonstrates an example of an applicable scenario for the identification system 100C of the third variation. This variation is an example of applying the described embodiment to a scenario where the state of an object RC represented by sensor data is identified. The identification system 100C of this variation is an example of the classification system 100, including a model generation device 1 and an identification device 2C. Similar to the described embodiment, the model generation device 1 and the identification device 2C can be interconnected via a network.

[0359] The data of the specified type processed in this variation (training data 31 and object sample 221C) are samples of sensor data (hereinafter also referred to as "data samples") generated by observing the object RC using a sensor SC. The sensor SC may be, for example, a camera, microphone, encoder, environmental sensor, vital signs sensor, medical examination device, vehicle sensor, home security sensor, etc. The classification task may be to classify the state of the object RC represented by the sensor data. Classification may include prediction (i.e., classifying the state from the time the sensor data was obtained to future states). Apart from these limitations, the identification system 100C of this variation may be configured similarly to the classification system 100 of the described embodiment.

[0360] Furthermore, the type of sensor SC can be appropriately selected based on the classification task. For example, if the object RC is a person, the state of the object RC can be the state of the person being classified. In this case, the sensor SC may include at least one of a camera, microphone, life sensor, and medical examination device. The state of the person being classified may be, for example, the probability of developing a specified disease, the probability of experiencing a change in physical health, or other health conditions. Alternatively, if the object is a driver of a vehicle, the state of the person being classified may be the driver's state (e.g., drowsiness, fatigue, composure, etc.).

[0361] As another example, if the object RC is industrial machinery, classifying the state of the object RC can be done by classifying (detecting or predicting) whether there is an anomaly in the industrial machinery. In this case, the sensor SC may include, for example, at least one of a microphone, an encoder, and an environmental sensor. Data samples may include encoder values ​​of the motor, temperature, operating sounds, etc.

[0362] As another example, the object RC is an object existing outside the vehicle, and classifying the state of the object RC can be done by classifying the conditions outside the vehicle. In this case, the sensor SC may include, for example, at least one of a camera and an onboard sensor. Classifying the conditions outside the vehicle may include classifying the attributes of objects existing outside the vehicle, classifying traffic congestion, classifying the risk of accidents, etc. Objects existing outside the vehicle may include, for example, roads, traffic lights, obstacles (people, objects), etc. Classifying the attributes of objects existing outside the vehicle may include classifying the occurrence of events such as the sudden appearance of people or vehicles, rapid acceleration, emergency braking, lane changes, etc.

[0363] As another example, the object RC may be an object existing in a specific location, such as outdoors or in a designated indoor space (e.g., inside a vinyl house), and the state of the object RC may be the state of the specific location. In this case, the sensor SC may include at least one of a camera, a microphone, and an environmental sensor. As a specific example, the object RC may be a plant, and the state of the specific location may be the planting status of the plant.

[0364] As another example, the object RC is, for instance, an object existing within a residence, and the state of the object RC can be classified as the state within the residence. In this case, the sensor SC may include, for example, at least one of a camera, a microphone, an environmental sensor, and a home security sensor.

[0365] (Model generation device)

[0366] In this variant, the model generation device 1 acquires multiple first datasets 30 in step S101. Each of the multiple first datasets 30 contains a combination of training samples 31 and positive solution labels 32. The training samples 31 contain data samples obtained by the sensor SC. The positive solution labels 32 are configured to represent the type (positive solution) of the state of the object RC represented by the training samples 31. Through the processing in step S102, the model generation device 1 uses the multiple training samples 31 to implement machine learning for the encoder 5 and decoder 6. Through the processing in step S103, the model generation device 1 uses the multiple first datasets 30 to implement machine learning for the classification model 7.

[0367] The model generation device 1 generates one or more second datasets 35 through the processes described in steps S104 to S113. Each of the second datasets 35 contains a combination of new training samples 36 and new positive solution labels 37. The new training samples 36 contain samples of sensor data. The new positive solution labels 37 are configured to represent the type (positive solution) of the RC state of the object represented by the corresponding new training sample 36. The model generation device 1 can repeat the processes described in steps S102 to S113 in the same manner as in the previous embodiment, through the process described in step S114. The model generation device 1 calculates the density of sample points through the processes described in steps S115 to S117. When the calculated density exceeds a threshold, an object can be selected from one or more second datasets 35.

[0368] The model generation device 1, through the processing in step S118, uses one or more generated second datasets 35 to perform machine learning on the classification model 7. Through the processing up to step S118, a trained classification model 7 is generated, possessing the ability to recognize the state of the object RC represented by the sensor data. In step S119, the model generation device 1 appropriately saves the information related to the trained encoder 5 and the classification model 7 as learning result data 125 to a designated storage area. The generated learning result data 125 can be provided to the recognition device 2C at any time.

[0369] (Identification device)

[0370] The identification device 2C is an example of the classification device 2. The hardware and software structures of the identification device 2C can be the same as those of the classification device 2 in the described embodiment. In this variation, the identification device 2C can be connected to the sensor SC via a communication interface or an external interface. Alternatively, the sensor SC can be connected to another computer, and the identification device 2C can be connected to that other computer, thereby allowing the acquisition of samples of sensor data from the sensor SC.

[0371] The identification device 2C performs information processing related to the state recognition of the object RC through the same processing flow as the classification device 2. That is, in step S201, the identification device 2C acquires an object sample 221C from the sensor SC. The object sample 221C contains a sample of sensor data generated by observing the object RC, which is the object whose state is being identified, using the sensor SC. In step S202, the identification device 2C uses the trained encoder 5 and the classification model 7 to classify the state of the object RC represented by the object sample 221C.

[0372] Next, in step S203, the identification device 2C outputs information related to the state of the classified object RC. In one example, the identification device 2C may directly output the state of the classified object RC to the output device. In another example, the identification device 2C may perform arbitrary information processing based on the state of the classified object RC. Specifically, if the state of the classified object RC is the health status of a person being classified, the identification device 2C may output a warning to inform the user of this situation when it determines that the person's health status is abnormal (e.g., the probability of a specified disease is classified as high). As another specific example, if the state of the classified object RC is the status of a driver being classified, the identification device 2C may output messages urging the driver to rest or prohibiting the switch from automatic to manual driving when the driver's drowsiness or fatigue level is classified as high. As yet another specific example, if the state of the classified object RC is whether there is an abnormality in the classified industrial machinery, the identification device 2C may output a warning to inform the user of this situation when it determines that the industrial machinery is abnormal or shows signs of abnormality. As another specific example, when the state of the classified object RC is the external condition of the classified vehicle, the identification device 2C can also determine the action command for the vehicle based on the classified external condition of the vehicle, and control the vehicle through the determined action command (for example, to temporarily stop the vehicle when a person is detected).

[0373] (feature)

[0374] According to the third specific example, in a scenario where a trained classification model 7 is to be generated that can identify the state of an object RC represented by sensor data, the cost of extensively collecting training samples can be reduced. Furthermore, by using the extensively obtained training samples (31, 36) for machine learning of the classification model 7, the state recognition performance of the generated trained classification model 7 can be improved. In the recognition device 2C, by using the generated classification model 7, high-precision state recognition can be expected.

[0375] <4.2>

[0376] In this embodiment, the model generation apparatus 1 is configured to perform all processes of machine learning by the encoder 5 and decoder 6, generation of new learning data (second dataset 35), and machine learning by the classification model 7. However, the apparatus structure for performing each process is not limited to this example. In another example, each process may be performed by one or more independent computers.

[0377] Figure 12This schematically illustrates an example of the structure of the model generation apparatus 1 in this variant. In this variant, the model generation apparatus 1 includes a first model generation apparatus 101, a data generation apparatus 102, and a second model generation apparatus 103. The first model generation apparatus 101 is one or more computers configured to perform machine learning-related processing with the encoder 5 and decoder 6. The data generation apparatus 102 is one or more computers configured to perform processing to generate new training samples 36. The second model generation apparatus 103 is one or more computers configured to perform machine learning-related processing with the classification model 7.

[0378] The hardware structure of each device (101, 102, 103) can be the same as that of the model generation device 1. Each device (101, 102, 103) can be directly connected. Alternatively, each device (101, 102, 103) can be connected via a network. In one example, data can be exchanged between the devices (101, 102, 103) through their connection. In another example, data can be exchanged between the devices (101, 102, 103) via storage media or the like.

[0379] The first model generation apparatus 101 operates as a computer, comprising a data acquisition unit 1111, a first training processing unit 112, and a storage processing unit 1112, by executing the machine learning-related parts of the model generation program 81 concerning the encoder 5 and decoder 6. The data generation apparatus 102 operates as a computer, comprising a data acquisition unit 1113, a transformation unit 113, a point assignment unit 114, a sample generation unit 115, an output unit 116, a label assignment unit 117, a gap removal unit 118, and a storage processing unit 1114, by executing the parts of the model generation program 81 concerning the generation of new training samples 36. The second model generation apparatus 103 operates as a computer, comprising a data acquisition unit 1115, a second training processing unit 119, and a storage processing unit 1116, by executing the machine learning-related parts of the model generation program 81 concerning the classification model 7. Therefore, in this modified example, in the processing flow of the model generation apparatus 1, the first model generation apparatus 101 performs the processing of step S102. The data generation device 102 performs the processing steps S104 to S117. The second model generation device 103 performs the processing steps S103 and S118.

[0380] Specifically, the control unit of the first model generation device 101 operates as a data acquisition unit 1111, acquiring multiple training samples 31. The acquisition of the multiple training samples 31 can be performed in the same manner as step S101. Then, the control unit of the first model generation device 101 operates as a first training processing unit 112, executing the processing of step S102. Thus, a trained encoder 5 and decoder 6 are generated. Furthermore, the control unit of the first model generation device 101 operates as a storage processing unit 1112, generating information related to the trained encoder 5 and decoder 6 as learning result data, and saving the generated learning result data to a predetermined storage area. Similar to step S119, the predetermined storage area can be appropriately selected. The generated learning result data can be provided to the data generation device 102 at any time. Moreover, the information related to the trained encoder 5 can be provided to the second model generation device 103 at any time.

[0381] The control unit of the second model generation apparatus 103 operates as a data acquisition unit 1115, acquiring multiple first datasets 30. The acquisition of the multiple first datasets 30 can be performed in the same manner as in step S101. Then, the control unit of the second model generation apparatus 103 operates as a second training processing unit 119, executing the processing described in step S103. This generates a temporary, fully trained classification model 7. Furthermore, the control unit of the second model generation apparatus 103 operates as a storage processing unit 1116, generating information related to the temporary, fully trained classification model 7 as learning result data, and saving the generated learning result data to a predetermined storage area. Similar to step S119, the predetermined storage area can be appropriately selected. The learning result data of the generated temporary, fully trained classification model 7 can be provided to the data generation apparatus 102 at any time.

[0382] The control unit of the data generation apparatus 102 operates as a data acquisition unit 1113, acquiring two or more training samples 31. The control unit of the data generation apparatus 102 operates as a conversion unit 113 to an interleaving unit 118, performing the processing steps S104 to S117 on the acquired two or more training samples 31. This generates one or more second datasets 35. The control unit of the data generation apparatus 102 operates as a storage processing unit 1114, saving the generated one or more second datasets 35 to a designated storage area. The processing flow of the data generation apparatus 102 is an example of a data generation method.

[0383] Through the processing in step S114, during repeated processing starting from step S102, the data generation device 102 can also provide one or more training samples 36 to the first model generation device 101, indicating the update of the trained encoder 5 and decoder 6. The control unit of the first model generation device 101 can also operate as a data acquisition unit 1111, acquiring one or more training samples 36 generated by the data generation device 102. The control unit of the first model generation device 101 operates as a first training processing unit 112, using one or more training samples 36 to execute the processing in step S102. Thus, the first model generation device 101 can also generate updated trained encoder 5 and decoder 6. The control unit of the first model generation device 101 can operate as a storage processing unit 1112, generating information related to the updated trained encoder 5 and decoder 6 as learning result data, and saving the generated learning result data to a specified storage area. The learning result data related to the updated trained encoder 5 and decoder 6 can be provided to the data generation device 102 at any time. Furthermore, information related to the updated and trained encoder 5 can be provided to the second model generation device 103 at any time.

[0384] Furthermore, the data generation device 102 can also provide one or more generated second datasets 35 to the second model generation device 103, instructing the temporary, trained classification model 7 to be updated. The control unit of the second model generation device 103 can also operate as a data acquisition unit 1115, acquiring one or more generated second datasets 35. The control unit of the second model generation device 103 operates as a second training processing unit 119, using one or more generated second datasets 35 to perform the processing of step S103. Thus, the second model generation device 103 can also generate an updated, temporarily trained classification model 7. The control unit of the second model generation device 103 can operate as a storage processing unit 1116, generating information related to the updated, temporarily trained classification model 7 as learning result data, and saving the generated learning result data to a designated storage area. The learning result data related to the updated, temporarily trained classification model 7 can be provided to the data generation device 102 at any time. After the data generation device 102 has acquired learning result data updated in any way from each device (101, 103), it can perform processing after step S104. Among the two or more training samples 31 acquired during the execution of the processing in step S104, there may be new training samples 36 generated up to the previous loop.

[0385] Furthermore, before executing step S118, the control unit of the second model generation device 103 operates as a data acquisition unit 1115, acquiring one or more second datasets 35 generated by the data generation device 102. The control unit of the second model generation device 103 operates as a second training processing unit 119, using the acquired one or more second datasets 35 to execute the processing of step S118. Thus, the second model generation device 103 generates a trained classification model 7. The control unit of the second model generation device 103 can operate as a storage processing unit 1116, generating information related to the trained classification model 7 as learning result data, and saving the generated learning result data to a designated storage area. The information related to the trained encoder 5 generated by the first model generation device 101, and the learning result data related to the trained classification model 7 generated by the second model generation device 103, can be provided to the classification device 2 at any time.

[0386] According to this modified example, the workload of each computational processing step can be distributed to each device (101, 102, 103). Furthermore, in the data generation device 102, the assignment of new correct answer labels 37 can be performed by another computer until the generation of new training samples 36 is completed. In this case, the label assignment unit 117 in the software structure of the data generation device 102 can be omitted. Step S113 can be omitted from the processing flow of the data generation device 102.

[0387] <4.3>

[0388] In the described embodiment, the classification model 7 is configured to take the feature values ​​obtained by the encoder 5 as input and output the classification result of the category to which the feature belongs based on the input feature values. However, the structure of the classification model 7 is not limited to this example. In another example, the computational steps of the classification task may be configured to not include the computation of the encoder 5.

[0389] Figure 13 This schematically illustrates an example of the structure of the classification model 70 in this variation. The classification model 70 is configured to accept input samples and output a classification result based on the category to which the features belong, according to the input sample. The type and structure of the machine learning model constituting the classification model 70 can be appropriately determined according to the implementation method. In one example, similar to the described implementation method, a neural network can be used for the classification model 70.

[0390] In this variation, similar to the embodiment described above, the machine learning of the classification model 70 can be performed in two steps: a first machine learning step (step S103) and a second machine learning step (step S118). In the machine learning step, the control unit 11 inputs the training samples (31, 36) contained in each dataset (30, 35) into the classification model 70 and performs forward propagation processing of the classification model 70. Through the forward propagation processing, the control unit 11 obtains an output from the classification model 70 corresponding to the classification result of the category of the features contained in each training sample 31. The control unit 11 calculates the error (classification error) between the obtained classification result and the true value represented by the corresponding correct answer label (32, 37). Apart from these aspects, the model generation apparatus 1 can perform the machine learning of the classification model 70 in steps S103 and S118 using the same method as in the embodiment described above. In step S119, the control unit 11 generates information related to the trained classification model 70 generated through machine learning as learning result data 125.

[0391] Furthermore, in this modified example, the classification model 70 can also be configured to output the posterior probability of the features contained in the sample for each category, as the result of classifying the features contained in the input sample. Correspondingly, a correction based on the linear sum 312 (312_1) based on entropy can be performed. In this case, in step S106, the control unit 11 uses the trained decoder 6 to decode the sample based on the feature quantity of the derived linear sum 312_1. That is, the control unit 11 inputs the derived linear sum 312_1 to the trained decoder 6 and performs forward propagation processing of the trained decoder 6. Through the forward propagation processing of the decoder 6, the control unit 11 generates a decoded sample based on the linear sum 312_1. Next, the control unit 11 inputs the obtained decoded sample to the temporary trained classification model 70 and performs forward propagation processing of the temporary trained classification model 70. Through the forward propagation operation of the classification model 70, the control unit 11 obtains from the temporarily trained classification model 70 an output corresponding to the result (i.e., the classification result) calculated for each category, which is the posterior probability of the features contained in the sample decoded by the trained decoder 6 based on the derived linear sum 312_1. In addition to these aspects, the model generation device 1 can, in the same manner as in the described embodiment, calculate the entropy and correct the derived linear sum 312_1 based on the gradient of the calculated entropy.

[0392] Furthermore, in this modified example, information related to the trained encoder 5 can be omitted from the learning result data 125. In the classification device 2, the control unit 21 refers to the learning result data 125 in step S202 to set the trained classification model 70. The control unit 21 inputs the acquired object sample 221 into the trained classification model 70 and performs forward propagation processing of the trained classification model 70. As a result of this processing, the control unit 21 can obtain an output from the classification model 70 corresponding to the classification result of the category to which the features contained in the object sample 221 belong.

[0393] According to this variation, the structure of classification model 70 is independent of the structure of encoder 5. Therefore, the structure of classification model 70 can be optimized for the classification task. Consequently, the computational efficiency of classification model 70 can be expected to be improved.

[0394] <4.4>

[0395] In the described embodiment, each machine learning model of encoder 5, decoder 6, classification model 7 (70), and inferrer 600 comprises a fully associative neural network. However, the type of neural network constituting each machine learning model is not limited to this example. In another example, at least one of the machine learning models may comprise a convolutional neural network, a recurrent neural network, etc. At least one of the neural networks constituting each machine learning model may also comprise, for example, other types of layers such as convolutional layers, pooling layers, normalization layers, and dropout layers.

[0396] Furthermore, in the described embodiments, the types of machine learning models constituting encoder 5, decoder 6, classification model 7 (70), and inferrer 600 are not limited to neural networks. The type of machine learning model is not particularly limited as long as the computational processing can be performed, and can be appropriately selected according to the implementation method. As another example, encoder 5 and decoder 6 may contain principal component vectors obtained through principal component analysis. Classification model 7 (70) may contain support vector machines. The machine learning method can be appropriately determined according to the type of machine learning model used.

[0397] <4.5>

[0398] In the described embodiment, the correction of the linear sum 312_1 based on entropy can be omitted. In this case, steps S106 to S108 can be omitted from the processing flow of the model generation device 1 (data generation device 102). In step S109, the control unit 11 can extract two sample points (311_1, 311_2) near the derived linear sum 312_1. Next, in step S110, the control unit 11 can project the derived linear sum 312_1 onto the straight line connecting the two extracted sample points (311_1, 311_2) in the feature space, thereby obtaining the projection point 313 corresponding to the derived linear sum 312_1. The machine learning in step S103 can be integrated into the machine learning in step S118. That is, the processing in step S103 can also be omitted, and in step S118, the control unit 11 uses multiple first datasets 30 and one or more generated second datasets 35 to perform machine learning on the classification model 7.

[0399] <4.6>

[0400] In this embodiment, the acquisition of projection points 313 through steps S109 and S110 can be omitted from the processing flow of the model generation device 1 (data generation device 102). Correspondingly, in the processing of step S111, the control unit 11 can also use the trained decoder 6 to acquire the samples decoded according to the modified linear sum 312_2 (or linear sum 312_1 if the processing of steps S106 to S108 is further omitted) as new training samples 36.

[0401] Furthermore, in the aforementioned embodiment, a specifier 600 is provided corresponding to the encoder 5 and decoder 6, and adversarial learning is performed between the encoder 5 and decoder 6 and the specifier 600. However, the structure of the machine learning for the encoder 5 and decoder 6 is not limited to this example. In the processing of step S102, adversarial learning can be omitted. In this case, the specifier 600 can be omitted.

[0402] <4.7>

[0403] In the described embodiment, the step S113 of assigning a new positive label 37 to the new training sample 36 can be performed at any time after the new training sample 36 is generated. Alternatively, the process of generating the new positive label 37 can be automated by appropriately inferring the category to which the features contained in the new training sample 36 belong. The category of the new training sample 36 can be inferred from the training samples 31 (31, 36) used in the generation of the new training sample 36. The inference method can be appropriately selected. For category inference, a temporarily trained classification model 7 (70) can be used.

[0404] Furthermore, after assigning a new correct answer label 37 to a new training sample 36 based on the category-based prediction result, the control unit 11 can accept corrections to the new correct answer label 37 assigned to the training sample 36 at any time. For example, the control unit 11 can also accept corrections to the new correct answer label 37 assigned to the training sample 36 after outputting the training sample 36 through step S112. The process of accepting such corrections is an example of the process specified by the acceptance category in step S113.

[0405] Furthermore, in the aforementioned embodiment, the processing of step S112 can be omitted from the processing flow of the model generation device 1 (data generation device 102). Correspondingly, the output unit 116 can be omitted from the software structure of the model generation device 1 (data generation device 102).

[0406] <4.8>

[0407] In the described embodiment, through the processing of steps S106 to S108, the derived linear sum 312_1 is corrected based on entropy, and the sample is decoded based on the corrected linear sum 312_2 (or its projection point 313), thereby attempting to generate new training samples 36 that are more conducive to improving the performance of classification model 7 (70). The metric for evaluating whether it contributes to such performance improvement is not limited to entropy and can be appropriately designed according to the implementation method.

[0408] As another example of an evaluation metric, reliability can be listed as the degree to which a distribution belongs to multiple training samples 31 (i.e., how close it is to the real data derived from training samples 31). The model generation device 1 may also, along with or replacing the entropy, correct the derived linear sum 312_1 based on reliability. Reliability can be calculated appropriately. Two methods for calculating reliability are illustrated below.

[0409] (I) First Method

[0410] The first method utilizes the recognizer in reliability assessment. In this case, the first training processing unit 112 may be configured to further implement machine learning of the recognizer using multiple training samples 31 contained in multiple first datasets 30. The machine learning of the recognizer can be configured such that the recognizer is trained to evaluate the degree to which an input sample belongs to the distribution of the multiple training samples 31. The training of this recognizer can be appropriately selected according to the implementation method.

[0411] Figure 14 This schematically illustrates an example of the machine learning process of the recognizer 650 in this variant. Figure 14In the example, the trainer 650 is constructed by implementing adversarial learning between the encoder 5 and decoder 6 and the trainer 650. The adversarial learning includes the steps of: training the trainer 650 using multiple samples decoded by the decoder 6 and multiple training samples 31 as input samples to identify the source of the input samples; and training the encoder 5 and decoder 6 such that the recognition performance of the trainer 650 decreases when samples decoded by the decoder 6 are input to the trainer 650. This adversarial learning is an example of training the trainer to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of multiple training samples 31. Additionally, Figure 14 For ease of explanation, the source of training sample 31 is represented as "true", and the source of decoded sample is represented as "false". The representation of the source of each sample can be changed as appropriate.

[0412] As an example of training processing, the control unit 11 inputs each training sample 31 to the encoder 5 and executes the computational processing of the encoder 5 and decoder 6. Through this computational processing, the control unit 11 generates decoded samples corresponding to each training sample 31. The control unit 11 inputs each sample (training sample 31 / decoded sample) to the recognizer 650 and executes the computation of the recognizer 650, thereby obtaining the result of recognizing the source of each sample from the recognizer 650. The control unit 11 adjusts the values ​​of the parameters of the recognizer 650 to reduce the error between the recognition result and the recognized true value (true / false). Thus, the recognizer 650 can be trained to acquire the ability to recognize the source of the input sample.

[0413] Furthermore, the control unit 11 inputs each training sample 31 to the encoder 5 and executes the computational processing of the encoder 5 and decoder 6. Through this computational processing, the control unit 11 generates decoded samples corresponding to each training sample 31. The control unit 11 inputs each obtained decoded sample to the recognizer 650 and executes the computation of the recognizer 650, thereby obtaining the result of recognizing the source of each decoded sample from the recognizer 650. The control unit 11 calculates the error in a manner that indicates the recognition result is incorrect (i.e., the more the source of the input sample is mistakenly identified as the training sample 31, the smaller the error), and adjusts the values ​​of the parameters of the encoder 5 and decoder 6 to reduce the calculated error. Thus, the encoder 5 and decoder 6 can be trained to generate decoded samples that degrade the recognition performance of the recognizer 650.

[0414] The machine learning model constituting the recognizer 650 can be of any type as long as it is capable of performing operations to identify the source of the input sample, and can be appropriately selected according to the implementation method. Figure 14In the example, the recognizer 650 comprises a fully associative neural network, including an input layer 651, one or more intermediate layers 652, and an output layer 653. Each layer 651-653 can be constructed similarly to each layer 51-53 of the encoder 5. Furthermore, the number of layers constituting the recognizer 650, the number of nodes in each layer, the associative relationships, and other structural elements can be appropriately determined according to the implementation method. During the training of the recognizer 650, similarly to the encoder 5, the control unit 11 performs backpropagation of the calculated error gradient using the error backpropagation method, thereby adjusting the values ​​of each parameter.

[0415] Similar to the adversarial learning with the inferrer 600, a gradient inversion layer (not shown) can be placed between the decoder 6 and the recognizer 650. In this case, the training of the recognizer 650 in adversarial learning can be performed concurrently with the training of the encoder 5 and decoder 6. Alternatively, the training of the recognizer 650 in adversarial learning can be performed alternately and repeatedly. During the training of the recognizer 650, the values ​​of the parameters of the encoder 5 and decoder 6 can be fixed, and during the training of the encoder 5 and decoder 6, the values ​​of the parameters of the recognizer 650 can be fixed. In this case, at least a portion of each training process (e.g., the process of generating decoded samples) can also be performed concurrently.

[0416] Furthermore, the type of neural network constituting the recognizer 650 is not limited to a fully associative neural network. Similar to the encoder 5, the recognizer 650 may include neural networks other than fully associative ones. The recognizer 650 may also include other types of layers. Moreover, the type of machine learning model constituting the recognizer 650 is not limited to a neural network. As long as it can perform the operation of recognizing the source of the input sample, the type of machine learning model constituting the recognizer 650 is not particularly limited and can be appropriately selected according to the implementation method. The machine learning method can be appropriately determined according to the type of machine learning model used.

[0417] The recognizer 650, trained through adversarial learning, acquires the ability to distinguish between training sample 31 and decoded samples (pseudo-samples) from decoder 6, depending on the degree of adversarial learning. Therefore, based on the trained recognizer 650, the reliability of the input sample can be evaluated based on the result of identifying the source of the input sample. That is, when the source of the input sample is training sample 31, the more errors the recognizer 650 makes with the decoded sample, the more difficult it is for the decoded sample to be distinguished from the real sample (training sample 31), and the higher the reliability of the decoded sample can be evaluated. On the other hand, when the source of the input sample is decoder 6, the more accurately the recognizer 650 identifies the decoded sample, the easier it is for the decoded sample to be distinguished from the real sample (training sample 31) (i.e., deviating from training sample 31), and the lower the reliability of the decoded sample can be evaluated. Therefore, based on the trained recognizer 650 generated through adversarial learning, the reliability of the samples decoded by the trained decoder 6 can be appropriately evaluated.

[0418] Furthermore, the method for training the recognizer 650 to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of multiple training samples 31 is not limited to the adversarial learning method described above, and can be appropriately selected depending on the implementation. In another example, the trained recognizer 650 may include principal component vectors obtained through principal component analysis. In this case, the ability of training the recognizer 650 to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of training samples 31 can be configured by performing principal component analysis on multiple training samples 31. To evaluate the reliability of the decoded sample, the control unit 11 can use the principal component vectors to project (i.e., compress) the decoded sample to a local space and then restore the decoded sample to its original space. The greater the difference between the original decoded sample before compression and the restored decoded sample, the more the decoded sample deviates from the distribution of the training samples 31, and the lower the reliability of the decoded sample can be evaluated. On the other hand, the smaller the difference between the original decoded sample before compression and the restored decoded sample, the more the decoded sample follows the distribution of the training samples 31, and the higher the reliability of the decoded sample can be evaluated.

[0419] In another example, the recognizer 650 may include a single-class recognizer (e.g., a single-class support vector machine). In this case, the recognizer 650 is trained to acquire the ability to evaluate the degree to which an input sample belongs to the distribution of training samples 31, for example, by solving a single-class recognition optimization problem using multiple training samples 31. The control unit 11 can use the trained single-class recognizer to evaluate the reliability of the decoded sample. Based on the single-class recognizer, it is possible to identify whether the decoded sample belongs to the distribution of training samples 31. That is, the more the decoded sample is identified as belonging to the distribution of training samples 31, the higher the reliability of the decoded sample can be evaluated; the more the decoded sample is identified as not belonging to the distribution of training samples 31, the lower the reliability of the decoded sample can be evaluated.

[0420] The control unit 11 can operate as the first training processing unit 112 at any time until the linear sum 312_1 is corrected through the processing in step S108, performing machine learning on the recognizer 650. If the machine learning of the recognizer 650 includes the adversarial learning, the control unit 11 can perform the machine learning of the recognizer 650 together with other training in step S102. If the processing is repeated from step S102 through the processing in step S114, the control unit 11 can also perform the machine learning of the recognizer 650 again. At this time, the control unit 11 can also further use one or more new training samples 36 obtained up to the previous loop in the machine learning of the recognizer 650. In the first method of this variation, the control unit 11 uses the trained recognizer 650 generated in this way to evaluate the reliability of the samples decoded by the linear sum 312.

[0421] Figure 15 This schematically illustrates an example of the data generation process in this modified example. Until the processing in step S108 is executed, the control unit 11 operates as a point designation unit 114, using the trained recognizer 650 to evaluate the degree to which the sample decoded by the trained decoder 6 according to the feature quantity of the derived linear sum 312_1 belongs to the distribution of multiple training samples 31. When the trained recognizer 650 is generated through the aforementioned adversarial learning, the evaluation of the degree to which the decoded sample belongs to the distribution of multiple training samples 31 is configured by identifying the source of the decoded sample using the trained recognizer 650. That is, the control unit 11 uses the trained decoder 6 to generate decoded samples based on the derived linear sum 312_1. The control unit 11 uses the trained recognizer 650 to identify the source of the obtained decoded sample. When the source of the input sample is the training sample 31, the more errors the recognizer 650 makes in the decoded sample, the higher the reliability assessment. When the source of the input sample is the decoder 6, the more accurately the recognizer 650 identifies the decoded sample, the lower the reliability assessment.

[0422] Subsequently, the control unit 11 operates as the point designation unit 114, and calculates the reliability of the samples decoded by the trained decoder 6 based on the evaluation of the degree to which the decoded samples belong to the distribution of the training samples 31. The reliability estimation method can be appropriately determined according to the implementation method, such that the higher the degree to which the decoded samples belong to the distribution of the training samples 31 is evaluated, the higher the reliability value becomes; and the lower the degree to which the decoded samples belong to the distribution of the training samples 31 is evaluated, the lower the reliability value becomes. In one example, the control unit 11 can calculate the evaluation value of the reliability R using the following equation 9.

[0423] [Number 9]

[0424] R = ||m - tr||…(Equation 9)

[0425] Additionally, m represents the recognition result of the trained recognizer 650. Tr represents the ground truth value of the recognition.

[0426] Furthermore, in step S108, the control unit 11 operates as the point designation unit 114, based on the calculated reliability gradient. The derived linear sum 312_1 is corrected to improve reliability. Therefore, the control unit 11 obtains the corrected linear sum 312_5. As an example of the correction method, the control unit 11 can also calculate the product S of the evaluated values ​​using the following equation 10.

[0427] [Number 10]

[0428] S = X × L × RL (Equation 10)

[0429] The control unit 11 can also further calculate the gradient of the product S. And based on the calculated gradient The correction amount is optimized to maximize the product S. Furthermore, Equation 10 assumes that the linear sum 312_1 is corrected based on entropy L along with the reliability R. If the correction based on entropy L is omitted, then entropy L can be omitted from the product S in Equation 10.

[0430] Figure 16 This is a diagram illustrating an example of the optimization method used to obtain the modified linear sum of 312_5 in this variation. (See diagram for example.) Figure 16 As shown, having obtained the values ​​of distance (X), entropy L, and reliability R, the control unit 11 can also base its calculations on the gradient. This allows the deriving of the maximum value 315 of the product S. The control unit 11 can acquire the point corresponding to the maximum value 315 as a corrected linear sum 312_5. For example... Figure 16 As shown, in the case of multiple maxima ( Figure 16 In the case of two values, the control unit 11 can use any one of the maximum values ​​as the corrected linear sum 312_5. Alternatively, if the correction based on entropy L is omitted, the control unit 11 can obtain the corrected linear sum 312_5 based on the distance (X) and the reliability R.

[0431] After obtaining the corrected linear sum 312_5, the control unit 11 can also obtain the projection point 313 corresponding to the corrected linear sum 312_5 through the processing of steps S109 to S110. Therefore, in the processing of step S111, the control unit 11 can generate a decoded sample as a new training sample 36 corresponding to the feature quantity of the projection point 323 of the corrected linear sum 312_5 by using the trained decoder 6. Alternatively, if the processing of steps S109 to S110 is omitted, the control unit 11 can generate a decoded sample as a new training sample 36 corresponding to the feature quantity of the corrected linear sum 312_5 by using the trained decoder 6 in the processing of step S111.

[0432] (II) Second Method

[0433] Figure 17 This schematically illustrates an example of the reliability R evaluation method and data generation process based on the second method of this variation. The second method utilizes the trained encoder 5 and decoder 6 in the reliability R evaluation. In the second method, until the processing of step S108 is executed, the control unit 11 operates as a point designation unit 114, evaluating the degree of difference between the samples decoded by the trained decoder 6 based on the derived linear sum 312_1 feature quantity and the training samples 31 contained in the plurality of first datasets 30.

[0434] In this variant, the control unit 11 uses the trained decoder 6 to decode the sample based on the derived linear sum 312_1. The evaluation of the degree of difference is performed by converting the decoded sample into feature values ​​using the trained encoder 5, and calculating the difference between the feature values ​​of the decoded sample and the feature values ​​of the linear sum 312_1. Alternatively, the evaluation of the degree of difference is performed by further using the trained decoder 6 to generate a re-decoded sample based on the feature values ​​of the decoded sample, and calculating the reconstruction error between the decoded sample and the generated re-decoded sample.

[0435] The trained encoder 5 and decoder 6 are generated using machine learning with training sample 31. Therefore, the more similar the decoded sample to training sample 31 is based on the derived linear sum 312_1, the smaller the differences (differences between features / reconstruction errors); conversely, the more the decoded sample deviates from training sample 31, the larger the differences. Thus, this method allows for the evaluation of the degree of difference between the decoded sample based on linear sum 312_1 and training sample 31.

[0436] The control unit 11 calculates the reliability R of the samples decoded by the trained decoder 6 based on the result of evaluating the degree of difference. The method for estimating the reliability R can be appropriately determined according to the implementation method, such that the greater the degree of difference is evaluated, the lower the value of reliability R becomes, and the smaller the degree of difference is evaluated, the greater the value of reliability R becomes. In one example, the control unit 11 can calculate the evaluated value of reliability R using the following formula 11.

[0437] [Number 11]

[0438] R = ||st|| 2 …(Equation 11)

[0439] s represents the decoded sample. t represents the re-decoded sample obtained by encoder 5 and decoder 6 re-encoding and decoding the decoded sample. When the sample is an image sample, the reliability R of Equation 11 is calculated by summing the squares of the differences in the brightness values ​​of each pixel.

[0440] Furthermore, in step S108, the control unit 11 operates as the point designation unit 114, based on the calculated reliability gradient. The derived linear sum 312_1 is corrected to improve reliability. Therefore, the control unit 11 obtains the corrected linear sum 312_5. The correction method can be the same as the first method. Similarly to the first method, the control unit 11 can also obtain the corrected linear sum 312_5 based on entropy L and reliability R. In another example, entropy L can be omitted, and the control unit 11 can also obtain the corrected linear sum 312_5 based on reliability R.

[0441] After obtaining the corrected linear sum 312_5, the control unit 11 can also obtain the projection point 313 corresponding to the corrected linear sum 312_5 through the processing of steps S109 to S110. Therefore, in the processing of step S111, the control unit 11 can generate a decoded sample as a new training sample 36 corresponding to the feature quantity of the projection point 323 of the corrected linear sum 312_5 by using the trained decoder 6. Alternatively, if the processing of steps S109 to S110 is omitted, the control unit 11 can generate a decoded sample as a new training sample 36 corresponding to the feature quantity of the corrected linear sum 312_5 by using the trained decoder 6 in the processing of step S111. The samples that can be appropriately encoded and decoded by the trained encoder 5 and decoder 6 are substantially contained within the distribution of the training samples, and the degree to which they are contained within the distribution can be evaluated according to the reliability R of Equation 11. Therefore, by using the reliability R for the metric, a new training sample 36 can be generated within the desired distribution.

[0442] (summary)

[0443] The sample decoded by the trained decoder 6 based on the feature quantities of points other than those corresponding to the training sample 31 in the feature space can be considered pseudo-data that mimics the real data (training sample). Depending on the situation, the pseudo-data may deviate significantly from the real data. According to the first method or the second method, the reliability of the decoded sample obtained based on the derived linear sum 312_1 can be evaluated, and the linear sum 312_1 can be corrected to improve the reliability. As a result, it is possible to suppress the addition of new training samples 36 that deviate significantly from the real data to the learning data. As a result, by using the obtained new training samples 36 for machine learning of the classification model 7 (70), it is possible to expect an improvement in the performance of the generated trained classification model 7 (70). In addition, the model generation device 1 can also employ both the first method and the second method simultaneously. That is, the control unit 11 can also calculate the reliability R by means of the evaluation of the recognizer 650 and the evaluation based on the differences. As a result, it is possible to further suppress the addition of new training samples 36 that deviate significantly from the real data to the learning data.

[0444] <4.9>

[0445] In the described embodiment, Equations 1 and 2 are listed as examples of the specified metrics for deriving the linear sum 312_1. However, the specified metrics are not limited to these. In another example, when the classification model 7 is configured to output the posterior probability of the features contained in the sample for each class, the control unit 11 may also use a temporarily trained classification model 7 to derive the point X that maximizes the minimum value of the cross-entropy H calculated according to Equation 12 below as the linear sum 312(312_1).

[0446] [Number 12]

[0447]

[0448] According to this method, it is possible to derive sample points that have different information from any sample point 311 as linear sums 312 (312_1).

[0449] Furthermore, in the aforementioned embodiment (modified example), the processing steps S115 to S117 can be omitted from the processing flow of the model generation device 1 (data generation device 102). In this case, the interruption part 118 can be omitted from the software structure of the model generation device 1 (data generation device 102).

[0450] Furthermore, in the aforementioned embodiment, the model generation device 1 can save the generated second dataset 35 to a designated storage area. The designated storage area can be appropriately selected depending on the embodiment.

Claims

1. A model generation apparatus, comprising: The data acquisition unit is configured to acquire multiple first datasets, each of which contains a combination of training samples and positive solution labels. The training samples are image samples that map to objects. The training samples contain features. The category of the feature is related to the type of object reflected in the image sample. The correct answer label is configured to represent the category to which the feature contained in the training sample belongs; The first training processing unit is configured to perform machine learning on the encoder and decoder using multiple training samples contained in the acquired multiple first datasets, and in the first training processing unit, The encoder is configured to convert samples into feature quantities. The decoder is configured to decode the sample based on the feature quantity. The machine learning of the encoder and decoder includes the following process: training the encoder and decoder such that the encoder converts each training sample into a feature value, and the decoder decodes the sample according to the obtained feature values, which conforms to each training sample. The conversion unit is configured to use the trained encoder to convert two or more training samples from the plurality of training samples into feature quantities, thereby obtaining two or more sample points in the feature space that correspond to each training sample respectively. The point-specified part is configured to derive a linear sum in the feature space according to a specified index, wherein the linear sum is the linear sum of the two or more sample points obtained, and the sum of the distances calculated from each sample point is the largest linear sum. The sample generation unit is configured to generate a decoded sample corresponding to the feature quantity of the linear sum as a new training sample by using the trained decoder. The labeling unit is configured to assign new positive solution labels to the new training samples, thereby generating a second dataset, wherein the new positive solution labels are configured to represent the category to which the features contained in the generated new training samples belong; as well as The second training processing unit is configured to perform machine learning on the classification model using the plurality of first datasets and one or more generated second datasets, and in the second training processing unit, The machine learning of the classification model includes the following process: training the classification model on each dataset such that the classification model classifies the categories to which the features contained in the training samples belong, and the result is consistent with the positive label. The object is a product, and the category of the feature is related to the type of defect contained in the product.

2. The model generation apparatus according to claim 1, wherein... The classification model is configured to output the posterior probability of the features contained in the sample for each category. The machine learning of the classification model is implemented by dividing it into a first machine learning using the multiple first datasets and a second machine learning using one or more second datasets. Through the implementation of the first machine learning method, a temporary, trained classification model is generated. The designated point portion is further configured as follows: Using the provisional, fully trained classification model, for each class, calculate the posterior probability of the features contained in the sample decoded by the trained decoder based on the derived linear sum of features. Based on the calculated posterior probability of each category, the entropy of the decoded sample is calculated, and The derived linear sum is corrected based on the calculated gradient of the entropy, so that the entropy increases. The sample generation unit is configured to generate, using the trained decoder, a decoded sample corresponding to the feature quantity of the modified linear sum as the new training sample.

3. The model generation apparatus according to claim 1 or 2, wherein... The first training processing unit is configured to further implement machine learning of the recognizer using multiple training samples contained in the plurality of first datasets. The machine learning of the recognizer is configured by training the recognizer to evaluate the degree to which an input sample belongs to the distribution of the plurality of training samples. The designated point portion is further configured as follows: Using the trained recognizer, the extent to which a sample decoded by the trained decoder based on the derived linear sum of features belongs to the distribution of the plurality of training samples is evaluated. Based on the evaluation of the aforementioned degree, the reliability of the samples decoded by the trained decoder is calculated, and The derived linear sum is corrected based on the calculated reliability gradient to improve the reliability. The sample generation unit is configured to generate, using the trained decoder, a decoded sample corresponding to the feature quantity of the modified linear sum as the new training sample.

4. The model generation apparatus according to claim 3, wherein The recognizer is trained by implementing adversarial learning between the encoder and the decoder and the recognizer. The adversarial learning includes the following steps: The recognizer is trained using multiple samples decoded by the decoder and the multiple training samples as input samples to identify the source of the input samples; and The encoder and the decoder are trained such that the recognition performance of the recognizer decreases when a sample decoded by the decoder is input into the recognizer. The evaluation of the degree to which the decoded sample belongs to the distribution of the plurality of training samples is constituted by identifying the source of the decoded sample using the trained recognizer.

5. The model generation apparatus according to claim 1 or 2, wherein... The designated point portion is further configured as follows: The degree of difference between the samples decoded by the trained decoder based on the derived linear sum of features and the training samples contained in the plurality of first datasets is evaluated. Based on the evaluation of the aforementioned degree, the reliability of the samples decoded by the trained decoder is calculated, and The derived linear sum is corrected based on the calculated reliability gradient to improve the reliability. The assessment of the degree of the aforementioned differences is constituted by the following method: The decoded samples are converted into feature values ​​by the trained encoder, and the difference between the obtained feature values ​​of the decoded samples and the feature values ​​of the linear sum is calculated; or The trained decoder is then used to generate a re-decoded sample from the features of the decoded sample, and the reconstruction error between the decoded sample and the generated re-decoded sample is calculated.

6. The model generation apparatus according to claim 1 or 2, wherein The machine learning in the encoder and decoder also includes: For a pair of training samples obtained from the plurality of first datasets, the encoder is used to obtain two feature quantities corresponding to the two training samples constituting each pair; The two feature quantities obtained for each pair are synthesized according to a specified inner ratio, thereby calculating the synthesized feature quantity for each pair; By using the decoder, decoded samples are generated for each pair based on the calculated synthetic feature quantity; The inferr is trained on each of the pairs such that the result of the inferr inferring the inner fraction based on the generated decoded samples conforms to the specified inner fraction. as well as The encoder and decoder are trained using the pairs mentioned above to generate decoded samples that degrade the performance of the inferr that infers the inner fraction.

7. The model generation apparatus according to claim 6, wherein... The point designation part is configured as follows: Extract two sample points from the two or more obtained sample points that are near the derived linear sum, and In the feature space, the derived linear sum is projected onto the straight line connecting the two extracted sample points, thereby obtaining the projection point corresponding to the linear sum. The sample generation unit is configured to generate, using the trained decoder, a decoded sample corresponding to the feature quantity of the projection point corresponding to the linear sum as the new training sample.

8. The model generation apparatus according to claim 7, wherein The two sample points are extracted by extracting two sample points from each training sample whose features belong to different categories.

9. The model generation apparatus according to claim 1 or 2, further comprising: The output unit is configured to, after generating the new training sample, configure the two or more training samples and the new training sample according to their positional relationship in the feature space and then output them.

10. The model generation apparatus according to claim 9, wherein The label-assigning part is configured as follows: After the output unit outputs the two or more training samples and the new training sample, it accepts the assignment of the category to which the features contained in the new training sample belong, and The new training samples are assigned new correct answer labels that represent the specified category.

11. The model generation apparatus according to claim 1 or 2, further comprising: The interleaving unit is configured to, when the trained encoder converts each training sample contained in the plurality of first datasets and the one or more second datasets into feature values, determine whether the density of sample points corresponding to each training sample in the feature space exceeds a threshold, and if the density exceeds the threshold, select interleaving objects from the one or more second datasets. The second training processing unit is configured to perform machine learning on the classification model, except for the second dataset selected as the objects of the interpolation.

12. The model generation apparatus according to claim 1 or 2, wherein The classification model is configured to accept the input of the feature quantity and output the classification result of the category to which the feature belongs based on the input feature quantity.

13. The model generation apparatus according to claim 1 or 2, wherein The classification model is configured to accept the input of the sample and output the classification result of the category to which the feature belongs based on the input sample.

14. The model generation apparatus according to claim 1 or 2, configured to repeatedly perform: The first training processing unit uses the generated second dataset to perform machine learning on the encoder and decoder again, thereby updating the trained encoder and decoder. The conversion unit uses the updated, trained encoder to obtain two or more sample points corresponding to two or more training samples from the plurality of first datasets and the plurality of second datasets. The linear sum of the two or more sample points obtained is derived from the point designation. The sample generation unit uses the updated, trained decoder to generate decoded samples corresponding to the feature values ​​of the linear sum as new training samples. as well as The labeling unit assigns new positive solution labels to the newly generated training samples, thereby generating a new second dataset.

15. The model generation apparatus according to claim 1 or 2, wherein The specified indicators include any one of the following formulas 1 and 2. Additionally, X represents the derived linear sum, x i Each of the sample points represents a sample point.

16. A sorting device, comprising: The acquisition unit is configured to acquire object samples via a camera; A classification unit includes a trained classification model generated by the model generation apparatus as described in any one of claims 1 to 15, and the classification unit is configured to classify the categories to which features contained in the acquired object samples belong by using the trained classification model; and The output unit is configured to output information related to the results of classifying the categories.

17. A model generation method, wherein a computer performs the following steps: Multiple first datasets are obtained, each containing a combination of training samples and positive label values. The training samples are image samples that map objects. In this step... The training samples contain features. The category of the feature is related to the type of object reflected in the image sample. The correct answer label is configured to represent the category to which the feature contained in the training sample belongs; Machine learning is performed on the encoder and decoder using multiple training samples contained in the acquired multiple first datasets, and in this step, The encoder is configured to convert samples into feature quantities. The decoder is configured to decode the sample based on the feature quantity. The machine learning of the encoder and decoder includes the following process: training the encoder and decoder such that the encoder converts each training sample into a feature value, and the decoder decodes the sample according to the obtained feature values, which conforms to each training sample. Using the trained encoder, two or more training samples from the plurality of training samples are converted into feature values, thereby obtaining two or more sample points in the feature space that correspond to each training sample respectively. A linear sum is derived in the feature space according to a specified index. The linear sum is the linear sum of the two or more sample points obtained, and the sum of the distances calculated from each sample point becomes the maximum linear sum. By using the trained decoder, a decoded sample corresponding to the feature quantity of the linear sum is generated as a new training sample; The new training samples are assigned new positive solution labels, thereby generating a second dataset. The new positive solution labels are configured to represent the categories to which the features contained in the generated new training samples belong. as well as Machine learning for the classification model is performed using the multiple first datasets and one or more second datasets generated therefrom, and in this step, The machine learning of the classification model includes the following process: training the classification model on each dataset such that the classification model classifies the categories to which the features contained in the training samples belong, and the result is consistent with the positive label. The object is a product, and the category of the feature is related to the type of defect contained in the product.

18. A computer-readable storage medium storing a model generation program, the model generation program being configured to cause a computer to perform the following steps: Multiple first datasets are obtained, each containing a combination of training samples and positive label values. The training samples are image samples that map objects. In this step... The training samples contain features. The category of the feature is related to the type of object reflected in the image sample. The correct answer label is configured to represent the category to which the feature contained in the training sample belongs; Machine learning is performed on the encoder and decoder using multiple training samples contained in the acquired multiple first datasets, and in this step, The encoder is configured to convert samples into feature quantities. The decoder is configured to decode the sample based on the feature quantity. The machine learning of the encoder and decoder includes the following process: training the encoder and decoder such that the encoder converts each training sample into a feature value, and the decoder decodes the sample according to the obtained feature values, which conforms to each training sample. Using the trained encoder, two or more training samples from the plurality of training samples are converted into feature values, thereby obtaining two or more sample points in the feature space that correspond to each training sample respectively. A linear sum is derived in the feature space according to a specified index. The linear sum is the linear sum of the two or more sample points obtained, and the sum of the distances calculated from each sample point becomes the maximum linear sum. By using the trained decoder, a decoded sample corresponding to the feature quantity of the linear sum is generated as a new training sample; The new training samples are assigned new positive solution labels, thereby generating a second dataset. The new positive solution labels are configured to represent the categories to which the features contained in the generated new training samples belong. as well as Machine learning for the classification model is performed using the multiple first datasets and one or more second datasets generated therefrom, and in this step, The machine learning of the classification model includes the following process: training the classification model on each dataset such that the classification model classifies the categories to which the features contained in the training samples belong, and the result is consistent with the positive label. The object is a product, and the category of the feature is related to the type of defect contained in the product.

Images