Information processing apparatus
By using feature quantity extraction and similar data classification in the information processing device, combined with self-supervised learning, the problem of positive label error affecting learning results is solved, the inference accuracy is improved and the performance of the machine learning model is optimized.
Patent Information
- Application Number
- CN202280093855.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-25
- Publication Date
- 2024-11-01
AI Technical Summary
When existing information processing devices use positive label errors, they will affect the learning results and lead to a decrease in inference accuracy.
The feature quantity extraction unit is used to extract the feature quantity of the input data, and the similar data classification unit clusters multiple input data to generate a learning completion model for classification, and reduces the impact of positive label errors through self-supervised learning and improves inference accuracy.
Through clustering and self-supervised learning, the inference accuracy can be improved, the impact of positive label errors on learning results can be reduced, over-learning can be prevented, and the performance of machine learning models can be optimized.
Smart Images

Figure CN118891640A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information processing device. Background Art
[0002] In the past, neural networks used for image recognition, dynamic images, graphs, etc., have extracted feature quantities in the data by making the information processing device learn data in various fields. As a means of extracting feature quantities, a CNN (convolutional neural network) is known that uses convolution operations in deep learning to obtain high recognition performance. In addition, as another means of extracting feature quantities, in the case of images, a neural network called ViT (visual transformation network) using Transformer as an application of ATTENTION (selective attention) is known, and in the case of graphs, a neural network called a graph transformation network is known. At this time, regardless of which method is used, in the case of a task to classify data, the degree of certainty for each classification is output, and the data with the highest degree of certainty is output. In particular, there is a known method of not outputting when the degree of certainty is low (for example, Patent Document 1).
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-117861 Summary of the invention
[0006] Problem that the invention aims to solve
[0007] Generally, in an information processing device that performs learning using a data set in which a correct answer label is assigned to each input data, as in the above-mentioned information processing device, errors in the correct answer label may affect the learning result and reduce the accuracy of inference.
[0008] The present disclosure is used to solve the above-mentioned problems, and its object is to provide an information processing device and an information processing method that can improve the inference accuracy.
[0009] Means used to solve problems
[0010] The information processing device disclosed in the present invention is characterized in that it comprises: a feature extraction unit, which extracts feature values of input data; a similar data classification unit, which classifies multiple input data whose feature values are similar to each other in the multiple input data included in the first data set into a second data set based on a first data set including multiple input data and feature values respectively extracted by the feature extraction unit for the multiple input data included in the first data set; and a model generation unit, which uses the second data set to generate a learned model for classifying the input data, namely, a first learned model.
[0011] Effects of the Invention
[0012] According to the present disclosure, since it is configured as described above, it is possible to improve the inference accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a diagram showing an example of the hardware configuration of the information processing device according to the first embodiment.
[0014] Figure 2 This is a block diagram showing the structure of the information processing device according to the first embodiment.
[0015] Figure 3 This is a flowchart showing the clustering process performed by the information processing device according to the first embodiment.
[0016] Figure 4 This is a schematic diagram showing the clustering process performed by the information processing device according to the first embodiment.
[0017] Figure 5 This is a diagram showing an example of a data set of images input to the information processing device according to the first embodiment.
[0018] Figure 6 This is a diagram showing an example of a data set of a graph input to the information processing device according to the first embodiment.
[0019] Figure 7 This is a diagram showing an example of a natural language data set input to the information processing device according to Embodiment 1.
[0020] Figure 8 This is a diagram showing an example of a data set of a time waveform of a signal input to the information processing device according to Embodiment 1.
[0021] Fig. 9 The data are experimental data showing the inference accuracy of the information processing device according to the first embodiment with respect to the test data.
[0022] Fig.10 This is a flowchart showing the learning process performed by the information processing device according to the second embodiment.
[0023] Fig.11 This is a flowchart showing the learning process performed by the information processing device according to the second embodiment.
[0024] Fig.12 This is a flowchart showing the learning process performed by the information processing device according to the second embodiment.
[0025] Fig.13 This is a flowchart showing the learning process performed by the information processing device according to the second embodiment.
[0026] Fig.14The data are experimental data showing the inference accuracy of the information processing device according to the second embodiment with respect to test data.
[0027] Fig.15 This is a flowchart showing the learning process performed by the information processing device according to the third embodiment.
[0028] Fig.16 This is a flowchart showing the learning process performed by the information processing device according to the third embodiment.
[0029] Fig.17 The data are experimental data showing the inference accuracy of the information processing device according to the third embodiment with respect to test data.
[0030] Fig.18 This is a flowchart showing the learning process performed by the information processing device according to the third embodiment.
[0031] Fig.19 This is a block diagram showing the structure of an information processing device according to a fourth embodiment.
[0032] Fig. 20 This is a flowchart showing a learning process performed by the information processing device according to the fourth embodiment.
[0033] Fig.21 This is a flowchart showing a learning process performed by the information processing device according to the fourth embodiment.
[0034] Fig. 22 This is a flowchart showing the learning process performed by the information processing device according to the fifth embodiment. DETAILED DESCRIPTION
[0035] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.
[0036] Implementation method 1.
[0037] <Hardware Structure>
[0038] Figure 1 1 is a diagram showing an example of the hardware structure of an information processing device 100 as embodiment 1 of the present application. The hardware of the information processing device 100 may be an independent computer not connected to an information network, or a server or client of a server-client system connected to a cloud or the like via an information network. In addition, the hardware may also be a smart phone or a microcomputer. In addition, in the case of assuming a factory, etc., it may also be a computer environment in a closed network in the factory, which is called edge computing.
[0039] The information processing device 100 has a built-in CPU (Central Processing Unit) 1, and the input / output interface 4 is connected to the CPU 1 via bus wiring. When a command is input via the input / output interface 4 by a user using machine learning operating the input unit 6, the CPU 1 executes a program stored in the ROM (Read Only Memory) 2a according to the command. Alternatively, the CPU 1 loads the program stored in the hard disk (HDD) 2c or SSD (Solid State Drive, not shown) into the RAM (Random Access Memory) 2b, and reads and writes and executes it as needed. Thus, the CPU 1 performs various processes to enable the information processing device 100 to function as a device having specified functions.
[0040] The CPU 1 outputs the results of various processes from the output device as the output unit 5 or sends them from the communication device as the communication unit 7 via the input / output interface 4 as needed, and then records them in the hard disk 2c. In addition, the CPU 1 receives various information from the communication unit 7 via the input / output interface 4 as needed, and calls them out from the hard disk 2c for use.
[0041] The input unit 6 is composed of a keyboard, a mouse, a microphone or a camera, etc. In addition, the output unit 5 is composed of an LCD (Liquid Crystal Display) or a speaker, etc. In addition, the program executed by the CPU 1 can be pre-recorded in a hard disk 2c or ROM 2a as a recording medium built into the information processing device 100. Alternatively, the program or data set can be pre-stored (recorded) in a removable recording medium 9 connected via the drive 8.
[0042] Such a removable recording medium 9 can be provided as so-called packaged software. Examples of the removable recording medium 9 include a floppy disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.
[0043] In addition, programs or data sets can be sent and received through systems (com ports) such as the WWW (World Wide Web) that connect multiple hardware via either wired or wireless connections or both. In addition, learning described later can be performed, and only the weight function obtained by learning can be sent and received by the above method.
[0044] For example, CPU 1 enables information processing device 100 to function as a machine learning device that performs machine learning calculations. In addition, in addition to being composed of general hardware such as CPU or GPU (Graphics Processing Unit) that is good at parallel calculations, the machine learning device can also be composed of dedicated hardware such as FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).
[0045] In addition, the information processing device 100 may be composed of multiple devices via a communication port, and the learning and reasoning described below may be implemented by hardware of different structures. In addition, the information processing device 100 may receive sensor signals connected to different hardware via a communication port, or receive multiple sensor signals via a communication port. In addition, multiple virtual hardware environments may be prepared in one hardware, and each virtual hardware may be processed as a separate hardware.
[0046] <Definition of language>
[0047] The data used for input assumes image data, graph data, text data, and time waveform data. In addition, the output is a multi-value classification relative to the input data. Multi-value classification is, for example, a method of machine learning that is classified into 10 values from 0 to 9 and outputs an arbitrary value. The data is supervised learning or semi-supervised learning. That is, supervised learning means that for each input data, there must be more than one classification value. In addition, semi-supervised learning means that, although not all input data necessarily have classification values, there are at least one or more input data for each classification value. In this embodiment, the classification value of the input data for the above-mentioned supervised learning or semi-supervised learning is called a correct answer label, and the data that is not accurately assigned a correct answer label for the input data is defined as a label error. In addition, the group of the above-mentioned input data and output data is called a data set.
[0048] The data set can be separated into learning data and test data. The learning data is used for clustering or machine learning, while the test data is not used for learning and is used to verify the characteristics obtained through learning. In addition, in the case where sufficient data can be prepared, such as when there are more than 5,000 data for each correct answer label, verification data can be prepared separately from the learning data and test data. In this case, the verification data plays the same role as the test data described above, while the test data is used only once to confirm the accuracy during the inference of the information processing device after learning is completed, and is not used during learning.
[0049] By using verification data in this way, over-learning for test data can be avoided, and when there is a deviation between the inference accuracy (inference accuracy) of the verification data and the test data, it can be judged that over-learning has been performed. Therefore, when verification data is used, a higher inference accuracy can be achieved even in an environment close to the actual environment. However, when the amount of data is small, even if verification data is prepared, sometimes the inference accuracy fluctuates due to over-learning or the selection method of input data during learning. Therefore, in such a case, it is expected not to use verification data or discuss the addition of new data.
[0050] <Overview of Embodiment 1>
[0051] Then use Figure 2 The outline of this embodiment will be described. Figure 2 2 is a block diagram showing the configuration of the information processing device 100. The information processing device 100 is configured by the above-mentioned hardware configuration and includes a control unit 10, an input unit 6, an output unit 5, a communication unit 7, and a storage unit 20.
[0052] Input data from the input unit 6, the communication unit 7, and the storage unit 20 are input to the control unit 10. The storage unit 20 is composed of, for example, a ROM 2a, a RAM 2b, a hard disk 2c, a drive 8, etc., and stores various information used by the information processing device 100 and various data and information such as results calculated by the information processing device 100.
[0053] The control unit 10 includes a data conversion unit 11, a feature extraction unit 12, a similar data classification unit 13, a model generation unit 14, and an input data classification unit 15. Based on the data input from the input unit 6 and the communication unit 7 and the data and information obtained from the storage unit 20, the data conversion unit 11, the feature extraction unit 12, the similar data classification unit 13, the model generation unit 14, and the input data classification unit 15 perform various processes. For example, the control unit 10 outputs the results obtained by performing various processes to the outside via the output unit 5 and the communication unit 7. In addition, for example, the control unit 10 stores the results obtained by performing various processes in the storage unit 20. In addition, the input unit 6, the communication unit 7, and the storage unit 20 constitute the input unit in the first embodiment. In addition, the output unit 5, the communication unit 7, and the storage unit 20 constitute the output unit in the first embodiment.
[0054] The data conversion unit 11 converts (transforms) the input data input to the information processing device 100 by performing a predetermined process on the input data, thereby generating new input data. In addition, the data conversion unit 11 constitutes the data generation unit in the first embodiment. The feature extraction unit 12 extracts the feature of the input data from the input unit 6, the communication unit 7, and the storage unit 20 and classifies the input data. In other words, the feature extraction unit 12 digitizes the feature of the input data from the input unit 6, the communication unit 7, and the storage unit 20.
[0055] The similar data classification unit 13 performs clustering processing on the input data input to the information processing device 100. In addition, the similar data classification unit 13 extracts the feature quantity of the input data, judges whether each result is similar through self-supervised learning, and generates a learning model. The model generation unit 14 learns based on the input data from the input unit 6, the communication unit 7 and the storage unit 20, the data generated by the data conversion unit 11, and the data obtained by the clustering processing by the similar data classification unit 13, and generates a learning model. In addition, the model generation unit 14 performs supervised learning in the data set with correct answer labels in the data classified by self-supervised learning. In addition, the model generation unit 14 performs supervised learning on the data set without correct answer labels, using the data newly assigned by the classification result in self-supervised learning as the correct answer label. In addition, in the data set with correct answer labels in the data classified by self-supervised learning, the data with inconsistent correct answer labels in each classification is removed from each classification, and only the data with consistent correct answer labels is used for supervised learning. For example, it is also possible that, when the first data set and the second data set contain multiple correct answer labels corresponding to each input data, the similar data classification unit generates a seventh data set, and the input data classification unit uses the seventh data set for supervised learning to generate a learning model, wherein the seventh data set is obtained by excluding from the second data set the input data corresponding to the correct answer labels other than the correct answer labels corresponding to the largest number of input data among the multiple correct answer labels contained in the second data set.
[0056] The input data classification unit classifies the input data by reasoning based on the learned model generated by the model generation unit. For example, the input data classification unit 15 includes: a first learning device 15A that performs reasoning and classification on the input data based on the first learned model generated by the model generation unit; and a second learning device 15B that performs reasoning and classification on the input data based on the second learned model generated by the model generation unit. In addition, the input data classification unit may also include other learning devices that perform reasoning on the input data based on learned models other than those described above. The details of each structure of the control unit 10 are described below.
[0057] Figure 3: is a flowchart showing the clustering processing performed by the information processing device 100. When a data set that can be multi-valued classified using clustering and contains label errors is defined as a first data set, the information processing device 100 separates the first data set into a similar set and a non-similar set by clustering. For example, the first data set contains label errors of more than 5% and less than 10%. In addition, for example, the information processing device 100 first obtains a first data set (first data set) of input data that can be multi-valued classified and contains label errors (step ST1). After performing the processing of step ST, the information processing device 100 determines whether the first data set is classified into a similar set of input data whose feature quantities are similar to each other, that is, a second data set (second data set) through the clustering processing of the similar data classification unit 13 (step ST2).
[0058] The similarity set obtained by clustering the first data set is set as the second data set (step ST2, step ST3), and the second data set is used to generate a learned model for classifying the input data, that is, the first learned model, through the model generation unit 14 (step ST4). Through this process, the first learning device 15A can infer the input data based on the first learned model.
[0059] like Figure 4 As shown in the schematic diagram of , clustering is performed as follows: without using the correct answer label assigned to the input data, the distance between similar data in multiple data is made close, and the distance between dissimilar data in multiple data is made far away. In this embodiment, clustering is a process that requires learning based on machine learning.
[0060] Clustering is a method of generating and learning a combination of input data. Therefore, various methods are known regarding the method of selecting the combination of input data, the structure of machine learning used for learning, the definition of the distance between input data, and the definition of the loss function that minimizes the distance, but any method can be used. In this embodiment, a method of using a method called self-supervised learning, which is a method called contrastive learning, for clustering is described. In addition, self-supervised learning means that although it is named supervised, it minimizes the distance without using the correct label, that is, learning.
[0061] The learning data is separated into similar sets and dissimilar sets by clustering, the data separated into similar sets is set as the second data set, and the data separated into dissimilar sets is discarded. The second data set is prepared by this method, and the first learning device (first learning device) is prepared to classify the second data set into N categories, which is the same number of categories as the first data set, that is, to perform N-value classification. In addition, the value of N is a specific integer greater than 2, which constitutes the first number and the third number in the first embodiment.
[0062] The performance of the first learning device can be confirmed by using the test data described above. When the test data is input to the first learning device after learning, the inference value outputted is compared with the correct answer label assigned to the test data, and the consistent case is counted as a correct answer, and the inconsistent case is counted as an incorrect answer. For example, if there are 10,000 test data and 9,000 of them are consistent with the correct answer label, it can be calculated as 90.00% (= (9000 / 10000) × 100).
[0063] Verification can be performed by performing a comparison based on the test data, and the result shows that the learning device that has learned the second data set as an N-value classification problem can give more correct answers to the test data than the learning device that has learned the first data set as an N-value classification problem. In addition, the above-mentioned test data and verification data are data that are not used in the generation of the learned model, and can be prepared as data different from the first data set (specific input data), or a part of the first data set can be pre-set as test data and verification data before the generation of the learned model.
[0064] <1st Dataset>
[0065] ○Correct answer tag
[0066] In the case of 10-value classification, the correct answer label usually uses integers from 0 to 9, but it does not necessarily have to be continuous or start from 0. In addition, it is also possible to put 1 only in the position of the corresponding correct answer label in the manner of setting the 1 to (1, 0, 0), the 2 to (0, 1, 0), and the 3 to (0, 0, 1) as in One Hot Vector, and output a 10×10 matrix in the case of 10-value classification. In addition, for easy understanding, 10-value classification is used for explanation, but in this embodiment, as long as it is 2-value classification or more, for example, ImageNet, a famous data set in image recognition, has 14 million images and more than 20,000 correct answer labels reflected in each image, but it can also be used in such large-scale data sets. In addition, in the regression problem, although it is different from the classification problem, when the range of the correct answer and output of the input data is, for example, a real number from 0 to 100, it can be converted into 100 discrete values such as 0~1, 1~2,..., 99~100, so that it can also be converted into a classification problem with more than 2 values, which can be applied to this embodiment.
[0067] ○Tag error
[0068] There are several cases of label errors described in this embodiment. Take CIFAR-10, which uses a multi-value classification data set for image classification problems, as an example. CIFAR10 assigns a label of any of the 10 values of airplane, car, bird, cat, deer, dog, frog, horse, ship, and truck to each input data. In the case of supervised learning, the correct answer label is assigned to all input data. In the case of semi-supervised learning, only part of the input data is assigned the correct answer label. Failure to obtain a label that is consistent with the input data is considered a label error. For example, even if a photo of a dog is taken, the above example is also met when the label is a cat.
[0069] In addition, input data that contains labels that are outside the range of multi-value classification is also defined as a label error. For example, for image data labeled as airplanes in CIFAR-10, if an image of an apple that does not match any of the CIFAR-10 labels is captured, it fits the above example.
[0070] In addition, sometimes the input data contains multiple labels. In this case, depending on the purpose of use, there may be cases where it is determined that the label is wrong or not. For example, when cats and dogs are both included in the input data with the label of cat in CIAFAR-10, it meets the above example. The input data has labels for both cats and dogs. If processing is performed as long as one of them exists, it is not a label error. On the other hand, if processing is performed that it is determined to be wrong if the labels of both cats and dogs are not output, it is determined to be a label error.
[0071] In addition, it is also defined as a label error when a label other than a multi-valued classification is included. For example, in CIFAR-10, if a label of apple is assigned to an object that is included in the correct label, it is determined to be a label error. When apples are included in CIFAR-10, it becomes an 11-value classification, and the input information with the label of apple can be removed. Therefore, in this case, label errors can be removed by preprocessing before clustering.
[0072] ○ Input data
[0073] Next, the data input to the information processing device will be described in detail. Figure 5In the case of the images shown, there are static images and dynamic images, but the multi-value classification problem of dynamic images can be considered to be obtained by continuously combining static images. Therefore, only static images are described in this embodiment. Among static images, there are color images and monochrome images. In this embodiment, the input to the information processing device is a combination of more than two channels such as RGB for color images. In contrast, the monochrome image has no difference in input data except that it is formed by one channel. In addition, regarding the processing when there are multiple channels, although there are multiple channels depending on the different algorithms of the information processing device, they are usually summarized into one channel based on a fully connected weight matrix for connecting channels. However, the method in this embodiment can also be any method.
[0074] In addition, regarding the size of the image, if it is a small image of 32 pixels × 32 pixels like MNIST and CIFAR10, the operation time can be shorter, but if it is 96 pixels × 96 pixels like STL10, there is no limit to the size, and it does not have to be square as mentioned above. Regarding the image, it does not need to be a CCD or CMOS camera, and an infrared sensor or radar signal that converts physical data into numerical data, a wireless signal, a sensor signal that obtains heat or sound, vibration, electric field, magnetic field, etc., a diagram displayed or created on a computer, or CAD, etc. can be used.
[0075] for Figure 6 The classification problem in the graph shown considers multiple problem settings. The graph is composed of nodes as points and edges as lines connecting points, and arbitrary information can be embedded in the nodes or edges. As the main classification problems in such a graph, the first is the problem of classifying nodes based on edge or graph information, the second is the problem of classifying edges based on node or graph information, and the third is the problem of learning multiple graphs and classifying the graphs. In addition, there are classification problems that can predict the features of nodes to be selected from a limited number of options, classification problems that can predict the features of edges to be selected from a limited number of options, etc., and they can be used differently according to the purpose.
[0076] As an example, it is known that a circuit is a graph, so the circuit is used as the basis for the description. When the input is set as a circuit diagram and the output is set as the output voltage between any terminals of the circuit in the circuit, as a problem of classifying the nodes, the circuit components are selected in a way that the desired output voltage is obtained. There are only limited types of circuit components such as capacitors, coils, diodes, resistors, etc., so it becomes a classification problem. Next, regarding the problem of classifying the edges, all the required components are entered into the graph that becomes the circuit diagram, and the problem of predicting the wiring that connects the components becomes a classification problem. Strictly, more than two nodes are required, but if there are more than two components, it becomes a multi-value classification problem, and therefore, it becomes the object of this embodiment. Next, regarding the problem of classifying the graph, for example, when a graph that becomes a circuit diagram is provided, it can be used to classify which of the boost power supply, buck power supply, and buck-boost power supply the graph is, or the graph is a power supply circuit, a sensor circuit, a communication circuit, or a control circuit.
[0077] exist Figure 7 In the classification problem in natural language processing shown, the input signal is provided with data such as a sentence, a paragraph, a section, a full text, etc., which is a part of the article block cut out. For example, when data of a certain news report is provided, the question of inference as to which of the following is classified: economics, politics, sports, or science is a classification problem, and the method of this embodiment can be used for such a problem. This is a classification problem evaluated in a sentence or a paragraph, but for example, a question such as providing a novel and inferring whose work and the type of book are also classification problems, and therefore the method of this embodiment can be used. In addition, emotional analysis in which input data is classified into joy, anger, sorrow, and happiness in the input is also a classification problem, and the method of this embodiment can be used for such a problem.
[0078] about Figure 8 The classification problem in the time waveform shown is to classify the time waveform when the time waveform of arbitrary physical information such as time on the horizontal axis and voltage or wave height value on the vertical axis is set as input data. For example, as shown in the example of the above-mentioned circuit, the problem of taking the time waveform of the circuit diagram as input and classifying the power circuit, sensor circuit, communication circuit, and control circuit according to the time waveform can also use the method of this embodiment. In addition, the horizontal axis is described as time, but any feature quantity can be used as long as it is a feature quantity with a physical range such as frequency and coordinates. In addition, it does not have to be a time waveform. For example, the time waveform can also be Fourier transformed, with the horizontal axis set to frequency and the vertical axis set to amplitude.
[0079] The above describes the main data, but it can also be data that can be input into AI, such as the iris dataset (feature quantities of 4 types of numerical values are classified into 3 categories), which can be a numerical data set with multiple parameters and can be represented in a table format. As long as it can be converted into a form that can be output through classification, it can be any input data.
[0080] ○The amount of input data
[0081] The amount of data varies depending on the data set, but if it is supervised learning, it is desirable to prepare more than 1,000 images or graphs, time waveforms, character strings and other input data for each correct answer label. In addition, it is not desirable to have a small variance of similar data in one correct answer label, and it is desirable to have a learning data set with a variance that can contain the expected result during inference. As a means of confirming whether there are similar variances, if the inference accuracy is the same even if all or part of the learning data and the inference data are replaced, it can be considered that the variances are similar.
[0082] In addition, in order to increase the input data, a method called data augmentation can also be used. However, in the case of images, although data augmentation can be used to increase the learning data through affine transformation, etc., it is difficult to fill in a single time waveform, etc., and it is not possible to use augmentation for all data.
[0083] In the case of less data for learning, learning can be performed by obtaining a similar data set with a large amount of data (such as the above-mentioned ImageNet) or a large amount of data obtained by a similar sensor, or by transferring learning or fine-tuning a small amount of data obtained by using variables or weight matrices as initial values. In addition, transfer learning is a method of learning by slightly changing the elements of the variables or weight matrices that become initial values, and fine-tuning is a method of learning only full connections with fixed variables or weight matrices. However, most of them also use transfer learning and fine-tuning in combination. For example, after optimizing the full connection by using several fine-tunings at the beginning, the feature quantity contained in the weight matrix can be optimized by transfer learning, etc., and the two can be used in combination.
[0084] In the case of semi-supervised learning, similar to supervised learning, there is a small amount of labeled data, and accordingly, there is a disadvantage that bias is generated in learning and the reasoning accuracy is reduced. Therefore, it is also possible to learn by learning in unsupervised learning and assigning the correct answer after learning, as in self-supervised learning. In this case, it is also desirable that the number of learning data without correct answer labels is more than 1,000 for each correct answer label.
[0085] <Information processing device>
[0086] ○Clustering
[0087] Clustering refers to a method of grouping data based on the similarity of input data. In many clusters, the number of groups is a hyperparameter determined by the designer or user of machine learning. In this embodiment, the number of correct labels is determined, so it is expected to be classified into the same number as the correct labels through clustering, for example, 10 in the case of CIFAR-10. Among the classical clustering algorithms, K-means is the most mainstream, but after the emergence of deep learning, clustering based on deep learning, clustering based on decision trees such as gradient boosting method, etc. are also known. Any method can be used in this embodiment. In this embodiment, clustering based on deep learning that is easy to obtain reasoning accuracy for a large amount of data is described.
[0088] As evaluation indicators of clustering, there are several methods such as the Adjusted Rand Index (ARI) and the Normalized Mutual Information (NMI), and these methods can also be used to evaluate the learned clustering. However, in the present embodiment, although the label error is included, the correct label is provided. Therefore, it can also be evaluated by the following methods, etc.: the similarity of the same correct label is used as an indicator, for example, by using the majority vote of the correct label assigned to each label to determine the label of the similar set obtained by clustering. For example, it can also be set to put 1000 elements in a certain similar set. In the case where 900 labels of the 1000 elements are composed of 1, 70 labels are composed of 7, and 30 labels are composed of 9, 900 labels of 1 are assigned to the 1000 elements by majority vote. However, it is necessary to process in a way to avoid the same label between different similar sets.
[0089] ○ Unsupervised learning
[0090] The above clustering is equivalent to unsupervised learning. In machine learning, it is usually classified into supervised learning that provides correct answer labels, unsupervised learning that does not provide correct answer labels at all, and reinforcement learning that maximizes the reward set as the goal even though there is no correct answer. Semi-teacher learning is equivalent to the middle of supervised learning and unsupervised learning, but it also uses correct answer labels in part, so it is sometimes defined as a method of supervised learning.
[0091] In this embodiment, the first data set is provided with correct labels, so supervised learning and semi-supervised learning can be performed. However, the characteristic is that instead of using these supervised learning, learning is performed by clustering as unsupervised learning, and the learning data is distilled (unnecessary data is removed) to create the second data set. Therefore, even if the data set contains a large number of label errors, as in this embodiment, the second data set can be created without affecting the error rate of the label or the quality of the data.
[0092] ○ Self-supervised learning
[0093] In this embodiment, a method called self-supervised learning is used in unsupervised learning based on deep learning. Self-supervised learning is studied as one of the Siamese Network methods, which is a basic method in Meta-Learning.
[0094] Meta-learning is a method of learning learning methods, which is mainly divided into metric-based learning, model-based learning, and optimization-based learning. As one of the distance learning methods, twin networks are studied. Distance learning is a method of making the distance of closer data close and the distance of farther data far away when considering the combination of more than two data. There are various known methods for defining distance, but there are known methods that use statistical distances such as Mahalanobis distance as the basis or mutual entropy, mutual information, cross entropy, Kulbak-Leibler information, or mutual correlation matrix as the basis to define distance, etc., and combine these statistics or information quantities to measure the similarity between feature quantities by one or more. In addition, it is also possible to simply find a vector as the result of feature extraction, and measure the similarity of two input data by cosine similarity based on two vectors for two input data. In addition, it is also a desired method of use to express the result of calculating the similarity between each input data as a distance matrix summarized in a matrix form.
[0095] In addition, in self-supervised learning, similar input data is produced by operations such as extracting a portion of the original input data or removing a portion, and the distance between data produced based on one input data is close. On the other hand, data produced based on data of other labels are processed in the same way, and the distance between data is made close for the distance of closer data and is made farther for the distance of data that can be judged as farther. In addition, regarding the features of the input data, feature quantities can be extracted by methods such as full connection, convolution, and Attention as deep learning processing. Therefore, the similarity between data is calculated by measuring the distance between the feature quantities.
[0096] The clustering in this embodiment needs to use clustering with high accuracy of classification as similar and dissimilar. This is because, when using clustering with low classification accuracy, a large amount of data is classified as dissimilar, and the variance of the learning data or learning data is reduced. As a result, the inference accuracy for the test data sometimes decreases. For clustering performance, it is possible to confirm the input data classified as non-similar, and determine whether it contains a large amount of data other than outliers. In the case of containing a large amount of such data, it is expected to use different clustering methods. In particular, for clustering based on deep learning, the accuracy of classification is mostly high, and for data that mostly contains label errors, a higher classification accuracy can be obtained.
[0097] ○Information processing device that processes input data
[0098] An information processing device for extracting feature quantities required for self-supervised learning is described. The first learning device is the same as the usual supervised learning for solving usual classification problems, and therefore, is not described. In addition, the difference between the first learning device and self-supervision is that the evaluation function defining the evaluation index is different, and the softmax function (soft maximum function) required for class classification is not used. The full connection immediately before the output layer of the first learning device is not necessarily required, and the operation of feature quantity extraction before inputting the full connection can also be aggregated into the desired number of categories. However, in most cases, the method of implementing the softmax function is in a trend of improving inference accuracy. In addition to the softmax function, it can also be a nonlinear function that deforms the softmax function, such as the log-softmax function.
[0099] Next, an example of a method for extracting feature quantities for various input data is shown. In the case of images, as mentioned above, Transformers based on CNN (convolutional neural network), MLP (Multi-Layer Perceptron), and Attension (selective attention) are mostly used. However, the image can also be processed by GNN (Graph Neural Network) used in the graph theory shown below or RNN (Relational Neural Network) used in time series processing, or by applying their techniques. In addition, although the above is deep learning, logistic regression, support vector machine, gradient boosting method, etc. can also be used. In this embodiment, these algorithms can also be any algorithms.
[0100] In particular, various algorithms are known in deep learning. In CNN, there are many algorithms such as VGG, ResNet, AlexNet, MobileNet, and EfficientNet, just because they have the common point of performing convolution. In addition, in MLP, methods such as MLP-Mixer are known, which can obtain high reasoning accuracy by processing images only through simple full connection, and these methods can also be used. In addition, there are known methods such as Vision Transformer that processes images through Transformer or Transformer combined with CNN feature extraction, which can be processed by these single methods or combinations.
[0101] For graphs, we use GNN (Graph Neural Network) or GCN (Graph Convolutional Network) which performs convolution on close nodes. Graphs are not arranged at equal intervals like images, so graphs cannot be directly input into deep learning. Therefore, we convert them into Figure 1 The adjacency matrix or degree matrix corresponding to each place is input. Here, the adjacency matrix is a method of expressing the presence or absence of connection between nodes in the form of a matrix, and becomes an N×N matrix when there are N nodes. In addition, in the case of an undirected graph where the edges do not have a direction, it becomes a symmetric matrix. In addition, the degree matrix is a method of expressing the number of edges contained in each node in the form of a matrix, and becomes an N×N matrix when there are N nodes, and becomes a diagonal matrix. The graph converted into such a matrix is input to GNN or GCN, and by passing through the hidden layer of GNN and the like to the full connection or softmax function just before the output layer multiple times, the graph can be treated as a classification problem.
[0102] In the case of time waveforms, RNN is mostly used, and GRU (Gated Recurrent Unit) and LSTM (Long Short-Term Memory) which are extensions of RNN have become the main technologies. In addition, there are also known technologies that combine the technologies of Transformer or Attention mechanism which is the basis of Transformer, and TCN (Temporal Convolutional Network) which uses discrete convolution. By using these technologies on input data, it is possible to input data to deep learning.
[0103] In the case of natural language processing, LSTM is known to process the above-mentioned time waveform, and its development is called Seq2Seq (sequence to sequence). In addition, the Attention mechanism is known as a development of Seq2Seq (sequence to sequence), and the Transformer technology is known as a development of the Attention mechanism. By using these technologies, natural language processing can be performed. In addition, LSTM can predict language based on the context of an article, but the problem is that it can only process a fixed length of the signal, so the accuracy varies depending on the length of the article. Seq2Seq introduces the concept of Encoder-Decoder to solve this problem.
[0104] However, Attention is a method that introduces correlation between words that make up an article and improves accuracy due to insufficient accuracy, but it cannot be parallelized and cannot process large-scale data sets. Therefore, since Transformer is a method that can parallelize Attention using dedicated hardware such as GPU, although there are differences in inference accuracy and calculation time, the underlying technology is common, so any method can be used in this embodiment.
[0105] In self-supervised learning, feature quantities are extracted by the above method. At this time, comparison data needs to be produced. When the input data is an image, the information processing device 100 extracts or removes a portion of the input data through the data conversion unit 11, performs affine transformation such as rotation and stretching, or superposition of white noise, etc. In the case of color images such as RGB, multiple images can be produced from one input image by changing the color balance or changing the clarity. In particular, it is known that the distance when extracting the feature quantity of images produced from the same input image is closer, so learning can be performed by performing processing to make the distance closer.
[0106] On the other hand, in the case of graphs other than images, natural languages, and time series data, most of them have physical constraints. For example, in the case of a graph network that processes a circuit diagram, it is possible to extract or remove a portion of an edge or node in the same way as an image, but when extracting or removing, it can only be transformed into data that follows physical laws such as Kirchhoff's law. As an example, when processing a circuit, the path of current flow needs to be a closed loop. Therefore, arbitrarily extracting edges and making a closed loop open in order to create a new graph network does not meet the physical constraints. Therefore, it is necessary to consider the physical constraints to create data.
[0107] In natural language processing, it is also possible to extract or remove a portion of an article, but the replacement of words using synonyms requires understanding of the context of the article, so it is difficult, and it is also difficult to randomly change the order of the article. However, in the case of articles, data is easier to obtain than other data, so most methods that can be used are to find similar articles from a large amount of data. In time series processing, it is also possible to extract or remove a portion of a waveform, but it should be noted that data that is subject to physical laws such as the continuity of the waveform cannot be processed by methods that do not follow physical laws when extracting or removing it. In addition, in the case of waveform deformation, random deformation is not expected, and deformation is expected to be performed under the conditions of specific theoretical formulas such as Fourier series expansion.
[0108] <Creation of the second data set>
[0109] The feature quantity of the data with label errors is extracted by self-supervised learning, and the result is used as the second data set consisting of similar sets of cluster numbers, and as Figure 3 As shown, data without similarity is removed so that it is not included in the second data set. In the second data set, the following results in the <Experimental Results> are obtained under the condition of using the same labels as the labels assigned to the first data set, but the labels of the data determined as similar sets in the results of self-supervised learning can also be changed to set as the second data set.
[0110] <Learning and inference by the first learning device>
[0111] The learning and reasoning of the first learning device are the same as those of ordinary deep learning. Specifically, a weight matrix such as convolution or attention is calculated for the input data, and the number of classifications is set to the same number as the correct answer label by a method such as full connection as a class classifier for collecting feature quantities, and the difference between the result obtained by implementing the softmax function during learning and the correct answer label is calculated. This difference is propagated from the output side to the input side through the error back propagation method to update the weight matrix.
[0112] On the other hand, in inference, the weight matrix and the weight of the full connection obtained through learning are calculated on the test data, and the output is output as the inference value. Nonlinear functions such as the softmax function used just before the output layer during learning are used to convert small differences in feature quantities into large ones, and are implemented in a way that clearly shows the difference between the correct label and the output of machine learning, and it is easy to update the weight matrix through error back propagation, so it is not necessarily necessary to use it during inference.
[0113] The expected method of use is to perform inference on the test data using the first learned model. In addition, after the feature extraction unit has classified the test data according to the similarity, performing inference using the first learned model is also an expected method of use. In this case, the feature extraction unit calculates the similarity between the multiple data in the second data set used by the first learned model, and only extracts the input data that is determined to be similar. In addition, when there are multiple test data, the feature of each test data can be calculated in the same way as when the first learned model is produced, and the similarity is calculated using the result, and only the data determined to be similar is inferred by the first learned model.
[0114] <Experimental Results>
[0115] use Fig. 9 The effect of this embodiment is illustrated by the experimental results. Fig. 9 This is an experimental result obtained by using CIFAR-10 data for the dataset. For CIFAR-10 data, 5% and 10% label errors were randomly assigned to the data through preprocessing. In addition, the learning data of CIFAR-10 is 5000 data with 5000 labels each, totaling 50000 data. Therefore, in the case of 5% label errors, 250 labels are assigned for each label, and a total of 2500 labels are assigned errors.
[0116] In addition, in the self-supervised learning model, a method called SwAV (Swapping Assignments between Views, paper name: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments) was used, which applied a method called SimCLR (paper name: A simple framework for contrastive learning of visual representations). In addition, the number of clustering categories as a hyperparameter is the same as the number of correct answer labels of CIFAR-10, which is 10.
[0117] When clustering was performed using this method, 1336 items in the first data set became non-similar data. Therefore, the remaining 48664 (=50000-1336) items were set as the second data set. Furthermore, when learning was performed using VGG13 (Visual Geometry Group 13), a type of CNN, the second data set was obtained. Fig. 9 The results of “Clustering + CNN” are shown in Figure 2.
[0118] According to the Fig. 9 From the results, it can be seen that the inference accuracy is 90.00% after 20 rounds (epochs; the number of times the weight matrix is updated). On the other hand, in the data with the above-mentioned 5% label error, when clustering is not performed and learning is performed through the same VGG13 as above, at 20 rounds, the accuracy is 89.03%, which can be confirmed to be a decrease of about 1% compared to clustering + CNN. In addition, when the same learning is performed on data with 10% label errors, the accuracy is 87.30%, which can be confirmed to be a decrease of 2.7% in inference accuracy compared to "clustering + CNN". This time it is shown up to 20 rounds, but even if the number of rounds is increased to about 200, the above relationship will not change, and it is the same as Fig. 9 Similarly, the inference result of “clustering + CNN” is the best.
[0119] According to this result, it is generally believed that it is best to increase the amount of data for machine learning, but when there are some errors in the labels, it is best to learn after removing the data with errors through clustering. Especially in a practical environment, for example, if it is an image composed of sensor data, it is easy to become a label error when data outside the object is written when the data is acquired. In addition, in the case of artificially assigning correct labels in the classification of waveforms or circuits, label errors are easy to occur depending on human skills, and it is difficult to manually calculate these label error rates. In particular, this embodiment is based on the following discovery: by removing data containing label errors, although the amount of data is reduced, the inference accuracy can be improved.
[0120] In addition, the effect obtained by removing data (distillation) by clustering is shown. The first is to prevent overlearning. Usually, by using large-scale machine learning with a large number of learnable parameters, it is possible to include label errors in learning. However, this is the result of overfitting the learning data and test data. Therefore, although a higher inference accuracy can be obtained in a closed data set such as a usual data set used to study machine learning, when the data is obtained in an actual environment such as a factory, the inference accuracy decreases. In this regard, when the method of this embodiment is used, the overlearning can be reduced.
[0121] The second is that humans can confirm the data that has been removed. Usually, the processing of machine learning is called a black box, and there is no way to clearly indicate to humans the processing of machine learning itself and the basis of the output obtained. On the other hand, humans confirm the classification results of the input data as intermediate processing, infer the trend of errors, and thus easily infer the reasons for the judgment of machine learning. For example, by understanding the trend that the subject is easily classified as a similar set when it is reflected in the center of the image, and is easily classified as a non-similar set when it is reflected in the corner of the image, it can be useful for optimizing the machine learning model.
[0122] The third is that if the second data set is produced once by clustering, there is no need to perform calculations multiple times. Clustering using self-supervised learning tends to consume more computing time and amount of calculations than conventional supervised learning. However, since the calculation obtains the second data set, there is no need for recalculation during learning and reasoning of the second learning device (second learning device). In particular, in the design of machine learning, the most time is required in the selection of supervised learning models and the production of learning models for reducing the impact of label errors on reasoning results. Therefore, the time spent on the above-mentioned clustering is relatively small and does not require manual work, so it can be expected to shorten the development period of machine learning.
[0123] The fourth is that it can also be used for smaller data sets. As mentioned above, clustering based on self-supervised learning can create learning data based on its own data and learn, so clustering can be performed even if the learning data is as little as 1,000 or less. However, the amount of data in the second data set also decreases, so it is expected to use a learned model that has been previously learned with similar data for fine-tuning. However, in the case of the amount of data or the time or computing resources consumed by calculation, it is also a better method to use transfer learning and fine-tuning in combination.
[0124] Implementation method 2.
[0125] <Outline>
[0126] In the first embodiment, data determined to be dissimilar are discarded during clustering. In contrast, in the information processing device 200 of the present embodiment, the following method is described: data determined to be dissimilar, i.e., deviation values, are aggregated and defined as a third data set, and learning is performed using the third data set.
[0127] <Deviation value>
[0128] The deviation value in this embodiment refers to, Figure 3 As shown, clustering is performed on the first data set, N similar sets are extracted, and the remaining non-similar data are aggregated into 1 and defined as a deviation value. For example, the 1336 input data discarded when SwAV is applied to CIFAR-10 shown in the experiment of Implementation 1 are consistent with the deviation value in this implementation. The data that aggregates the deviation value is set as the third data set.
[0129] <New Tags>
[0130] like Fig.10 As shown, for example, the information processing device 200 sets the non-similar data that does not classify the first data set into a similar set by clustering as a third data set (third data set) as a deviation set (No in step ST2, step ST14), assigns a deviation value label (first label) to the third data set (step ST15), and combines it with the second data set to create a fourth data set (fourth data set) (step ST16). In the case where the second data set is an N-value classification, by setting the deviation value label to N+1, the fourth data set can be a data set with a label of N+1 value.
[0131] <Learning by the second learning device using the fourth data set>
[0132] The second learning device is trained using the fourth data set (step ST17). A portion of the data of the deviating value label in the fourth data set is defined as test data. When the number of deviating value label data is greater than the number of data of each label in the second data set, the number is selected to be the same as the test data of the second data set. When the number of deviating value label data is less than the number of data of each label in the second data set, it is expected to be output at the same ratio as the second data set. For example, in the case of CIFAR-10, there are 5000 learning data for each label and 1000 test data for each label, so 20% is set as test data.
[0133] In contrast, the clustering-based deviation values shown in the experiment of the first embodiment are 1336, so 270 of the 1336, which is equivalent to 20%, are defined as test data, and the remaining 1066 are defined as learning data. However, when the learning data of the third data set is roughly less than 1000, it is easy to cause over-learning, so it is desirable to process it by the first learning device shown in the first embodiment. Although the above-mentioned 1066 learning data are not enough, in order to show the effect, the experimental results are shown at the end of this embodiment.
[0134] The third data set produced as described above is combined with the second data set to produce a fourth data set, and the second learning device is used to learn using a machine learning algorithm in the same manner as the first learning device to generate a second learning completed model. The difference from the first learning device is that learning is performed by N+1 value classification, and the inference accuracy is confirmed using test data. Since this is the same process as in the first embodiment, it is not repeated in this embodiment.
[0135] <Third learning device using third data set>
[0136] When it is determined that there are enough data of outliers, a third learning device (not shown) can be created. Fig.11 : This is a flowchart showing the processing of the third learning device of the information processing device 200. The third learning device, like the first learning device, uses the third data set as the N-value classification for learning (step ST21, step ST22). Thus, the third learned model is generated by the model generation unit 14. The information processing device 200 obtains a test data set that can perform multi-value classification and contains label errors (step ST23), and uses the second learning device to infer the test data set to determine whether it is classified as an outlier (step ST24). Based on the results of steps ST22 and ST24, the information processing device 200 outputs the inference result for the third data set (step ST25). The method of preparing learning data and test data is the same as the above-mentioned <Learning of the second learning device using the fourth data set>, so it will not be repeated. However, in order to prepare the third learning device, in the case of 10-value classification, it is desirable that the number of data per label is more than 1000. In addition, the third data set is data containing many abnormal values that are difficult to obtain. Therefore, although it also depends on the type of data, it is desirable to use it when data augmentation such as images can be used.
[0137] <Handling when the inference is an outlier value>
[0138] Four processing methods are described in the case where the result of inference performed on input data not included in the first data set (input data not used in the generation of the learning model) by the second learning device is determined to be an outlier label. The first method is a method of using a learning device that has learned from the first data set to infer data (first input data) determined to be an outlier label, the second method is a method of using a learning device that has learned from the second data set to infer data determined to be an outlier label, the third method is a method of using a learning device that has learned from the third data set to infer data determined to be an outlier label, and the fourth method is to output that it cannot be determined when an outlier label is assigned.
[0139] The first method is easy to obtain high inference accuracy when there is enough data and few label errors. In this case, the third data becomes a sufficient amount of data and has few label errors, so the inference accuracy of the learning device itself that has learned using the first data set is also easy to become high.
[0140] The second method can achieve higher inference accuracy in an actual environment with many label errors. However, in this case, learning is performed using data from which the second label error has been removed. Therefore, if the input data that is determined to be an outlier label is itself an outlier, it is easy to be determined as an incorrect answer.
[0141] The third method is effective in an actual environment where there is a sufficient amount of data and there are many data classified as non-similar by clustering. In particular, it is effective in a scene where the determination of the deviation value is important because it becomes a machine learning device that is good at deviation values. However, in the case of data produced for machine learning such as CIFAR-10, the amount of data is small or the data classified as deviation values is small, so the inference accuracy is easy to decrease.
[0142] Regarding the fourth method, in the past information processing, even if abnormal data is entered, it is possible to avoid forced data output by setting it to N+1 value classification. In the case of actual environments such as medical data (X-ray or MRI image diagnosis) where the final judgment is made by humans, it is also possible to make a non-compulsory judgment. By making a non-compulsory judgment, the error rate can be greatly reduced.
[0143] However, the above is a guideline, and any method may be used, or a plurality of methods may be used in combination, depending on the label error rate in the first data set, the type of data, the performance to be obtained, and the like.
[0144] <Transforming the input data that is a deviation>
[0145] The following method is described: the second learning device infers the input data, and the inference is performed after the input data determined as outlier labels are deformed (converted) by the data conversion unit 11. As described in the first embodiment, for example, affine transformation or noise superposition can be performed on the image.
[0146] Specifically, in an image, for input data judged as a deviation value label, in addition to cutting out and extracting a portion of the input data and removing a portion of the image, a combination of affine transformations such as enlargement and reduction, rotation, and the addition of noise, and in the case of color images such as RGB, changing the color balance or clarity, etc. are performed. As a result, for example, more than 1,000 images can be generated based on one image.
[0147] The first method is a method using a learning device that has learned using the first data set, the second method is a method using a learning device that has learned using the second data set, and the third method is a method using a learning device that has learned using the third data set. The fourth method is a method using a learning device that has learned using the fourth data set. Although four processing methods are described, the features of each case are the same as the method described in <Processing when inference is a deviation value>, so the same description will not be repeated, and only the differences will be described.
[0148] In the first method, a learning device that has learned using the first data set makes an inference on P input data (second input data) newly generated by the data conversion unit 11 by superimposing and deforming the test data 1 to P (≥3) times, counts the number of times each label of each inference result, and sets the label assigned the most times (for example, in the case where the test data is image data representing integers from 0 to 9, when P=4000 and 0 is 100 times, 1 is 100 times, 2 is 200 times, 3 is 300 times, 4 is 400 times, 5 is 500 times, 6 is 600 times, 7 is 700 times, 8 is 700 times, and 9 is 900 times, 9, which has the largest number of inference times of 900, as a label, is output) as an inference value. In this case, when the label error exceeds 5%, the result is likely to be biased, but when the label error is less than 5% and learning is performed using a sufficient amount of data, a stable result can be obtained. In addition, the value of P constitutes the second number in the second embodiment.
[0149] In the second method, if Fig.12As shown, the first learning device performs inference (step ST34) on P input data (second input data) newly generated (step ST33) by the data conversion unit 11 by superimposing and deforming the test data 1 to P (≥3) times, and counts the number of times each label of each inference result, and sets the label assigned the largest number of times as the inference value (step ST35) in the same way as the first method. In the second method, in the case of an abnormal value not included in the learning data, it is difficult to correctly determine even if the generated data is increased, but in most cases, it has the effect of improving the inference accuracy.
[0150] In the third method, the newly generated P input data are inferred by the third learning device in the same manner as the above method. The third learning device of the third method is an information processing device that is good at outliers, so if input data that can be learned by the third learning device can be prepared, the inference accuracy can be improved.
[0151] In the fourth method, if Fig.13 As shown, the second learning device infers multiple input data (second input data) obtained by transforming the test data more than 3 times P (P>2), discards the data classified as deviation values, and uses the inference values classified as other than deviation values (step ST44). The number of inferences is counted in the same way as the first method, and the inference result with the largest number of inferences within the correct answer label other than the first label is output.
[0152] The above description counts the number of inferences and calculates by majority decision, but the following method can also be used as shown in Implementation 4: the information entropy is calculated based on the average value of the output results of the information processing device, and the output information entropy becomes the label with the smallest. In addition, no matter which of the above methods is used, P can be greater than 2. In addition, it is also possible that the similar data classification unit performs a predetermined processing (for example, 1 to P times of superposition deformation) on the first input data that is determined to be non-similar or classified as the first label through reasoning based on the feature extraction unit or the second learned model, thereby generating the above-mentioned second input data based on one input data.
[0153] <Experimental Results>
[0154] Fig.14 The result obtained by performing inference on the test data by the second learning device, removing the input data classified as outliers, and calculating the inference accuracy of the results classified as other than outliers is shown. Fig.14As shown, when the first data set, which directly uses the learning data of CIFAR-10, is classified into 10 values, the CNN-based VGG13 is 83.78% in 20 rounds. In contrast, in the second learning device that classifies the fourth data set into 11 values, the CNN-based VGG13 is the same as above, and the 20 rounds are 84.20%, which shows an improvement of about 0.5% in accuracy. In addition, the data classified as outliers in the second learning device are 521 out of 10,000 test data. These data are discarded and not compared with the correct answer labels, so they do not affect the inference accuracy.
[0155] In the process of "Transforming the Input Data that Becomes a Deviation Value", the following processing was performed: the input data that becomes a deviation value was transformed by inference through the fourth second learning device, about 1000 combinations were made, and inference was performed again through the second learning device to count the number of occurrences of data other than the deviation value. When such processing was performed, the number of occurrences of data other than the deviation value was 84.49%, which was the same as Fig.14 Similarly, we can see that the inference accuracy has increased by about 0.7%. Fig.14 The difference is that all test data are compared with the correct answer labels, so the reasoning accuracy is improved overall.
[0156] However, the deformation method becomes a hyperparameter. For example, in the case of CIFAR-10, the following characteristics are observed: although data is produced by rotation or stretching through affine transformation, the inference accuracy is easily improved by including data with a rotation angle of more than ±15 degrees and less than ±45 degrees, or although there is no meaning of stretching when the vertical and horizontal stretching is less than ±10%, the inference accuracy deteriorates when it exceeds ±30%. Therefore, the optimal deformation conditions must be searched manually or mechanically through large-scale computers. However, if the deformation conditions can be optimized in the case of a computer environment or the range of deviation is roughly known, the inference accuracy can be improved by a simple method.
[0157] Implementation method 3.
[0158] <Outline>
[0159] As shown in Embodiment 1, the number of classifications by clustering becomes a hyperparameter that must be determined by the designer of machine learning. For data that is assigned a correct answer label and has a certain number of classifications as in the first data set, it is sufficient to divide it into that number, but the number of divisions of data in an actual environment cannot be clearly determined in most cases. The method of the information processing device 300 based on this embodiment can be used in such a situation.
[0160] <Clustering>
[0161] As described in Embodiment 1, the machine learning for clustering assumes the use of algorithms such as k-means and self-supervised learning. Each algorithm needs to define the number of clusters as a hyperparameter. The learning device of the information processing device 300 in Embodiment 3 performs learning by classifying the input data into the defined number of clusters, and generates a fourth learned model through the model generation unit 14.
[0162] <Optimization of the number of categories>
[0163] As in Embodiments 1 and 2, the data of CIFAR-10, which is data with a known number of classifications, is used for explanation. However, the actual target data set is data with an unknown number of classifications. This can be used in many scenarios in actual environments, such as when it is unknown how many classifications two or more measurement results obtained through physical experiments can be classified into, or when it is unknown how many categories the customer classes that purchased the product can be classified into.
[0164] The information processing device 300 calculates the number of clusters in sequence from M (the fourth number, M is greater than or equal to 2) = 2, which is a specific integer, and as shown in Implementation 2, assigns M different labels to each similar set, and classifies the similar sets into learning data and test data as an M-value classification problem. However, it is mostly possible to assume the number of classifications N based on empirical rules, etc. In this case, clustering can also be started from a positive integer M greater than the number of classifications N. This is to reduce the amount of calculation, and it can be expected that the more clusters there are, the higher the inference accuracy. Assuming that the same number of clusters as the learning data is defined, it is sufficient to set each learning data as a similar set one by one, so the inference accuracy can be set to 100% regardless of the accuracy conditions. However, if the number of clusters is too large, the purpose of clustering is lost.
[0165] Therefore, if Fig.15 As shown in FIG. 1 , when the number of clusters is unknown (step ST51), it is calculated from integers greater than 2. In addition, when the number of classifications N can be estimated based on empirical rules or the like (step ST61), as shown in FIG. Fig.16 As shown, when M is less than N, M=N is set (step ST67). However, in order to shorten the calculation time for searching for the optimal number of clusters, clustering is performed in the direction in which M increases from M=N (step ST52), and the value of M is gradually increased such as M+1, M+2, etc., and the inference value in the target index such as clustering accuracy is verified. Then, M is output when the inference value in the target index becomes maximum (step ST55, step ST66). However, in order to prove that M is maximum, the case of M+1 must be calculated, so at least the case of M+1 needs to be calculated.
[0166] <Experimental Results>
[0167] exist Fig.17 The following results are shown in FIG. 1 : The horizontal axis takes the number of clusters, performs M-value classification as the target indicator in this embodiment, allocates learning data and test data in a ratio of 80 to 20 from the data in which the labels are assigned to the M-values, and the vertical axis takes the inference results when learning is performed using the learning data and inference is performed using the test data. Fig.17 As shown in the figure, it can be seen that the inference accuracy for the test data produced by clustering increases monotonically until the number of clusters reaches 10. When the number of clusters reaches 11, the accuracy decreases by about 1%, so it can be seen that the inference accuracy for the test data reaches a maximum when the number of clusters reaches 10. It can be seen that when the number of clusters is further increased, the inference accuracy reaches the same level as when the number of clusters is 10 when the number of clusters reaches 13 or 18. Although multiple maximum values appear, the minimum number of clusters, 10, is selected.
[0168] For problems with an unknown number of categories, the same method can be used to calculate the optimal number of clusters. Therefore, even if the number of categories is unknown and the data cannot become a classification problem, it can be assigned new labels through clustering and converted into labeled test data.
[0169] <Use of non-similar data>
[0170] Fig.18 A flowchart is shown for the case where a new label is assigned to data determined to be non-similar by clustering in the same manner as in Implementation Example 2 and inference is performed. As described above, when M>N, the first data set is classified into M values by clustering (step ST72), and each similar data is assigned a new label and set as a data set (step ST73). In addition, the non-similar data is set as a data set (non-classified data set), and a new label (second label) is assigned (step ST74, step ST75), and combined with the label of the M value classification to form a fifth data set (fifth data set) (step ST76). Then, the fifth data set can also be set as an M+1 value classification to form a fourth learning device for learning (step ST77). In addition, the information processing device 300 can also use a sixth data set (sixth data set) that does not have a correct answer label, which is different from the first data set, as input data in the similar data classification unit.
[0171] Thus, even data that cannot become a classification problem because the number of classifications is unknown can be given new labels for learning. In addition, by adding non-similar data to new labels, a learning device that can determine abnormal values as deviation values by the method shown in Implementation 2 can be constructed.
[0172] Implementation method 4.
[0173] <Outline>
[0174] With respect to the outputs in the learning apparatus described in Embodiments 1 to 3, the degree of certainty of the inference result can be calculated by a method that considers information entropy.
[0175] <Information Entropy>
[0176] The information processing device 400 of this embodiment is relative to the information processing device 100 of embodiment 1, and the control unit 10 further includes an information entropy calculation unit 16 and a threshold setting unit 17. The information processing device 400 of this embodiment is based on the discovery of the following effect: the higher the inference accuracy, the smaller the information entropy. For example, in VGG13 of embodiments 1 to 3, when the outputs of the softmax function of the case where the inference results are correct and the case where the inference results are incorrect are sorted in order from large to small, and the average is taken, as follows.
[0177] Correct answer
[0178] [0.937, 0.05, 0.01, 0.003, 0.0012, 0.00051, 0.00022, 0.0001, 0.00005, 0.00002]
[0179] Non-correct solution
[0180] [0.702, 0.207, 0.0563, 0.021, 0.0079, 0.0032, 0.0013, 0.00065, 0.00032, 0.00015]
[0181] In this case, in the learning device shown in Embodiments 1 to 3, the label that matches the value of the maximum output of the softmax function is output as an inference candidate, as in the usual learning device. However, although there is a clear difference between 0.937 when it is determined to be a correct answer and 0.702 when it is determined to be an incorrect answer, the processing is the same, and in the case of an incorrect answer, it can be considered that the information of the candidates other than the inference candidate is discarded. That is, in the case of an incorrect answer, although other inference candidates are also listed as candidates, in the usual learning device, it can be considered that the information is discarded.
[0182] In addition, the total value of the output of the softmax function is normalized to 1, so the output of the softmax function can be treated as the probability that the inference is a correct answer (the accuracy of the inference, the inference value), and not only VGG13, but any learning device can use the softmax function just before the output layer, so as to evaluate with the same index. However, the softmax function is a function represented by an exponential function, so the output is prone to a large difference. During inference, it is also a desirable method to normalize by a method other than an exponential function such as the softmax function.
[0183] The information entropy calculation unit 16 calculates the information entropy based on the average value of the output of the softmax function when the above-mentioned solution is correct and the average value of the output of the softmax function when the solution is incorrect. Thus, the information entropy under each condition can be calculated. Even in the above-mentioned average value, the information entropy when the solution is correct becomes a value smaller than the information entropy when the solution is incorrect.
[0184] Fig. 20 The flowchart of the following processing is shown: after obtaining a test data set that can be multi-valued classified and contains label errors (step ST81), the process of calculating the average of the inference value when the inference is correct and the average of the inference value when the inference is incorrect (step ST83, step ST85), and the information entropy when the inference is correct (step ST82S is yes) is calculated by the information entropy formula, that is, H_correct (the first information entropy) and the information entropy when the inference is incorrect (step ST82 is no) is calculated (step ST84, step ST86, step ST88). In this way, when the inference result is obtained, the degree of certainty of the inference can be calculated based on the information entropy.
[0185] <Threshold>
[0186] Fig.21 Shown in Fig. 20 A flowchart of a process of performing inference based on the threshold value set by the threshold setting unit 17 based on the information entropy obtained in . Fig.21 In the process, the test data is inferred by the first learning device (step ST92), and when the output result of the softmax function of the inference result is larger than the information entropy that becomes the threshold value (yes in step ST93), the second learning device that performs N value classification is used (step ST94, step ST95). However, it is not necessary to use the second learning device, and a learning device that has learned using the first data set or a learning device that has learned the third data set using an algorithm different from that of the first learning device may be used.
[0187] At this time, although the setting of the threshold becomes a parameter, it is desirable to determine the value of the threshold between H_correct and H_wrong. This is because values smaller than H_correct are less likely to be incorrect solutions, and values larger than H_wrong are more likely to be incorrect solutions. The amount of data allocated to H_wrong is small, making it difficult to improve the inference performance. Among the results inferred by the first learning device through such processing, for the results with small information entropy and high accuracy, the results are output, and for the results with large information entropy and low accuracy, the results inferred by different learning devices are output, thereby showing that the inference accuracy can be improved.
[0188] <Learning device for threshold determination>
[0189] In a learning device that performs threshold judgment, when a learning device that has learned using the first data set is used instead of the first learning device, when the data set has few label errors, it is easy to result in a relatively small information entropy, and the width between H_correct and H_wrong also becomes smaller, and a higher inference result can be obtained.
[0190] When a learning device that has been trained using the second data set is used instead of the first learning device, the data is labeled incorrectly, so when there are many outliers in the test data, the information entropy tends to become large, but when it is assumed that there are few outliers in the test data, a higher inference result can be obtained. When a learning device that has been trained using the fourth data set is used instead of the first learning device, when it is assumed that there are many outliers in the test data, a higher inference result can be obtained.
[0191] <Model for input data above the threshold>
[0192] Even in Fig.21 When the value becomes above the threshold, it is not necessary to use the second learning device. As in the case described in the above-mentioned <learning device for performing threshold judgment>, reasoning can also be performed by a learning device that has learned using the first data set, the third data set, and the fourth data set.
[0193] As shown in Implementation 2 of <Deforming the Input Data that Becomes a Deviation Value>, when the information entropy becomes above a threshold, deformation may be performed until the information entropy becomes below the threshold, and the label below the threshold may be output as an inference value. In addition, the deformation method may be changed according to the inference candidate. For example, when the inference candidate is determined to be an apple, since the apple is close to a circle, it is necessary to be able to identify it as an apple even if it is rotated. On the other hand, when the inference candidate is determined to be a car, since it is unrealistic for the car to be reversed 90 degrees, it is expected that even if the rotation angle is large, it will be around ±10 degrees. In this way, by deforming in a way that matches the actual conditions, the inference accuracy can be improved.
[0194] <For input data above the threshold, use another model until it falls below the threshold>
[0195] Alternatively, a fifth learning device composed of a plurality of learning devices different from the first learning device may be constructed for the result of the threshold determination, and the learning device may repeatedly perform inference on input data that is above the threshold until a value below the threshold is output. However, depending on the input data, there may be a case where convergence does not occur. Therefore, in this case, when a determination cannot be made even if all the learning devices perform inference, an indeterminate value is output, or a determination is made by majority decision based on the output results of the plurality of learning devices of the fifth learning device, and an inference value is output based on the inference result of the learning device that outputs the minimum information entropy among the plurality of learning devices of the fifth learning device.
[0196] Implementation method 5.
[0197] <Outline>
[0198] When the information entropy described in Implementation 4 is used, existing integrated reasoning can be performed efficiently. In integrated reasoning, two or more learning devices that have been trained are prepared for the same data set, and one input data is reasoned by the different learning devices, and the sum of the inference results or the majority decision is taken as the inference result. However, there are usually differences in the inference accuracy for input data in different learning devices. In this regard, the present implementation shows that the higher the inference accuracy, the higher the weight is assigned and the sum is taken, so that the inference accuracy can be improved.
[0199] <Integrated Reasoning>
[0200] Integrated reasoning takes the sum of the results of multiple reasoning results. In this embodiment, in addition to VGG13, Resnet18 and Densenet121 are also used for integrated reasoning. In addition, integrated reasoning can also use the softmax function, but when the softmax function is used, it is normalized and processed by an exponential function. Therefore, it has a tendency to depend on specific reasoning results (such as VGG13), and the reasoning accuracy is difficult to improve.
[0201] In contrast, if the result of 10-value classification is output by fully connecting before the softmax function, higher inference accuracy can be obtained. The average value of the inference results before softmax for 10,000 test data of CIFAR-10 for VGG13, Resnet18, and Densenet121 is shown.
[0202] VGG13:
[0203] [6.033, 1.100, 0.5481, 0.2501, -0.0525, -0.3022, -0.594, -1.216, -2.329, -3.436]
[0204] Resnet18:
[0205] [5.507, 0.318, -0.265, -0.492, -0.619, -0.746, -0.839, -0.917, -0.953, -0.993]
[0206] Densenet121:
[0207] [5.004, 0.07, -0.369, -0.495, -0.568, -0.647, -0.704, -0.748, -0.767, -0.784]
[0208] Next, the average value of the output results when the answer is correct is shown below.
[0209] VGG13:
[0210] [6.199, 1.015, 0.5345, 0.2569, -0.0423, -0.2905, -0581, -1.219, -2.37, -3.50]
[0211] Resnet18:
[0212] [5.616, 0.2178, -0.3013, -0.5036, -0.6195, -0.7430, -0.8329, -0.9088, -0.9428, -0.9815]
[0213] Densenet121:
[0214] [5.070, 0.007, -0.385, -0.497, -0.5673, -0.645, -0.700, -0.7425, -0.761, -0.778]
[0215] Next, the average value of the output results when there is an incorrect answer is shown below.
[0216] VGG13:
[0217] [4.003, 2.1348, 0.7128, 0.1674, -0.1769, -0.444, -0.754, -1.181, -1.805, -2.656]
[0218] Resnet18:
[0219] [4.044, 1.666, 0.217, -0.337, -0.614, -0.7903, -0.920, -1.030, -1.089, -1.146]
[0220] Densenet121:
[0221] [3.953, 1.222, -0.1148, -0.469, -0.583, -0.675, -0.766, -0.831, -0.855, -0.880]
[0222] This result shows that the larger the value, the more accurate it is, and the more negative the value, the further away from the prediction. Therefore, a general learning device outputs an inference value that matches the maximum value.
[0223] In this way, the information entropy for the average value can be calculated by calculating the average value of the inference results. In addition, regardless of the average value, the correct answer, or the incorrect answer, the maximum value becomes a close value in the three learning devices, so it is not easy to rely on a certain inference result as in the case of implementing the softmax function. In addition, in the above example, the information entropy of the average value of VGG13 is 1.1, the information entropy of the average value of Resnet18 is 0.90, and the information entropy of the average value of Densenet121 is 0.83.
[0224] Next, regarding the inference results for the test data in each learning device, VGG13 is 92.39%, Resnet18 is 93.07%, and Densenet121 is 94.06%. According to the result, it can be seen that the order of inference accuracy from high to low is Densenet121, Resnet18, and VGG13. It can also be seen that the order of information entropy from small to large is Densenet121, Resnet18, and VGG13. Therefore, it can be confirmed that the higher the inference accuracy of the learning device, the easier it is for the information entropy to become smaller. Even if different data sets or different learning devices are used for verification, the trend is the same. Therefore, when information entropy is used as a weight, the accuracy of integrated reasoning can be improved.
[0225] <Use of information entropy in ensemble learning>
[0226] When the sum of the inference results of the above-mentioned Densenet121, Resnet18, and VGG13 is taken for comparison with the correct answer label, the inference accuracy is 94.59%. In this regard, the higher the inference accuracy of the learning device, the smaller the information entropy. Therefore, the inference accuracy can be improved by setting the inverse of the information entropy as the weight contained in the function. That is, based on a function such as f(·), when the information entropy of VGG13 is set to entropy1, the information entropy of Resnet18 is set to entropy2, and the information entropy of Densenet121 is set to entropy3, the calculation is performed according to f(1 / entropy1)*VGG13+f(1 / entropy2)*Resnet18+f(1 / entropy3)*Densenet121, thereby improving the inference accuracy. As an example, when f(·) is an identity mapping, f(x)=x, so it can be calculated according to (1 / entropy1)*VGG13+(1 / entropy2)*Resnet18+(1 / entropy3)*Densenet121.
[0227] When integrated reasoning is performed based on this formula, the reasoning accuracy of information entropy is 94.65%, which is 0.06% higher than the 94.59% of integrated reasoning. In addition, after the softmax function is implemented, the sum is 94.39% without applying the above weights, which is 0.2% lower than the 94.59% of the comparison object.
[0228] <Ensemble reasoning algorithm using information entropy>
[0229] Fig. 22The flowchart of integrated reasoning is shown. For test data that can be multi-valued classified like the first data set (step ST81), reasoning is performed by two or more information processing devices, and two or more reasoning results are output (step ST02). Then, information entropy is calculated based on the average value of the output results (step ST03, step ST04), and the function containing the inverse of information entropy in the component is used as a weight and multiplied with the output results of each learning device and then summed (step ST05, step ST06), thereby being able to form integrated reasoning using information entropy.
[0230] Although the improvement in inference accuracy is small, it has the effect of being able to improve inference accuracy through simple calculations. In addition, when higher inference accuracy is required, for example, more than 10 learning devices are sometimes combined for inference, but depending on the learning devices incorporated, there are also learning devices that cause deterioration in inference accuracy. In the past, optimization was performed by learning using human rules of thumb, more weight parameters, and full connections connecting learning devices, but it can be processed by a method based on information entropy, so optimization is not required. In addition, even in the case of optimizing weights when higher inference accuracy is required, the optimization problem can be solved based on values close to the optimal value, so the optimal value of the weight of each learning device can be obtained with a smaller number of calculations.
[0231] In addition, the present disclosure allows free combination of the various embodiments, modification of arbitrary components of the various embodiments, or omission of arbitrary components in the various embodiments.
[0232] Industrial Applicability
[0233] The information processing device of the present disclosure can be used to classify input data.
[0234] Description of Reference Numerals
[0235] 11 data conversion unit, 12 feature extraction unit, 13 similar data classification unit, 14 model generation unit, 15 input data classification unit, 16 information entropy calculation unit, 100, 200, 300, 400, 500 information processing device.
Claims
1. An information processing device, characterized in that: The information processing device comprises: A feature extraction unit that extracts a feature of input data; a similar data classification unit that classifies a plurality of input data whose feature quantities are similar to each other among the plurality of input data included in the first data set into a second data set based on the first data set including the plurality of input data and the feature quantities respectively extracted by the feature quantity extraction unit for the plurality of input data included in the first data set; as well as The model generation unit generates a first learned model which is a learned model for classifying input data using the second data set.
2. The information processing device according to claim 1, characterized in that The first data set and the second data set include a plurality of correct answer labels corresponding to each input data. The similar data classification unit generates a seventh data set by excluding, from the second data set, input data corresponding to correct answer labels other than a correct answer label corresponding to the largest number of input data among a plurality of correct answer labels included in the second data set.
3. The information processing device according to claim 1, characterized in that The model generation unit includes an input data classification unit that uses the first learned model to perform inference on specific input data that is not used to generate the first learned model.
4. The information processing device according to claim 3, characterized in that When a specific integer greater than or equal to 2 is N, the second data set includes N correct answer labels and a plurality of input data corresponding to the N correct answer labels. When input data among the multiple input data included in the first data set that are not classified as the second data set are set as the third data set, the model generation unit assigns a first label different from the correct answer label included in the second data set to the third data set, and uses a fourth data set composed of the second data set and the third data set to generate a second learned model, which is a learned model for classifying the input data into any label among the N correct answer labels included in the second data set and the first label.
5. The information processing device according to claim 3, characterized in that: When input data not classified into the second data set among a plurality of input data included in the first data set is set as the third data set, The model generation unit generates a third learned model that is learned using input data included in the third data set and a correct answer label corresponding to the input data. The input data classification unit infers the first input data that is not included in the first data set and is determined to be dissimilar by the feature extraction unit, using the third learned model.
6. The information processing device according to claim 4, characterized in that The similar data classification unit performs a predetermined process on the first input data determined to be non-similar or classified as the first label by the feature extraction unit or based on the inference of the second learned model, thereby generating two or more input data, i.e., second input data, from one input data. The input data classification unit classifies the second input data based on the first learned model, and outputs a correct answer label corresponding to the input data that is classified most frequently as a classification result.
7. The information processing device according to claim 4, characterized in that: The similar data classification unit performs a predetermined process on the first input data determined to be non-similar or classified as the first label by the feature extraction unit or based on the inference of the second learned model, thereby generating two or more input data, i.e., second input data, from one input data. The input data classification unit classifies the second input data based on the second learned model, and outputs a correct answer label corresponding to the input data that is most frequently classified among the correct answer labels other than the first label as a classification result.
8. An information processing device, characterized in that: The information processing device comprises: A feature extraction unit that extracts a feature of input data; a similar data classification unit, which classifies a part or all of the plurality of input data included in the first data set into N data sets consisting of a plurality of input data with similar features, based on the first data set including the plurality of input data and the features respectively extracted by the feature extraction unit for the plurality of input data included in the first data set, and assigns N different labels to each of the N data sets; a model generation unit that generates a learned model using a portion of each of the N data sets, the learned model being used to classify input data in a manner corresponding to any of the labels assigned to the N data sets; as well as an input data classification unit that classifies the input data by inference based on the learned model generated by the model generation unit, The similar data classification unit is set to the 5th data set with the value of N as the number of correct answer labels based on the following reasoning accuracy, which is the reasoning accuracy of the input data classification unit when classifying the input data not used by the model generation unit in the generation of the learned model in the N data sets through reasoning based on the learned model generated by the model generation unit.
9. The information processing device according to claim 8, characterized in that: The first data set includes M correct answer labels and a plurality of input data corresponding to the M correct answer labels, where M is a specific integer greater than 2. The similar data classification unit sets the fifth data set to the minimum N that is greater than or equal to M and at which the inference accuracy becomes maximum with respect to the number of classifications.
10. The information processing device according to claim 8, characterized in that: In the input data classification unit, a sixth data set that is different from the first data set and has no correct answer label is used as input data.
11. The information processing device according to claim 8, characterized in that: When input data that is not classified as the fifth data set by the similar data classification unit among a plurality of input data included in the first data set is set as a non-classified data set, the similar data classification unit assigns a second label different from the labels assigned to the fifth data set to the non-classified data set, The model generation unit generates a fourth learned model using the fifth data set and the non-classified data set, wherein the fourth learned model is a learned model for classifying input data in a manner corresponding to the labels respectively assigned to the fifth data set and any of the second labels.
12. The information processing device according to claim 1, characterized in that: The first data set used for learning includes 5% or more errors in correct answer labels.
13. An information processing device, characterized in that: The information processing device comprises: A feature extraction unit that extracts a feature of input data; a similar data classification unit that classifies a plurality of input data whose feature quantities are similar to each other among the plurality of input data included in the first data set into a second data set based on the first data set including the plurality of input data and the feature quantities respectively extracted by the feature quantity extraction unit for the plurality of input data included in the first data set; a model generating unit that generates a plurality of learned models including a first learned model using the second data set, the first learned model being a learned model for classifying input data; an input data classification unit that classifies input data by reasoning based on the learned model generated by the model generation unit; as well as an information entropy calculation unit that calculates a first information entropy based on an average of inference accuracies when the inference result of the input data classification unit is a correct answer, and calculates a second information entropy based on an average of inference accuracies when the inference result is an incorrect answer, When the accuracy of inference based on the first learned model is below a threshold set between the first information entropy and the second information entropy, the input data classification unit infers the input data based on a learned model other than the first learned model generated by the model generation unit.
14. The information processing device according to any one of claims 1 to 13, characterized in that: The similar data classification unit generates a learned model through self-supervised learning based on a result of classification performed by the similar data classification unit.
15. An information processing device, characterized in that: The information processing device has: A feature extraction unit that extracts a feature of input data; a similar data classification unit that classifies a plurality of input data whose feature quantities are similar to each other among the plurality of input data included in the first data set into a second data set based on the first data set including the plurality of input data and the feature quantities respectively extracted by the feature quantity extraction unit for the plurality of input data included in the first data set; a model generation unit that generates a plurality of learned models including a first learned model using the second data set, the first learned model being a learned model for classifying input data; an input data classification unit that classifies the input data by inference based on the learned model generated by the model generation unit; as well as an information entropy calculation unit that calculates information entropy based on an average of the accuracy of inferences performed by the input data classification unit for each of the plurality of learned models that have been learned using the second data set, The input data classification unit uses a function that includes the inverse of the information entropy of each of the multiple learned models as a weight, accumulates the outputs of each of the multiple learned models corresponding to each of the information entropies, and performs inference based on the sum of the accumulated results.
