Learning device, learning method and learning program

Abstract

To provide a learning device, learning method and learning program capable of improving accuracy of a learned model.SOLUTION: A learning device comprises an image acquisition unit that acquires an image, a feature acquisition unit that acquires features of the image acquired by the image acquisition unit for each category, and a learning unit that performs machine learning of a learning model using the images. The learning unit has a plurality of first learning models corresponding to each of a plurality of categories, individually performs machine learning of the first learning models corresponding to each category using images having features belonging to the category, and combines parameters of the plurality of first learning models to generate a second learned model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] Patent Document 1 discloses a learning data generation device that includes a defective part data storage unit that stores defective part data, which is image data of defective part images; a good product data storage unit that stores good product data, which is image data of good product images; a learning data generation unit that generates learning data, which is image data of learning images in which good product images are combined with defective part images, based on the defective part data, good product data, and generation parameters; a learning data storage unit that stores the learning data; and a generation parameter setting unit that sets the generation parameters. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-27424 Summary of the Invention [Problem to be solved by the invention]

[0004] One method for inspecting an item involves inputting an image containing the item into a trained model generated by machine learning, and inspecting the item according to the prediction results output from the trained model. In order to improve prediction accuracy, it is preferable to generate a trained model using a large amount of training data. However, in an actual inspection process, defective products are not often generated, making it difficult to obtain many defective images (actual images) of defective products. Therefore, it takes time to obtain many defective images, and therefore it takes time to generate a trained model. Therefore, as described above, pseudo images of defective images are generated, and machine learning of the training model is performed using the pseudo images as training data.

[0005] In order to improve the accuracy of a trained model, it is necessary to perform machine learning of the trained model using images that have properties with low performance (features that are weak points (weak points)) in the trained model. Therefore, the learning device generates pseudo images related to properties with low performance (such as the size and position of defects) and repeatedly performs machine learning of the trained model using these pseudo images as training data. However, if learning is repeated using only pseudo images with properties with low performance, the accuracy of the properties with low performance may improve, but the accuracy of properties that originally had high performance may relatively decrease. As a result, the accuracy of the trained model may decrease.

[0006] The present disclosure aims to provide a learning device, a learning method, and a learning program that can improve the accuracy of a trained model. [Means for solving the problem]

[0007] A learning device according to one aspect of the present disclosure includes an image acquisition unit that acquires images, a feature acquisition unit that acquires features of the images acquired by the image acquisition unit for each category, and a learning unit that performs machine learning of a learning model using the images, wherein the learning unit has a plurality of first learning models corresponding to each of a plurality of categories, and individually performs machine learning of the first learning models corresponding to the categories using images having features belonging to the categories, and combines parameters of the plurality of first learning models to generate a second learned model. [Effects of the Invention]

[0008] According to the present disclosure, the accuracy of trained models can be improved. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the hardware configuration of a computer used in a prediction system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a functional configuration of the prediction system. [Figure 3]FIG. 3 is a diagram illustrating a learning method in the learning unit. [Figure 4] FIG. 4 is a diagram illustrating an example of a neural network. [Figure 5] FIG. 5 is a flowchart showing an example of a learning method and generation of a trained model as a processing flow. [Figure 6] FIG. 6 is a flowchart showing an example of a prediction method as a processing flow. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Description of the embodiments of the present disclosure] First, the contents of the embodiments of the present disclosure will be listed and described. At least some of the embodiments described below may be combined in any combination.

[0011] (1) A learning device according to one aspect of the present disclosure includes an image acquisition unit that acquires images, a feature acquisition unit that acquires features of the images acquired by the image acquisition unit by category, and a learning unit that performs machine learning of a learning model using the images. The learning unit has a plurality of first learning models corresponding to each of a plurality of categories, and individually performs machine learning of the first learning models corresponding to the categories using images having features belonging to the categories, and combines parameters of the plurality of first learning models to generate a second learned model.

[0012] In a learning device according to one aspect of the present disclosure, the learning unit individually performs machine learning on a first learning model corresponding to a category using an image having characteristics belonging to that category. Therefore, the learning device can generate a first learning model specialized for each of multiple categories. The learning unit then combines parameters from the multiple first learning models to generate a second trained model. In this way, the learning device generates a first learning model specialized for each category and combines parameters from the multiple first learning models to generate a second trained model, thereby preventing performance differences between categories in the second trained model. Therefore, the learning device can improve the accuracy of the trained model.

[0013] (2) In the learning device of (1), the feature acquisition unit may acquire features corresponding to categories based on predetermined prompts. In this configuration, the prompts are set in advance, so that it is possible to avoid missing out on acquiring necessary features.

[0014] (3) A learning method according to one aspect of the present disclosure includes an image acquisition step of acquiring an image, a feature acquisition step of acquiring features of the image acquired in the image acquisition step by category, and a learning step of performing machine learning of a learning model using the image, wherein the learning step has a plurality of first learning models corresponding to each of a plurality of categories, and performs machine learning of the first learning models corresponding to the categories individually using images having features belonging to the categories, and generates a second trained model by combining parameters of the plurality of first learning models.

[0015] In a learning method according to one aspect of the present disclosure, in the learning step, machine learning of a first learning model corresponding to a category is individually performed using images having features belonging to the category. Therefore, the learning method can generate a first learning model specialized for each of multiple categories. Then, in the learning step, parameters of multiple first learning models are combined to generate a second trained model. In this way, the learning method generates a first learning model specialized for each category and combines parameters of multiple first learning models to generate a second trained model, thereby preventing performance differences between categories in the second trained model. Therefore, the learning method can improve the accuracy of the trained model.

[0016] (5) A learning program according to one aspect of the present disclosure causes a computer to execute an image acquisition step of acquiring an image, a feature acquisition step of acquiring features of the image acquired in the image acquisition step by category, and a learning step of performing machine learning of a learning model using the image, wherein the learning step has a plurality of first learning models corresponding to each of a plurality of categories, and performs machine learning of the first learning models corresponding to the category individually using images having features belonging to the category, and combines parameters of the plurality of first learning models to generate a second learned model.

[0017] In a learning program according to one aspect of the present disclosure, in the learning step, machine learning of a first learning model corresponding to a category is individually performed using images having features belonging to the category. Therefore, the learning method can generate a first learning model specialized for each of multiple categories. Then, in the learning step, parameters of multiple first learning models are combined to generate a second trained model. In this way, the learning program generates a first learning model specialized for each category and combines parameters of multiple first learning models to generate a second trained model, thereby preventing performance differences between categories in the second trained model. Therefore, the learning program can improve the accuracy of the trained model.

[0018] [Details of the embodiments of the present disclosure] Specific examples of embodiments of the present disclosure will be described below with reference to the drawings. The present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.

[0019] The prediction system 1 according to this embodiment is, for example, a computer system that predicts the presence or absence of defects in the inspection of an item. The inspection of the item is, for example, an appearance inspection of the item. In this embodiment, the result predicted by the prediction system 1 is referred to as the "prediction result."

[0020] The prediction system 1 uses machine learning to predict the presence or absence of defects. Machine learning is a technique for autonomously finding laws or rules by iteratively (repeatedly) learning based on given information. The prediction system 1 performs machine learning using a machine learning model. For example, the prediction system 1 may perform machine learning using a gradient boosting decision tree (GBDT), machine learning using multiple regression analysis, or machine learning using a convolutional neural network (CNN) configured to include a convolutional layer and a pooling layer. A convolutional neural network is a type of deep learning that uses a neural network with a multilayer structure.

[0021] The prediction system 1 trains a machine learning model by repeating learning, and acquires this machine learning model as a trained model. This corresponds to the learning phase. In the learning phase, the prediction system 1 functions as a learning device. The trained model is a machine learning model that is predicted to be optimal for predicting the presence or absence of defects. The prediction system 1 processes input data using the trained model and outputs a prediction result of the presence or absence of defects, which corresponds to the operation phase (prediction phase). In the operation phase, the prediction system 1 functions as a prediction device.

[0022] Trained models are portable between computer systems. Therefore, a trained model generated in one computer system can be used in another computer system. Of course, one computer system may execute both the generation and use of trained models. That is, the prediction system 1 may execute both the learning phase and the operation phase, or may not execute either the learning phase or the operation phase. In this embodiment, the prediction system 1 executes both the learning phase and the operation phase.

[0023] In the learning phase, the prediction system 1 uses training data. The training data is image data. The image data may include good product images, defective images, and pseudo images. Good product images and defective images are real images that have been captured. Good product images are images that do not contain defective parts. Defective images are images that contain defective parts. Pseudo images include pseudo good product images and pseudo defective images. The prediction system 1 generates a trained model by performing machine learning using the training data. In the operation phase, the prediction system 1 obtains prediction results by providing input data to the trained model. The input data is image data captured during an inspection process, etc.

[0024] [System Configuration] FIG. 1 is a diagram illustrating an example of a general hardware configuration of a computer 100 constituting the prediction system 1. For example, the computer 100 includes a processor 101, a main memory 102, an auxiliary memory 103, a communication control unit 104, an input device 105, and an output device 106. The processor 101 executes an operating system and application programs. The main memory 102 is composed of, for example, ROM and RAM. The auxiliary memory 103 is composed of, for example, a hard disk or flash memory, and generally stores a larger amount of data than the main memory 102. The communication control unit 104 is composed of, for example, a network card or a wireless communication module. The input device 105 is composed of, for example, a keyboard, a mouse, a touch panel, etc. The output device 106 is composed of, for example, a monitor and a speaker.

[0025] Each functional element of the prediction system 1 is realized by a learning program 110 and a prediction program 120 pre-stored in the auxiliary storage unit 103. Specifically, each functional element is realized by loading the learning program 110 or the prediction program 120 onto the processor 101 or the main storage unit 102 and executing the learning program 110 or the prediction program 120. The processor 101 operates the communication control unit 104, the input device 105, or the output device 106 in accordance with the learning program 110 or the prediction program 120, and reads and writes data from and to the main storage unit 102 or the auxiliary storage unit 103. Data or databases required for processing are stored in the main storage unit 102 or the auxiliary storage unit 103.

[0026] The learning program 110 and the prediction program 120 may be provided by being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the learning program 110 and the prediction program 120 may be provided via a communication network as a data signal superimposed on a carrier wave.

[0027] The prediction system 1 may be configured with one computer 100 or multiple computers 100. When multiple computers 100 are used, these computers 100 are connected via a communication network such as the Internet or an intranet, thereby logically constructing a single prediction system 1.

[0028] Fig. 2 is a diagram showing an example of the functional configuration of the prediction system 1. As shown in Fig. 2, the prediction system 1 includes an image acquisition unit 10, a feature acquisition unit 11, an image generation unit 12, a learning unit 13, a storage unit 14, a prediction unit 15, and a database 20.

[0029] The image acquisition unit 10 is a functional element that acquires image data. The image acquisition unit 10 acquires image data from a database 20. The image acquisition unit 10 can access the database 20 that stores image data. The database 20 can be used to train a learning model. The database 20 may be, for example, a component of the prediction system 1, or may be built in a computer system separate from the prediction system 1. The prediction system 1 and the database 20 may be connected via a communication network, or both the prediction system 1 and the database 20 may be built in a single computer.

[0030] There is no limitation on the method for preparing the image data to be stored in the database 20. For example, the image data may be stored in the database 20 by an operator's input work, or may be automatically collected and stored in the database 20 by the prediction system 1 or another computer system. The database 20 stores, as image data, actually captured images of good products and defective products, as well as pseudo images (pseudo good product images and pseudo defective images) generated by the image generation unit 12. The image acquisition unit 10 outputs the acquired image data to the feature acquisition unit 11.

[0031] The feature acquisition unit 11 is a functional element that acquires (extracts) features of image data. The feature acquisition unit 11 acquires features (properties) of the image acquired by the image acquisition unit 10 for each category. The feature acquisition unit 11 inputs the image acquired by the image acquisition unit 10 into a trained model to acquire the image features. The trained model is a machine learning model that finds defective parts from the input image and acquires quantifiable features of the defective parts. A known machine learning model can be used as the trained model. The features acquired by the feature acquisition unit 11 may include "color of the image," "texture of the image," "shape of the defective part (foreign object)," "size of the defective part," "position of the defective part," etc. The feature acquisition unit 11 associates the acquired features with the image for each category and stores them in the storage unit 14.

[0032] The image generation unit 12 is a functional element that generates training images. The image generation unit 12 performs image processing on the image data to generate training images (pseudo images). In this embodiment, the image generation unit 12 generates training images for defective images. The image generation unit 12 can generate one or more pseudo defective images based on one defective image.

[0033] The image generation unit 12 generates a pseudo-failure image including a defective part based on the features for each category stored in the storage unit 14. For example, when the size of the defective part is used as the feature, the image generation unit 12 generates a pseudo-failure image in which the size of the defective part is changed in increments of 0.1 within a range of 0.5 to 5.0 times. Furthermore, when the size, position, and color of the defective part are used as the feature, the image generation unit 12 generates a pseudo-failure image having features such as "double size, center, yellow." The image generation unit 12 may generate a pseudo-failure image based on a category and size specified by an operator. The image generation unit 12 inputs the features and the defective image into a trained model to generate a pseudo-failure image. The trained model is a machine learning model (image generation AI) that generates an image based on an image and features.

[0034] The learning unit 13 is a functional element that performs machine learning. The learning unit 13 performs machine learning of a neural network. The learning unit 13 performs machine learning of a learning model using images including defective images and pseudo-defective images stored in the database 20. The learning unit 13 inputs learning data into the input layer of the neural network, and repeatedly (iteratively) performs a learning process on the neural network to generate a trained model. The learning unit 13 stores the trained model obtained by repeating the learning process a predetermined number of times in the memory unit 14.

[0035] FIG. 3 is a diagram illustrating a learning method in the learning unit 13. As illustrated in FIG. 3, the learning unit 13 has a plurality of first learning models M1 to Mn corresponding to a plurality of categories (features), respectively. The number of first learning models M1 to Mn (the number n) corresponds to the number of categories. The learning unit 13 individually performs machine learning of the first learning models corresponding to the categories using images having features belonging to the categories. In the example illustrated in FIG. 3, for example, the learning unit 13 performs machine learning of the first learning model M1 using an image having the feature prp1, performs machine learning of the first learning model M2 using an image having the feature prp2, and performs machine learning of the first learning model M3 using an image having the feature prp3. The features prp1 to prpn are, for example, the size of the defect part (e.g., 1.2 times), the position of the defect part (e.g., (x, y) = (100, 100)), the hue (e.g., 100), the angle (e.g., 90°), etc.

[0036] The learning unit 13 performs machine learning of the second learning model M based on parameters of the multiple first learning models M1 to Mn, and generates a trained model (second trained model) that combines the parameters. The learning unit 13 generates the trained model by ensemble learning.

[0037] 2, the storage unit 14 is a functional element that stores data and the like. The storage unit 14 stores the features acquired by the feature acquisition unit 11 by category. The categories may be, for example, "color tone of image," "texture of image," "shape of defective part (foreign substance)," "size of defective part," "position of defective part," etc.

[0038] The prediction unit 15 is a functional element that predicts the presence or absence of a defect using a trained model. The prediction unit 15 inputs input data to a second trained model. In response to inputting the input data to the neural network NW (see FIG. 4) of the trained model, the prediction unit 15 acquires a prediction result including an output value output from the neural network NW.

[0039] FIG. 4 is a diagram showing an example of a neural network NW. As shown in FIG. 4, the neural network NW is composed of, for example, a first layer which is an input layer L1, a second layer, a third layer, and a fourth layer which are intermediate layers (hidden layers) L2, and a fifth layer which is an output layer L3. The first layer outputs an input value x=(x0, x1, x2, ... xp) having p+1 parameters as elements to the second layer as is. Each of the second, third, and fourth layers converts the total input into an output using an activation function and passes the output to the next layer. The fifth layer also converts the total input into an output using an activation function, and this output is the output value y=(y0, y1, ... , yp) of the neural network NW having q+1 parameters as elements. q )

[0040] In this embodiment, the neural network NW receives the pixel values ​​of each pixel of image data as input data and outputs information indicating a prediction result. The input layer L1 of the neural network NW is provided with neurons equal to the number of pixels in the image. The output layer L3 of the neural network NW is provided with neurons for outputting information related to the prediction result of the target object. The presence or absence of a defect can be predicted based on the output values ​​(likelihoods) of the neurons in the output layer.

[0041] [Operation of the prediction system] (Learning phase) The learning method and the method for generating a trained model will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the learning method and the generation of a trained model as a processing flow S1. The processing flow S1 corresponds to the learning phase.

[0042] In step S11, the image acquisition unit 10 acquires an image from the database 20. In step S12, the feature acquisition unit 11 acquires features of the image. In step S13, the learning unit 13 performs machine learning using the training images. The learning unit 13 individually performs machine learning of first learning models M1 to Mn corresponding to the category using training images having features belonging to the category.

[0043] In step S14, the learning unit 13 determines whether to terminate learning. If the termination condition for machine learning is satisfied, the learning unit 13 terminates learning, and if the termination condition is not satisfied, the learning unit 13 continues machine learning. The termination condition is set arbitrarily. For example, the termination condition may be set based on the error from the correct answer, or may be set based on the number of training data to be processed, i.e., the number of times learning is performed.

[0044] If learning is to continue (step S14: NO), the learning unit 13 acquires the next training image and performs the processes from step S13 onwards for that training image. If learning is to end (step S14: YES), the learning unit 13 combines the parameters of the first training models M1 to Mn to acquire a trained model in step S15. Thus, in the learning phase, the prediction system 1 generates a trained model by performing machine learning using training data.

[0045] (Operational phase) The prediction method will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the prediction method as a processing flow S2. The processing flow S2 corresponds to the operation phase and is an example of the prediction method according to the present disclosure.

[0046] In step S21, the prediction unit 15 acquires input data. In step S22, the prediction unit 15 inputs the processed data into a trained model (second trained model) and outputs a prediction result obtained by the trained model. The method for outputting the prediction result by the prediction unit 15 is not particularly limited. For example, the prediction unit 15 may output the prediction result to the output device 106, store it in a predetermined database, or transmit it to another computer system.

[0047] [effect] As described above, in the prediction system 1 according to this embodiment, the learning unit 13 individually performs machine learning on the first learning models M1 to Mn corresponding to each category using images having features belonging to the category. Therefore, the prediction system 1 can generate first learning models M1 to Mn specialized for each of multiple categories. Then, the learning unit 13 generates a trained model by combining parameters of the multiple first learning models M1 to Mn. In this way, the prediction system 1 generates first learning models M1 to Mn specialized for each category and generates a trained model by combining parameters of the multiple first learning models M1 to Mn, thereby preventing differences in performance between categories in the trained models. Therefore, the prediction system 1 can improve the accuracy of the trained models.

[0048] Although the embodiments of the present invention have been described above, the present invention is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention.

[0049] In the above embodiment, the feature acquisition unit 11 acquires image features using a trained model that finds defective parts from an input image and acquires quantifiable features of the defective parts. However, the feature acquisition unit 11 may acquire features based on a predetermined prompt. The prompt may be set by an operator or automatically. For example, when a prompt such as "Please find defective parts from the image and provide as many quantifiable features as possible for the defective parts" is input, the feature acquisition unit 11 acquires and outputs text such as "image color," "size of the defective parts," and "location of the defective parts." The trained model is a machine learning model that acquires feature text from an input image based on the prompt. A known machine learning model can be used as the trained model. In this configuration, the prompt is set in advance, which prevents necessary features from being overlooked.

[0050] In the above embodiment, the neural network NW is described as being composed of a first layer, which is an input layer L1, a second layer, a third layer, and a fourth layer, which are intermediate layers (hidden layers) L2, and a fifth layer, which is an output layer L3. However, the neural network may include one or more intermediate layers. [Explanation of symbols]

[0051] 1. Prediction system 10...Image acquisition unit 11...Feature acquisition section 12...Image generation unit 13...Study Department 14...Storage section 15...Prediction Department 20...Database 100...Computer 101...Processor 102...Main memory section 103…Auxiliary storage unit 104...Communication control unit 105...Input device 106...Output device 110...Study Program 120...Prediction program L1...input layer L2…middle layer L3: Output layer M...Second learning model M1: First learning model M2: First learning model M3: First learning model Mn…First learning model NW...neural network

Claims

1. an image acquisition unit that acquires an image; a feature acquisition unit that acquires features of the images acquired by the image acquisition unit for each category; a learning unit that performs machine learning of a learning model using the image, The learning unit a plurality of first learning models corresponding to the plurality of categories, individually performing machine learning of the first learning model corresponding to the category using the image having the feature belonging to the category; A learning device that combines parameters of multiple first learned models to generate a second learned model.

2. The learning device according to claim 1 , wherein the feature acquisition unit acquires the features corresponding to the categories based on a predetermined prompt.

3. an image acquisition step of acquiring an image; a feature acquisition step of acquiring features of the image acquired in the image acquisition step for each category; A learning step of performing machine learning of a learning model using the image, In the learning step, a plurality of first learning models corresponding to the plurality of categories, individually performing machine learning of the first learning model corresponding to the category using the image having the feature belonging to the category; A learning method that combines parameters of multiple first learning models to generate a second learned model.

4. an image acquisition step of acquiring an image; a feature acquisition step of acquiring features of the image acquired in the image acquisition step for each category; a learning step of performing machine learning of a learning model using the image, In the learning step, a plurality of first learning models corresponding to the plurality of categories, individually performing machine learning of the first learning model corresponding to the category using the image having the feature belonging to the category; A learning program that combines parameters of multiple first learned models to generate a second learned model.

Images