Prediction program, prediction method, and prediction device

Abstract

To provide a prediction program, a prediction method, and a prediction device that improve prediction accuracy even when using a trained model generated by machine learning using pseudo images.SOLUTION: A prediction program causes a computer to execute: an acquisition step of acquiring a captured image; a generation step of performing image processing on the acquired image to generate a training image; a training step of generating a trained model by machine learning using the generated training image; and a prediction step of inputting a target image into the trained model and outputting a prediction result. In the prediction step, image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input into the trained model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a prediction program, a prediction method, and a prediction device. [Background technology]

[0002] Patent Document 1 discloses a learning data generation device that includes a defective part data storage unit that stores defective part data, which is image data of defective part images; a good product data storage unit that stores good product data, which is image data of good product images; a learning data generation unit that generates learning data, which is image data of learning images in which good product images are combined with defective part images, based on the defective part data, good product data, and generation parameters; a learning data storage unit that stores the learning data; and a generation parameter setting unit that sets the generation parameters. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-27424 Summary of the Invention [Problem to be solved by the invention]

[0004] One method for inspecting an item involves inputting an image containing the item into a trained model generated by machine learning, and inspecting the item according to the prediction results output from the trained model. To improve prediction accuracy, it is preferable to generate a trained model using a large amount of training data. However, in an actual inspection process, defective products are not often produced, making it difficult to obtain many defective images (actual images) of defective products. Therefore, it takes time to obtain many defective images, and therefore it takes time to generate a trained model. Therefore, as described above, pseudo images of defective images are generated, and machine learning of the training model is performed using the pseudo images as training data.

[0005] However, trained models generated by machine learning using pseudo-images are trained to specialize in the characteristics of the pseudo-images, which means that while the prediction accuracy for pseudo-images improves, the prediction accuracy for real images decreases.

[0006] The present disclosure aims to provide a prediction program, a prediction method, and a prediction device that can improve prediction accuracy even when using a trained model generated by machine learning using pseudo-images. [Means for solving the problem]

[0007] A prediction program according to one embodiment of the present disclosure is a prediction program that is executed on at least one processor and predicts an event using a trained model, and causes a computer to execute the following steps: an acquisition step of acquiring a captured image; a generation step of performing image processing on the acquired image to generate a training image; a learning step of generating a trained model through machine learning using the generated training image; and a prediction step of inputting a target image into the trained model and outputting a prediction result; in the prediction step, image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input into the trained model. [Effects of the Invention]

[0008] According to the present disclosure, prediction accuracy can be improved even when using a trained model generated by machine learning using pseudo images. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the hardware configuration of a computer used in a prediction system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a functional configuration of the prediction system. [Figure 3] FIG. 3 is a diagram illustrating an example of a neural network. [Figure 4]FIG. 4 is a diagram illustrating an example of image processing. [Figure 5] FIG. 5 is a diagram illustrating an example of image processing. [Figure 6] FIG. 6 is a flowchart showing an example of a learning method and generation of a trained model as a processing flow. [Figure 7] FIG. 7 is a flowchart showing an example of a prediction method as a processing flow. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Description of the embodiments of the present disclosure] First, the contents of the embodiments of the present disclosure will be listed and described. At least some of the embodiments described below may be combined in any combination.

[0011] (1) A prediction program according to one aspect of the present disclosure is a prediction program that is executed by at least one processor and predicts an event using a trained model, and causes a computer to execute the following steps: an acquisition step of acquiring a captured image; a generation step of performing image processing on the acquired image to generate a training image; a learning step of generating a trained model through machine learning using the generated training image; and a prediction step of inputting a target image into the trained model and outputting a prediction result. In the prediction step, image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input into the trained model.

[0012] In a prediction program according to one aspect of the present disclosure, in the prediction step, a target image is subjected to image processing using the same method as that used to generate a training image (pseudo image), and the target image after the image processing is input to a trained model. This allows the prediction program to reduce the difference between the target image and the training image used in machine learning. Therefore, the prediction program can improve prediction accuracy even when using a trained model generated by machine learning using pseudo images.

[0013] (2) In the prediction program of (1) above, the generation step may generate training images using a neural network including an input layer, one or more intermediate layers, and an output layer. With this configuration, training images can be appropriately generated based on images.

[0014] (3) In the prediction program of (1) or (2) above, the generation step may generate training images that are identical in appearance to the image. For example, many images of non-defective products can be obtained in an inspection process, etc. Therefore, it is not necessary to generate training images of various types for non-defective product images. Therefore, the generation step generates training images that are identical in appearance to the non-defective product images. This reduces the processing load for generating training images in the prediction program.

[0015] (4) In the prediction program of any one of (1) to (3), the acquisition step may acquire a defective image containing a defective part as the image, and the generation step may generate a learning image in which at least one of the shape, size, position, and color of the defective part in the defective image is changed. With this configuration, learning images of various forms can be generated.

[0016] (5) In any one of the prediction programs (1) to (4) above, the generation step may generate multiple training images from a single image. In this configuration, multiple training images can be generated, allowing machine learning to be performed using multiple training images. This improves the accuracy of the trained model.

[0017] (6) A prediction method according to one aspect of the present disclosure is a prediction method executed by at least one processor and predicts an event using a trained model, and includes an acquisition step of acquiring a captured image, a generation step of performing image processing on the acquired image to generate a training image, a learning step of generating a trained model by machine learning using the generated training image, and a prediction step of inputting a target image into the trained model and outputting a prediction result, in which image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input into the trained model.

[0018] In a prediction method according to one aspect of the present disclosure, in the prediction step, a target image is subjected to the same image processing as that used to generate a training image (pseudo image), and the target image after image processing is input to a trained model. This makes it possible for the prediction method to reduce the difference between the target image and the training image used in machine learning. Therefore, the prediction method can improve prediction accuracy even when using a trained model generated by machine learning using pseudo images.

[0019] (7) A prediction device according to one aspect of the present disclosure is a prediction device that includes at least one processor and predicts an event using a trained model, and includes: an acquisition unit that acquires a captured image; a generation unit that performs image processing on the image acquired by the acquisition unit to generate a training image; a learning unit that generates a trained model through machine learning using the training image generated by the generation unit; and a prediction unit that inputs a target image to the trained model and outputs a prediction result, wherein the prediction unit performs image processing on the target image using the same method as the image processing, and inputs the target image after the image processing to the trained model.

[0020] In a prediction device according to one aspect of the present disclosure, a prediction unit performs image processing on a target image that is the same as the image processing used to generate a training image (pseudo image), and inputs the processed target image into a trained model. This allows the prediction device to reduce the difference between the target image and the training image used in machine learning. Therefore, the prediction device can improve prediction accuracy even when using a trained model generated by machine learning using pseudo images.

[0021] [Details of the embodiments of the present disclosure] Specific examples of embodiments of the present disclosure will be described below with reference to the drawings. The present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.

[0022] [System Overview] The prediction system 1 according to this embodiment is, for example, a computer system that predicts the presence or absence of defects in the inspection of an item. The inspection of the item is, for example, an appearance inspection of the item. In this embodiment, the result predicted by the prediction system 1 is referred to as the "prediction result."

[0023] The prediction system 1 uses machine learning to predict the presence or absence of defects. Machine learning is a technique for autonomously finding laws or rules by iteratively (repeatedly) learning based on given information. The prediction system 1 performs machine learning using a machine learning model. For example, the prediction system 1 may perform machine learning using a gradient boosting decision tree (GBDT), machine learning using multiple regression analysis, or machine learning using a convolutional neural network (CNN) configured to include a convolutional layer and a pooling layer. A convolutional neural network is a type of deep learning that uses a neural network with a multilayer structure.

[0024] The prediction system 1 trains a machine learning model by repeating learning, and acquires this machine learning model as a trained model. This corresponds to the learning phase. In the learning phase, the prediction system 1 functions as a learning device. The trained model is a machine learning model that is predicted to be optimal for predicting the presence or absence of defects. The prediction system 1 processes input data using the trained model and outputs a prediction result of the presence or absence of defects, which corresponds to the operation phase (prediction phase). In the operation phase, the prediction system 1 functions as a prediction device.

[0025] Trained models are portable between computer systems. Therefore, a trained model generated in one computer system can be used in another computer system. Of course, one computer system may execute both the generation and use of trained models. That is, the prediction system 1 may execute both the learning phase and the operation phase, or may not execute either the learning phase or the operation phase. In this embodiment, the prediction system 1 executes both the learning phase and the operation phase.

[0026] In the learning phase, the prediction system 1 uses training data. The training data is image data. The image data may include images of good products and images of defective products. A good product image is an image that does not contain defective parts. A defective image is an image that contains defective parts. The prediction system 1 generates a trained model by performing machine learning using the training data. In the operation phase, the prediction system 1 obtains prediction results by providing input data to the trained model. The input data is image data captured during an inspection process, etc.

[0027] [System Configuration] FIG. 1 is a diagram illustrating an example of a general hardware configuration of a computer 100 constituting the prediction system 1. For example, the computer 100 includes a processor 101, a main memory 102, an auxiliary memory 103, a communication control unit 104, an input device 105, and an output device 106. The processor 101 executes an operating system and application programs. The main memory 102 is composed of, for example, ROM and RAM. The auxiliary memory 103 is composed of, for example, a hard disk or flash memory, and generally stores a larger amount of data than the main memory 102. The communication control unit 104 is composed of, for example, a network card or a wireless communication module. The input device 105 is composed of, for example, a keyboard, a mouse, a touch panel, etc. The output device 106 is composed of, for example, a monitor and a speaker.

[0028] Each functional element of the prediction system 1 is realized by a learning program 110 and a prediction program 120 pre-stored in the auxiliary storage unit 103. Specifically, each functional element is realized by loading the learning program 110 or the prediction program 120 onto the processor 101 or the main storage unit 102 and executing the learning program 110 or the prediction program 120. The processor 101 operates the communication control unit 104, the input device 105, or the output device 106 in accordance with the learning program 110 or the prediction program 120, and reads and writes data from and to the main storage unit 102 or the auxiliary storage unit 103. Data or databases required for processing are stored in the main storage unit 102 or the auxiliary storage unit 103.

[0029] The learning program 110 and the prediction program 120 may be provided by being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, the learning program 110 and the prediction program 120 may be provided via a communication network as a data signal superimposed on a carrier wave.

[0030] The prediction system 1 may be configured with one computer 100 or multiple computers 100. When multiple computers 100 are used, these computers 100 are connected via a communication network such as the Internet or an intranet, thereby logically constructing a single prediction system 1.

[0031] Fig. 2 is a diagram illustrating an example of the functional configuration of the prediction system 1. As illustrated in Fig. 2, the prediction system 1 includes an acquisition unit 10, a generation unit 11, a learning unit 12, a storage unit 13, a prediction unit 14, and a database 20.

[0032] The acquisition unit 10 is a functional element that acquires image data. The acquisition unit 10 acquires image data from a database 20. The acquisition unit 10 can access the database 20 that stores the image data. The database 20 can be used to train a learning model. The database 20 may be, for example, a component of the prediction system 1, or may be built in a computer system separate from the prediction system 1. The prediction system 1 and the database 20 may be connected via a communication network, or both the prediction system 1 and the database 20 may be built in a single computer.

[0033] There is no limitation on the method for preparing the image data to be stored in the database 20. For example, the image data may be stored in the database 20 by an operator's input work, or may be automatically collected by the prediction system 1 or another computer system and stored in the database 20. The database 20 stores, as image data, images of non-defective products and images of defective products that have actually been photographed. The acquisition unit 10 outputs the acquired image data to the generation unit 11.

[0034] The generation unit 11 is a functional element that generates training images. The generation unit 11 performs image processing on images of the image data to generate training images (pseudo images). The generation unit 11 generates training images for each of good product images and defective images. The training images are pseudo-good product images and pseudo-defective images. The generation unit 11 can generate one or more pseudo-good product images based on one good product image. The generation unit 11 can generate one or more pseudo-defective images based on one defective image. In this embodiment, the generation unit 11 generates training images using a neural network.

[0035] FIG. 3 is a diagram showing an example of a neural network NW. As shown in FIG. 3, the neural network NW is composed of, for example, a first layer which is an input layer L1, a second layer, a third layer, and a fourth layer which are intermediate layers (hidden layers) L2, and a fifth layer which is an output layer L3. The first layer outputs an input value x=(x0, x1, x2, ... xp) having p parameters as elements to the second layer as is. Each of the second, third, and fourth layers converts the total input into an output using an activation function and passes the output to the next layer. The fifth layer also converts the total input into an output using an activation function, and this output is the output value y=(y0, y1, ... , yp) of the neural network NW having q parameters as elements. q )

[0036] In this embodiment, the generation unit 11 generates a pseudo-good product image with no visual change based on a good product image. FIG. 4 is a diagram showing an example of pseudo-good product image generation by the generation unit. As shown in FIG. 4, the generation unit 11 generates a pseudo-good product image G2 based on a good product image G1. The generation unit 11 generates a pseudo-good product image G2 with no visual change from the good product image G1 based on the good product image G1. The generation unit 11 generates the pseudo-good product image G2 by adding noise to the good product image G1.

[0037] In this embodiment, the generation unit 11 generates a pseudo-failure image based on a defective image by changing at least one of the shape, size, position, and color of the defective part. The properties of the defective part, such as the shape, size, position, and color, can be set arbitrarily or can be selected randomly. FIG. 5 is a diagram showing an example of pseudo-failure image generation by the generation unit. As shown in FIG. 5, the generation unit 11 generates a pseudo-failure image G4 based on a defective image G3. Based on the defective image G3, the generation unit 11 generates a pseudo-failure image G4 that has a different appearance (defective part DP) from the defective image G3. The generation unit 11 adds noise to the defective image G3 to generate the pseudo-failure image G4.

[0038] The learning unit 12 is a functional element that performs machine learning. The learning unit 12 performs machine learning of the neural network NW. The learning unit 12 performs machine learning of a learning model using images including the pseudo-good product images G2 and the pseudo-defective images G4 generated by the generation unit 11. The learning unit 12 inputs learning data to the input layer L1 and repeatedly (iteratively) performs a learning process on the neural network NW to generate a trained model. The learning unit 12 stores the trained model obtained by repeating the learning process a predetermined number of times in the memory unit 13.

[0039] The prediction unit 14 is a functional element that predicts the presence or absence of defects using a trained model. The prediction unit 14 performs image processing on input data (target image) using the same method as the image processing performed on the image by the generation unit 11. Based on the input data, the prediction unit 14 generates processed data that is identical in appearance to the input data. The prediction unit 14 generates processed data to which noise has been added to the input data. In this embodiment, the prediction unit 14 generates processed data from the input data using a neural network.

[0040] The prediction unit 14 inputs the processed data into the trained model. In response to inputting the processed data into the neural network NW of the trained model, the prediction unit 14 acquires a prediction result including an output value output from the neural network NW. In this embodiment, the neural network NW inputs pixel values ​​of each pixel of image data, which is the processed data, and outputs information indicating the prediction result. The input layer L1 of the neural network NW is provided with neurons equal to the number of pixels in the image. The output layer L3 of the neural network NW is provided with neurons for outputting information related to the prediction result of the object. The presence or absence of a defect can be predicted based on the output values ​​(likelihood) of the neurons in the output layer.

[0041] [Operation of the prediction system] (Learning phase) The learning method and the method for generating a trained model will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the learning method and the generation of a trained model as a processing flow S1. The processing flow S1 corresponds to the learning phase.

[0042] In step S11, the acquisition unit 10 acquires an image from the database 20. In step S12, the generation unit 11 generates learning images (pseudo-good product image G2 and pseudo-defective product image G4) based on the image.

[0043] In step S13, the learning unit 12 performs machine learning using the training images. In step S14, the learning unit 12 determines whether to terminate learning. If the termination condition for machine learning is satisfied, the learning unit 12 terminates learning, and if the termination condition is not satisfied, the learning unit 12 continues machine learning. The termination condition is set arbitrarily. For example, the termination condition may be set based on an error, or may be set based on the number of training data to be processed, i.e., the number of times learning is performed.

[0044] If learning is to continue (step S14: NO), the learning unit 12 acquires the next training image and performs the processes from step S13 onwards for that training image. If learning is to end (step S14: YES), the learning unit 12 acquires a trained model in step S15. Thus, in the learning phase, the prediction system 1 generates a trained model by performing machine learning using training data.

[0045] (Operational phase) The prediction method will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the prediction method as a processing flow S2. The processing flow S2 corresponds to the operation phase and is an example of the prediction method according to the present disclosure.

[0046] In step S21, the prediction unit 14 acquires input data. In step S22, the prediction unit 14 generates processing data from the input data.

[0047] In step S23, the prediction unit 14 inputs the processed data into the trained model and outputs the prediction result obtained by the trained model. The method of outputting the prediction result by the prediction unit 14 is not particularly limited. For example, the prediction unit 14 may output the prediction result to the output device 106, store it in a predetermined database, or transmit it to another computer system.

[0048] [effect] As described above, in the prediction system 1 according to this embodiment, the prediction unit 14 performs image processing on the input data using the same method as the image processing used to generate training images (pseudo images), and inputs the processed data after the image processing into the trained model. This allows the prediction system 1 to reduce the difference between the input data and the training data used in machine learning. Therefore, the prediction system 1 can improve prediction accuracy even when using a trained model generated by machine learning using pseudo images.

[0049] In the prediction system 1 according to this embodiment, the generation unit 11 generates training images using a neural network NW. In this configuration, training images can be appropriately generated based on image data.

[0050] In the prediction system 1 according to this embodiment, the generation unit 11 generates pseudo-good product images (learning images) that are identical in appearance to good product images. Many good product images can be obtained in an inspection process, etc. Therefore, it is not necessary to generate pseudo images of various forms. Therefore, the generation unit 11 generates pseudo-good product images that are identical in appearance to good product images. This reduces the processing load for generating pseudo-good product images in the prediction system 1.

[0051] In the prediction system 1 according to this embodiment, the generator 11 generates pseudo-failure images by changing at least one of the shape, size, position, and color of the defective portion in a defect image. In an actual inspection process, the occurrence of defective products is not high, so it is difficult to obtain many defect images. With the above configuration, pseudo-failure images of various shapes can be generated.

[0052] Although the embodiments of the present disclosure have been described above, the present disclosure is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present disclosure.

[0053] In the above embodiment, the generator 11 generates training images using a neural network, but the image processing performed by the generator 11 is not limited to this.

[0054] In the above embodiment, the neural network NW is described as being composed of a first layer, which is an input layer L1, a second layer, a third layer, and a fourth layer, which are intermediate layers (hidden layers) L2, and a fifth layer, which is an output layer L3. However, the neural network may include one or more intermediate layers. [Explanation of symbols]

[0055] 1. Prediction system 10…Acquisition part 11...Generation section 12...Study Department 13...Storage section 14...Prediction Section 20...Database 100...Computer 101...Processor 102...Main memory section 103…Auxiliary storage unit 104...Communication control unit 105...Input device 106...Output device 110...Study Program 120...Prediction Program G1...Good quality image G2…pseudo good product image G3...Bad image G4...pseudo-faulty image L1...input layer L2: Intermediate layer (hidden layer) L3: Output layer NW...neural network S1...Processing flow S2...Processing flow S11...Step S12...Step S13...Step S14...Step S15...Step S21...Step S22...Step S23...Step

Claims

1. A prediction program that is executed on at least one processor and predicts an event using a trained model, an acquisition step of acquiring the captured image; a generation step of performing image processing on the acquired image to generate a learning image; A learning step of generating the trained model by machine learning using the generated training image; A prediction step of inputting a target image into the trained model and outputting a prediction result; on the computer, A prediction program in which, in the prediction step, image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input into the trained model.

2. The prediction program according to claim 1 , wherein the generating step generates the training images using a neural network including an input layer, one or more intermediate layers, and an output layer.

3. The prediction program according to claim 1 or 2, wherein the generating step generates the training image that has the same appearance as the image.

4. In the acquiring step, a defective image including a defective portion is acquired as the image, 3. The prediction program according to claim 1, wherein the generating step generates the learning image by changing at least one of the shape, size, position, and color of the defective portion in the defective image.

5. The prediction program according to claim 1 or 2, wherein the generating step generates a plurality of the learning images from one of the images.

6. A method for predicting an event using a trained model, the method being executed on at least one processor, comprising: an acquisition step of acquiring the captured image; a generation step of performing image processing on the acquired image to generate a learning image; A learning step of generating the trained model by machine learning using the generated training image; A prediction step of inputting a target image into the trained model and outputting a prediction result; A prediction method in which, in the prediction step, image processing is performed on the target image using the same method as the image processing, and the target image after the image processing is input to the trained model.

7. A prediction device comprising at least one processor and predicting an event using a trained model, an acquisition unit that acquires the captured image; a generation unit that performs image processing on the image acquired by the acquisition unit to generate a learning image; a learning unit that generates the trained model by machine learning using the training image generated by the generation unit; A prediction unit that inputs a target image into the trained model and outputs a prediction result, The prediction unit performs image processing on the target image using the same method as the image processing, and inputs the target image after the image processing into the trained model.

Images