Quality determination device, and program and method therefor

Abstract

To determine quality flexibly and accurately.SOLUTION: A quality determination device comprises: a learning unit that forms image data from a small amount of video of good products, and generates a feature amount map for all of the image data for learning; and a determination unit that determines the quality of products in the video by referring to a learning model constructed by learning using the image data of the good products. The learning model inputs the acquired image data into a predetermined network structure, and performs learning based on the feature amount of a compression range that can compress the image without misalignment and without being affected by background, which is extracted from the network structure.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a pass / fail determination device for determining the quality of a product from a video, its program, and method.

Background Art

[0002] For example, Patent Document 1 discloses an identification method, a sorting method, and a sorting device for an object to be sorted that can identify and sort the object to be sorted. At the time of this sorting, machine learning including at least good part information, defective part information, and background information regarding the object to be sorted is performed, and image information of the object to be sorted is acquired during and / or after the transfer process of the object to be sorted to perform a pass / fail determination of the object to be sorted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the system of Patent Document 1, information on defective products is required during machine learning, which is time-consuming and increases the amount of data accordingly.

Means for Solving the Problems

[0005] The invention according to the first aspect is a pass / fail determination device that determines the quality of a product from a video. It includes a learning unit that forms image data from a small amount of good-quality videos, generates and learns a feature map for all of the image data, and a determination unit that refers to a learned model constructed by learning using the image data related to the good-quality products and determines the pass / fail of the product in the video. The learned model is learned based on feature amounts within a compression range that can compress the image without being affected by the background and without displacement, by inputting the acquired image data into a predetermined network structure and extracting the feature amounts from the network structure.

Effect of the Invention

[0006] According to the present invention, by determining the pass / fail of a product based on a video of good-quality products, flexible responses can be made to various products.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Modes for Carrying Out the Invention

[0008] (1) Configuration of the determination device

[0009] FIG. 1 is a schematic diagram showing the configuration of a determination device 20 according to the present embodiment. For convenience, in the following description, it is assumed that the product 5 is a plastic bottle. However, the product 5 of the determination device 20 according to the present embodiment is not limited to this and may be any object.

[0010] The determination device 20 acquires images from the video of the product 5 via the camera 15, and determines whether or not an abnormality has occurred in the product 5 from the images. Further, the determination device 20 transmits the determination result to the user terminal device 30 operated by the monitor. Such a determination device 20 can be realized by an arbitrary computer. Here, the determination device 20 includes a storage unit 21, an input / output unit 22, a communication unit 23, and a processing unit 24. Note that the determination device 20 may be realized as hardware using an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0011] The storage unit 21 stores the image information acquired from the video in which the product 5 appears, and is realized by an arbitrary storage device such as a memory and a hard disk. Specifically, the storage unit 21 stores information for executing the abnormality determination model. The "abnormality determination model" is constructed using a predetermined network structure and normal image data, and determines that it is abnormal when the input image is not normal. Here, the abnormality determination model is constructed by calculating a predetermined statistic from the normal image data using a predetermined network structure. For example, the abnormality determination model can be constructed by a framework such as PaDiM using a network structure such as Wide-ResNet.

[0012] The input / output unit 22 is realized by a keyboard, a mouse, a touch panel, or the like, and inputs various information to the computer and outputs various information from the computer.

[0013] Further, when it is determined that there is an abnormality in the product 5, the input / output unit 22 outputs an image in a manner that the abnormal portion can be recognized. For example, when an abnormality occurs in the image of the product 5, the input / output unit 22 outputs an image in which the degree of abnormality is reflected.

[0014] The communication unit 23 is realized by an arbitrary network card or the like and enables communication with communication devices on the network by wire or wirelessly. When an abnormality occurs in the image of the product 5, the communication unit 23 transmits "caution information" to the user terminal device 30 or the like.

[0015] The processing unit 24 executes various information processes and is realized by a processor such as a CPU or a GPU (Graphics Processing Unit) and a memory. Here, when the program stored in the storage unit 21 is read into the CPU, GPU, etc. of the computer, the processing unit 24 functions as an image acquisition unit 24A, a determination unit 24B, and a learning unit 24C.

[0016] The image acquisition unit 24A acquires an image of the product 5 via the camera 15 or the like, and stores the acquired image in the storage unit 21 at any time.

[0017] The determination unit 24B determines whether the acquired image of the product 5 is abnormal using an abnormality determination model.

[0018] The learning unit 24C constructs an abnormality determination model. Specifically, the learning unit 24C constructs an abnormality determination model using normal video data for learning.

[0019] The construction of the abnormality determination model is performed by deep learning using image data obtained from a video related to the product 5. In deep learning, an image obtained from a video of a non-defective product 5 moving on a belt conveyor is learned. A video is taken for about 5 minutes. Therefore, learning can be performed with a small amount of video.

[0020] Deep learning is performed as follows. Image data is formed from a video of good products moving on a belt conveyor. A feature map is generated for all of the image data. The specific method of generating a feature map from each image data is based on the feature amount extracted from the network structure by inputting the acquired image data into a predetermined network structure. For example, when the acquired image is 224×224 pixels, the image is compressed to 1 / 8 in both vertical and horizontal directions to generate a feature map of 28×28 pixels. In other words, a feature map having feature amounts for each region formed when the vertical and horizontal directions of the acquired image are divided into 28 parts is generated. This compression varies depending on how the detection target appears. If there is displacement or a lot of background, the accuracy drops. Therefore, the inventor has found a compression range in which the image can be compressed without displacement and without being affected by the background. Here, when the acquired image is 224×224 pixels, compressing the image to 1 / 8 in both vertical and horizontal directions was considered the optimal compression. However, if the appearance of the detection target is good, compression to 1 / 14 in both vertical and horizontal directions may also be acceptable. Note that Wide-ResNet or the like can be used as the network structure.

[0021] (2) Operation of the determination device FIG. 2 is a flowchart for explaining the operation of the determination device 20 according to the present embodiment.

[0022] The determination device 20 forms image data from a video of products moving on a belt conveyor. A feature map is generated for all of the image data (R2). The specific method of generating a feature map from each image data is the same as that during deep learning.

[0023] Next, the determination device 20 calculates the degree of abnormality for each divided region (R3). The degree of abnormality is calculated by comparing the N-dimensional normal distribution of the feature amounts of each divided region obtained from the good product image data stored in the storage unit 21 with the feature amounts of each divided region obtained from the acquired image data. Further, the determination device 20 creates a heat map according to the degree of abnormality of each region.

[0024] The method for determining the threshold of abnormality is as follows. N normal images are randomly divided into N1 training images and N2 images for determining the threshold of abnormality. Next, from the N1 normal image data, the average value and covariance matrix of the feature amounts corresponding to each region are obtained. Then, for the N2 verification images, the Mahalanobis distance is calculated based on the average value and covariance matrix calculated above, and the threshold of abnormality is determined based on this (for example, the average value of the N2 Mahalanobis distances + standard deviation * 3).

[0025] When the degree of abnormality in each of the divided regions is greater than the threshold value, the determination device 20 determines that an abnormality has occurred. Each of the divided regions is combined so as to be easy to view visually to obtain an image of a part of the product. In the preprocessing, if there are stored background regions, regions where the light is too strong, edge portions of the product, etc., they may be determined as abnormal even though they are not abnormal. Therefore, if such regions are present in the final abnormality determination location, they are ignored. That is, the final abnormality determination location is the common part between the location detected by the abnormality determination model and the five locations of the detected product obtained in the preprocessing. And an attention signal is output (R4-Yes). The determination device 20 transmits an email or the like stating that an abnormality has occurred to the user terminal device 30 of the monitor in response to the output of the attention signal.

[0026] (3) Effects According to this embodiment, the detection of the quality of the product can be easily realized by a method combining image analysis techniques, rather than introducing a large-scale inspection room with adjusted shooting conditions.

[0027] In addition, since it is an algorithm that learns using only normal images, it is not necessary to prepare a sample of abnormal products in advance.

[0028] <Other Embodiments> The present disclosure is not limited to the above-described embodiments as they are. The present disclosure can be embodied by modifying the components without departing from the gist thereof at the implementation stage. Further, the present disclosure can form various disclosures by appropriately combining a plurality of components disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components may be appropriately combined from different embodiments.

Explanation of Reference Numerals

[0029] 5 Commodity 15 Camera 20 Determination Device 21 Storage Unit 22 Input / Output Unit 23 Communication Unit 24 Processing Unit 24A Image Acquisition Unit 24B Determination Unit 24C Learning Unit

Claims

1. In a pass / fail determination device that determines the quality of a product from a video, a learning unit that forms image data from a small amount of good-quality videos, generates a feature map for all of the image data, and performs learning; a determination unit that determines the pass / fail of the product in the video by referring to a learning model constructed by learning using the image data related to the good-quality product; comprising: The learning model is trained based on feature amounts within a compression range that can compress the image without being affected by background and without misalignment, which are extracted from a predetermined network structure when the acquired image data is input into the predetermined network structure. The pass / fail determination device is characterized by this.

2. The determination unit determines the common part between the location detected by the anomaly determination model for the final anomaly determination location and the detection target location obtained by preprocessing. The pass / fail determination device according to Claim 1.

3. The determination unit randomly divides N normal images into N1 learning images and N2 images for determining an anomaly threshold. Next, from the N1 normal image data, the average value and covariance matrix of the feature amounts corresponding to each region are obtained. For the N2 verification images, the Mahalanobis distance is calculated based on the average value and covariance matrix calculated above, and the anomaly threshold is determined based on this. The pass / fail determination device according to Claim 1 or 2.

4. In a pass / fail determination device that determines the quality of a product from a video, a learning unit that forms image data from a small amount of good-quality videos, generates a feature map for all of the image data, and performs learning; a determination unit that determines the pass / fail of the product in the video by referring to a learning model constructed by learning using the image data related to the good-quality product; comprising: The learning model is trained based on feature amounts within a compression range that can compress the image without being affected by background and without misalignment, which are extracted from a predetermined network structure when the acquired image data is input into the predetermined network structure. The pass / fail determination program is characterized by this.

5. The determination unit determines the common part between the location detected by the anomaly determination model for the final anomaly determination location and the detection target location obtained by preprocessing. The pass / fail determination program according to Claim 4.

6. The determination unit randomly divides N normal images into N1 learning images and N2 images for determining the abnormal threshold. Next, from the N1 normal image data, the average value and covariance matrix of the feature amounts corresponding to each region are obtained. For the N2 verification images, the Mahalanobis distance is calculated based on the average value and covariance matrix calculated above, and the abnormal threshold is determined based on this. The pass / fail determination program according to claim 4 or 5.

7. In a pass / fail determination device that determines the quality of a product from a video, forming image data from a small amount of good-quality videos, and generating and learning a feature map for all of the image data; referring to a learning model constructed by learning using the image data related to the good-quality products, and determining the quality of the product in the video; comprising: The learning model is learned based on feature amounts within a compression range that can compress the image without displacement and without being affected by the background, which are extracted from a predetermined network structure when the acquired image data is input into the predetermined network structure. A pass / fail determination method characterized by this.

8. The determination unit determines the common part between the location detected by the abnormal determination model for the final abnormal determination location and the detection target location obtained by preprocessing. The pass / fail determination method according to claim 7.

9. The determination step randomly divides N normal images into N1 learning images and N2 images for determining the abnormal threshold. Next, from the N1 normal image data, the average value and covariance matrix of the feature amounts corresponding to each region are obtained. Then, for the N2 verification images, the Mahalanobis distance is calculated based on the average value and covariance matrix calculated above, and the abnormal threshold is determined based on this. The pass / fail determination device according to claim 7 or 8.

Images