Defect inspection device, defect inspection method, learning device, inference device, trained model, and trained model generation method

Abstract

To solve a problem of being unable to detect only defects on an inspection surface of a test object since a pattern printed on a transparent resin plate becomes visible by transmitted light that passes through the test object when the transparent resin plate is attached to a surface opposite to the inspection surface of a glass plate and has the printed pattern, using a defect inspection system of Patent Document 1.SOLUTION: A defect inspection device according to the present disclosure includes a light source that irradiates an inspection surface of a glass member with only light with wavelengths in a deep ultraviolet region, an imaging device that receives light specularly reflected from the inspection surface and captures an image of the inspection surface, and a processing device that determines whether or not there is a defect on the inspection surface based on the image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a defect inspection device, a defect inspection method, a learning device, an inference device, a trained model, and a trained model generation method. [Background technology]

[0002] Although it is possible to reduce defects such as bubbles and surface scratches in a glass plate, they cannot necessarily be completely removed, and therefore, measures such as removing the parts of the glass plate where bubbles, etc. are present in the inspection process are necessary. For this reason, various devices have been proposed for inspecting defects such as bubbles present in a manufactured transparent plate-like object such as a glass plate.

[0003] The defect inspection system described in Patent Document 1 is composed of a first defect inspection device, a second defect inspection device, and a processing device. The first defect inspection device includes a first light source and a first camera. The first light source irradiates light onto the plate-like body and transmits the light. The first camera collects the transmitted light that has passed through the plate-like body and captures a bright-field image. The processing device searches for a bright area in the bright-field image using a signal value that is higher than the signal value of the background component of the bright-field image as a threshold value, and when a bright area is extracted as a result of the search, the processing device uses the bright area to determine whether or not a defect area exists in the plate-like body.

[0004] The second defect detection device includes a second light source and a second camera. The second light source irradiates the surface of the plate-like body with illumination light. The second camera collects the reflected light obtained by reflection on the irradiated surface of the plate-like body and captures a bright-field reflection image. The second camera is provided on the same side as the second light source when viewed from the plate-like body. The processing device extracts a dark area in the image reflected by the plate-like body from the bright-field reflection image captured by the second camera, and determines that a defect exists in the plate-like body if the dark area faces a bright area closely. The processing device obtains position information in the thickness direction of a defect located on the plate-like body based on the positional deviation between a real image of the defect formed as an image of the defect in the bright-field reflection image and a mirror image of the defect. Here, the real image of the defect refers to an image formed by light incident on the surface of the glass plate from a direction inclined with respect to the surface of the glass plate, reflecting on the back surface of the glass plate, and then passing through the optical path of the reflected light. In addition, the mirror image of a defect refers to an image formed when light incident on the surface of the glass plate from a direction inclined to the surface of the glass plate passes through a defective area in the optical path within the glass plate and is reflected on the back surface of the glass plate. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2010-48745 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the case where the object to be inspected for defects is an object having the surface of a glass plate irradiated with light as the inspection surface, a transparent resin plate attached to the surface of the glass plate opposite the inspection surface, and a pattern printed on the transparent resin plate, when the defect inspection system described in Patent Document 1 is used, the pattern printed on the resin plate is visualized by the transmitted light that has passed through the object to be inspected or the light that is incident from the inspection surface and reflected by the surface opposite the inspection surface. For this reason, there has been a problem in that both the pattern printed on the resin plate and the defects on the inspection surface are visualized, making it impossible to judge only the defects on the inspection surface.

[0007] The present disclosure has been made to solve the above-mentioned problems, and aims to obtain an image that visualizes only defects present on an inspected surface without visualizing a pattern pattern printed on a resin plate. The present disclosure also provides a defect inspection device that determines whether or not there is a defect on an inspected surface by using an image that visualizes only defects present on the inspected surface without visualizing a pattern pattern printed on the resin plate. [Means for solving the problem]

[0008] The defect inspection device according to the present disclosure includes a light source that irradiates the inspected surface of a glass member with only light having a wavelength in the deep ultraviolet region, an imaging device that receives light specularly reflected from the inspected surface and captures an image of the inspected surface, and a processing device that determines whether or not there is a defect in the inspected surface based on the image. Effect of the Invention

[0009] The light irradiated to the surface to be inspected is light having a wavelength in the deep ultraviolet region and has a high reflectance on the surface to be inspected, so that the light is reflected by the surface to be inspected and does not penetrate to the resin plate on which the pattern is printed. Therefore, even if the object to be inspected is a glass plate having a transparent resin plate bonded to the surface opposite to the surface to be inspected and a pattern pattern is printed on the transparent resin plate, the present disclosure can obtain an image that visualizes only the defects present on the surface to be inspected without visualizing the pattern pattern printed on the resin plate. In addition, the present disclosure can obtain a defect inspection device that judges whether there is a defect on the surface to be inspected by using an image that visualizes only the defects present on the surface to be inspected. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a configuration diagram of a defect inspection device according to a first embodiment. [Diagram 2] FIG. 2 is a configuration diagram of an object to be inspected in the first embodiment. [Diagram 3] FIG. 2 is a hardware configuration diagram of a processing device according to the first embodiment. [Figure 4] 4 is a flowchart showing a process flow of the defect inspection device in the first embodiment. [Diagram 5] FIG. 11 is a configuration diagram of a defect inspection device according to a second embodiment. [Figure 6] FIG. 11 is a configuration diagram of a neural network in the second embodiment. [Figure 7] 11 is a flowchart showing a process performed by a learning device in the second embodiment. [Figure 8] 11 is a flowchart showing the processing performed by the inference device and the learned verification unit in the second embodiment. [Figure 9] FIG. 11 is a configuration diagram of a defect inspection device according to a third embodiment. [Figure 10] FIG. 11 is a configuration diagram of a neural network in the third embodiment. [Figure 11] 11 is a flowchart showing a process performed by a learning device in the third embodiment. [Figure 12]11 is a flowchart showing the processing performed by the inference device in embodiment 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described with reference to the drawings. In each drawing, the same reference numerals are used to denote common or corresponding elements, and the description is simplified or omitted. The configurations shown in the embodiments below are examples of the technical ideas of the present disclosure, and may be combined with other known technologies, or multiple technical ideas described in the present disclosure may be combined. In addition, it is also possible to omit or modify a part of the configuration without departing from the gist of the present disclosure.

[0012] Embodiment 1 In this embodiment, the defect inspection device 1 is applied to, for example, defect inspection of a member used in a refrigerator door. The member to be inspected is composed of a glass plate, which is a transparent glass member, and a transparent resin plate bonded to the glass plate. To improve the design of the refrigerator door, for example, a pattern is printed on the transparent resin plate.

[0013] 1 is a configuration diagram of a defect inspection device 1 in embodiment 1. The defect inspection device 1 includes an illumination control unit 11, a power supply device 12, an illumination device 13, a diffusion plate 14, an imaging lens 15, a camera 16, a processing device 17, and a display 18.

[0014] The illumination control unit 11 controls the power supply device 12 to control turning on and off the illumination 13 and the intensity of the light irradiated onto the object under inspection 2. The power supply device 12 is a device that supplies power to the illumination 13.

[0015] The illumination 13 is a light source that irradiates only light having a wavelength in the deep ultraviolet region onto the object 2 from an oblique direction toward the surface 211 to be inspected. In the following description, the light having a wavelength in the deep ultraviolet region is also simply referred to as light. Here, the object 2 to be inspected will be described with reference to FIG. 2. FIG. 2 is a configuration diagram of the object 2 to be inspected in the first embodiment. As shown in FIG. 2, the object 2 to be inspected is a flat plate-like panel having a two-layer structure in which a plate-like transparent glass plate 21 and a plate-like transparent resin plate 22 are bonded together. The glass plate 21 is made of, for example, a tempered glass material. The light is irradiated onto the surface of the glass plate 21 of the object 2 to be inspected. The surface of the glass plate 21 to which the light is irradiated is the surface 211 to be inspected. The surface 211 to be inspected is a flat surface. The transparent resin plate 22 is bonded onto the surface of the glass plate 21 that faces the surface 211 to be inspected, that is, the back surface of the glass plate 21. A pattern 23 is printed on the surface of the transparent resin plate 22 that is to be bonded to the glass plate 21. The wavelength of the deep ultraviolet region refers to a wavelength of 100 nm to 380 nm.

[0016] Here, the wavelength of the light irradiated to the surface 211 to be inspected differs depending on the spectral characteristics of the material constituting the glass plate 21. Spectroscopic characteristics are characteristics unique to each molecule constituting a substance, and when the spectral characteristics differ, the wavelength of light reflected or absorbed differs. When the glass plate 21 is composed of a general transparent glass plate, the light irradiated to the surface 211 to be inspected is light with a wavelength in a wavelength band of 10 nm around 370 nm. Note that, depending on the material constituting the glass, the wavelength may be 370 nm or less. Light with a wavelength in the deep ultraviolet region is reflected by the surface 211 to be inspected without passing through the glass plate 21. The reflectance does not need to be 100%, and light may be incident on the glass plate 21 as long as it does not reach the pattern, for example.

[0017] The diffusion plate 14 is a plate-like object attached to the illumination 13 so that the light irradiated from the illumination 13 onto the surface 211 to be inspected is uniform and even. Next, an imaging device for imaging the surface 211 to be inspected will be described. The imaging device is composed of an imaging lens 15 and a camera 16. The imaging lens 15 is a lens that can transmit light with wavelengths in the deep ultraviolet region. The imaging lens 15 forms an image from the light irradiated by the illumination 13 and specularly reflected by the surface 211 to be inspected. Here, specular reflection means that the light irradiated from the illumination 13 is reflected at the same angle as the angle at which it was irradiated, in the opposite direction. In other words, in specular reflection, the angle of incidence and the angle of reflection of the light irradiated onto the surface 211 to be inspected are equal. The camera 16 is an area sensor type camera sensitive to light with wavelengths in the deep ultraviolet region, and images the surface 211 to be inspected of the object 2 to be inspected. The camera 16 is disposed at a position where it can receive the light specularly reflected by the surface 211 to be inspected. That is, the camera 16 is disposed on the same side of the surface 211 to be inspected as the illumination 13. In the image of the surface 211 to be inspected captured by the camera 16, the defective parts appear dark, and the non-defective parts appear bright. The defective parts of the surface 211 to be inspected have an uneven shape, which causes the reflection of the irradiated light to change slightly and not become a regular reflection. Therefore, the defective parts appear dark in the image. Furthermore, since the irradiated light is reflected by the surface 211 to be inspected, it does not reach the pattern design 23. Therefore, the image of the surface 211 to be inspected captured by the camera 16 is an image that shows only the defects on the surface 211 to be inspected.

[0018] The processing device 17 judges whether the surface 211 to be inspected has a defect based on the image of the surface 211 to be inspected. The processing device 17 has a camera control function and an image processing function. The camera control function is, for example, a function of controlling the timing and the photographing range of the surface 211 to be inspected by the camera 16. The image processing function is, for example, a function of performing binarization processing or labeling processing on the image of the surface 211 to be inspected photographed by the camera 16. Here, the binarization processing is, for example, a process of assigning 0 to each pixel of the image of the surface 211 to be inspected if the pixel is greater than a predetermined brightness or luminance threshold value, and assigning 1 if the pixel is smaller than the predetermined brightness or luminance threshold value. In this case, the pixel of the bright part is assigned 0, and the pixel of the dark part is assigned 1. The labeling processing is a process of classifying a plurality of regions into groups by adding the same label to pixels that are assigned the same value in the binarization processing and are connected.

[0019] The processing device 17 measures the area of ​​each region formed by a plurality of pixels to which the same label is added. The processing device 17 then judges whether the measured area is equal to or larger than a predetermined value. If the area of ​​the dark region is equal to or larger than the predetermined value, the processing device 17 judges that the surface 211 to be inspected has a defect. If the area of ​​the dark region is less than the predetermined value, the processing device 17 judges that the surface 211 to be inspected has no defect. Based on this judgment, the processing device 17 judges whether the surface 211 to be inspected has a defect. The judgment method is not limited to this. For example, the processing device 17 may make the judgment using the sum of the areas measured for each region, or may make the judgment using the number of regions whose areas are equal to or larger than the predetermined value. The threshold value and the predetermined value may be set, for example, based on whether the size is such that a person can recognize the defect as a defect when visually inspecting the object to be inspected.

[0020] The display 18 displays on its screen the image of the inspected surface 211 captured by the camera 16, the result of the processing device 17 determining whether there is a defect in the inspected surface 211, and the result of identifying the position of the defect. The position of the defect is identified, for example, by superimposing a rectangular image surrounding the area determined to be defective on the image of the inspected surface 211 and displaying it on the screen.

[0021] Next, a description will be given of the hardware configuration of the processing device 17. Fig. 3 is a diagram showing the hardware configuration of the processing device 17. The processing device 17 is composed of an arithmetic unit 5, a storage device 6, an input device 7, an auxiliary storage device 8, an output device 9, and a signal line 10.

[0022] The processing performed by the processing device 17 can be realized by, for example, the arithmetic device 5. The arithmetic device 5 realizes the function of the processing device 17 by reading out a necessary program from the auxiliary storage device 8 and executing the processing. The arithmetic device 5 is, for example, a processor, and the processor is an IC (Integrated Circuit) that performs arithmetic processing. Specific examples of the processor are a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). In addition, it may be a sonal computer, a microcomputer board, or an FPGA (Field Programmable Gate Array) board.

[0023] The storage device 6 is a main storage device of the processing device 17. The main storage device temporarily stores calculations of the processing performed by the arithmetic device 5. For example, the storage device 6 is a RAM (Random Access Memory).

[0024] The input device 7 inputs an image to the arithmetic unit 5. The input device 7 is an input interface of the processing unit 17.

[0025] The auxiliary storage device 8 is an auxiliary storage device of the processing device 17. For example, the auxiliary storage device 8 stores initial value data, programs, error logs, output data, and the like required for the operation of the processing device 17. For example, the auxiliary storage device 8 is a ROM (Read Only Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive).

[0026] The output device 9 outputs an image of the inspected surface 211, the result of determining the presence or absence of a defect, and the result of identifying the position of the defect, to the display 18. The output device 9 is an output interface of the processing device 17.

[0027] A signal line 10 is a transmission path for transmitting and receiving data between the components shown in FIG.

[0028] Next, the process performed by the defect inspection device 1 will be described. FIG. 4 is a flowchart showing the flow of the process of the defect inspection device 1. The defect inspection device 1 starts operation when the power is turned on. In step S11, the illumination 13 irradiates the surface 211 to be inspected with light having a wavelength in the deep ultraviolet region. Next, in step S12, the camera 16 receives the light specularly reflected by the surface 211 to be inspected and captures the surface 211 to be inspected. In step S13, the processing device 17 performs a luminance binarization process on the image captured by the camera 16. In step S14, the processing device 17 performs a labeling process on the luminance binarized image. In step S15, the processing device 17 judges whether or not the area of ​​the labeled dark portion of the image is equal to or greater than a predetermined value. If the area of ​​the dark portion is not equal to or greater than the predetermined value, the process proceeds to step S16, and the processing device 17 judges that the surface 211 to be inspected does not have a defect. If the area of ​​the dark region is equal to or larger than the predetermined value, the process proceeds to step S17, and the processing device 17 determines that there is a defect in the inspected surface 211. When the process of step S16 or step S17 ends, the defect inspection device 1 ends the process.

[0029] The processing device 17 may be configured as a terminal device, may be configured as a server, or may be configured as a terminal device and a server. When the processing device 17 is configured as two or more devices, each of the two or more devices has, for example, a hardware configuration shown in FIG. 3, and performs communication between the devices by wire or wirelessly. When the processing device 17 is configured as a server, the lighting control unit 11, the power supply device 12, the lighting 13, the diffusion plate 14, the imaging lens 15, the camera 16, and the display 18 are installed on-site, and an image of the inspected surface 211 captured by the camera 16 is transmitted to the processing device 17 configured as a server by a communication device (not shown). The processing device 17 uses the received image to determine whether there is a defect and transmits the determination result to the communication device. The determination result received by the communication device is displayed on the display 18.

[0030] According to the configuration of the present embodiment, the defect inspection device 1 includes a light source that irradiates the inspected surface 211 with light having a wavelength in the deep ultraviolet region, an image capture device that receives light specularly reflected by the inspected surface 211 and captures an image of the inspected surface 211, and a processing device 17 that judges whether or not there is a defect in the inspected surface 211 based on the image. Therefore, even if the inspected object 2 is a glass plate 21 having a transparent resin plate 22 bonded to a surface facing the inspected surface 211 and a pattern pattern 23 printed on the transparent resin plate 22, an image can be obtained that visualizes only the defects present on the inspected surface 211 without visualizing the pattern pattern 23 printed on the resin plate 22. In addition, the present disclosure can obtain a defect inspection device 1 that judges whether or not there is a defect in the inspected surface 211 by using an image that visualizes only the defects present on the inspected surface 211.

[0031] Embodiment 2 5 is a configuration diagram of a defect inspection device 3 in embodiment 2. The defect inspection device 1 in embodiment 1 and the defect inspection device 3 in embodiment 2 differ in the configuration and operation of a processing device 37. Therefore, in this embodiment, only the points different from embodiment 1 will be described, and a description of the same configuration and operation as embodiment 1 will be omitted.

[0032] 5, the processing device 37 includes a first storage unit 371, a learning device 372, a learned model storage unit 373, an inference device 374, and an inspection unit 375. Note that the illumination control unit 31, the power supply unit 32, the illumination 33, the diffusion plate 34, the imaging lens 35, the camera 36, ​​and the display 38 in the second embodiment are the same as the illumination control unit 11, the power supply unit 12, the illumination 13, the diffusion plate 14, the imaging lens 15, the camera 16, and the display 18 in the first embodiment, respectively.

[0033] The first storage unit 371 stores a non-defective image of the inspected surface 211. Here, a non-defective image is an image of the inspected surface 211 captured by the imaging device, in which the inspected surface 211 has no defects or includes defects smaller than a predetermined size. Here, an image including defects smaller than a predetermined size may be, for example, an image determined under the same conditions as those under which the processing device 17 determined in the first embodiment that the inspected surface 211 has no defects, or an image determined by a user that the inspected surface 211 has no defects.

[0034] The learning device 372 includes a first data acquisition unit 3721 and a model generation unit 3722. The first data acquisition unit 3721 acquires a non-defective image of the inspected surface 211 from the first storage unit 371 as learning data. The model generation unit 3722 uses the learning data to extract feature amounts from the non-defective image, and generates a trained model that reconstructs and outputs an image similar to the non-defective image from the feature amounts. Here, a method in which the model generation unit 3722 generates a trained model will be described.

[0035] The model generating unit 3722 can use known algorithms such as supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. As an example, a case where a neural network is applied when performing unsupervised learning will be described. Unsupervised learning refers to a method of learning features in learning data that does not include results by providing the learning device with the learning data. A learning algorithm used by the model generating unit 3722 is deep learning, such as convolution processing (Convolution Neural Network), which learns to be able to extract the feature amount itself. The model generating unit 3722 can execute a process of generating a trained model according to the learning algorithm. The model generating unit 3722 learns to reconstruct and output an image similar to a good product image by so-called unsupervised learning using a neural network, for example.

[0036] FIG. 6 is a diagram showing the configuration of a neural network. A neural network is composed of an input layer consisting of a plurality of neurons, a middle layer consisting of a plurality of neurons, and an output layer consisting of a plurality of neurons. The middle layer may be one layer or two or more layers. For example, in the case of a three-layered neural network as shown in FIG. 6, when a plurality of inputs are input to X1 to X3, which are the input layers, the values ​​are multiplied by weighting coefficients W1, w11 to w16, and the results are input to Y1 and Y2, which are the middle layers. The results input to the middle layers are further multiplied by weighting coefficients W2, w21 to w26, and the results are output from Z1 to Z3, which are the output layers. The output results vary depending on the values ​​of weighting coefficients W1 and W2.

[0037] The model generating unit 3722 uses a neural network to learn the features of the good-quality image from the good-quality image acquired by the first data acquiring unit 3721. That is, the model generating unit 3722 learns by adjusting the weighting coefficients W1 and W2 so that the reconstructed image output from the output layer after inputting the good-quality image to the input layer approaches the good-quality image input to the input layer. The model generating unit 3722 can use an autoencoder as one of the neural networks. The autoencoder includes an encoder that extracts features from the good-quality image, which is input data, and a decoder that reconstructs an image similar to the good-quality image from the features. The encoder is composed of an input layer and an intermediate layer of the neural network, and the decoder is composed of an intermediate layer and an output layer of the neural network. The model generating unit 3722 may perform machine learning according to other known methods, such as genetic programming, functional logic programming, or a support vector machine.

[0038] The model generation unit 3722 generates and outputs a trained model by executing the above-mentioned learning.

[0039] The trained model storage unit 373 stores the trained model generated by the model generation unit 3722.

[0040] The inference device 374 includes a second data acquisition unit 3741 and an inference unit 3742. The second data acquisition unit 3741 acquires an image of the inspected surface 211 captured by the camera 36. The learned model stored in the learned model storage unit 373 and the image of the inspected surface 211 acquired by the second data acquisition unit 3741 are input to the inference unit 3742. By inputting the image of the inspected surface 211 to the learned model, the inference unit 3742 can reconstruct an image similar to a non-defective image from the image of the inspected surface 211 and output it.

[0041] The inspection unit 375 compares the image similar to the non-defective image output by the inference unit 3742 with the image of the inspected surface 211 input by the inference device 374 to the trained model, and if they are similar, determines that there is no defect in the inspected surface 211. If they are not similar, the inspection unit 375 determines that there is a defect in the inspected surface 211. Here, a method of comparing the image similar to the non-defective image output by the inference unit 3742 with the image of the inspected surface 211 input by the inference device 374 to the trained model will be described. The inspection unit 375 can use a known algorithm such as Zscore or pattern matching as a method of comparing the images.

[0042] Next, the processing performed by the learning device 372 in the second embodiment will be described. FIG. 7 is a flowchart showing the processing performed by the learning device 372. The learning device 372 starts operating when the processing device 37 is powered on. In step S21, the first data acquisition unit 3721 acquires a good product image from the first storage unit 371. In step S22, the model generation unit 3722 performs a learning process using the good product image acquired from the first storage unit 371. In step S23, the trained model storage unit 373 stores the trained model generated by the model generation unit 3722. Note that the learning device 372 may start the processing at the timing when a predetermined operation by a user is detected.

[0043] Next, the processing performed by the inference device 374 and the inspection unit 375 in the second embodiment will be described. FIG. 8 is a flowchart showing the processing performed by the inference device 374 and the inspection unit 375. The inference device 374 and the inspection unit 375 start operating when the processing device 37 is powered on. In step S24, the second data acquisition unit 3741 acquires an image of the inspected surface 211 from the camera 36. In step S25, the inference unit 3742 inputs the image of the inspected surface 211 to the trained model. In step S26, the trained model extracts features from the image of the inspected surface 211, reconstructs an image similar to the non-defective image from the extracted features, and outputs the image. In step S27, the inspection unit 375 compares the image similar to the non-defective image output by the inference unit 3742 with the image of the inspected surface 211 input by the inference device 374 to the trained model, and judges whether the inspected surface 211 has a defect. The process ends when the inspection unit 375 determines whether or not there is a defect on the inspected surface 211. Note that the inference device 374 and the inspection unit 375 may start the process at the timing when it is detected that the camera 36 has captured an image of the inspected surface 211.

[0044] The learning device 372 can also learn by adding or removing good-quality images during learning. For example, the learning device 372 can learn by adding good-quality images of a type of the object 2 to the learning data that is different from the type of the object 2 for which good-quality images have been acquired in advance, or can learn by removing good-quality images of some types of the object 2 from the good-quality images of a plurality of objects 2 acquired in advance. The learning device 372 can also learn by using good-quality images of another object 2 that is similar in shape to the object 2 for which the presence or absence of a defect is actually determined. Furthermore, the learning device 372 can execute learning by using good-quality images of a transparent resin panel instead of the glass plate 21.

[0045] In addition, a good-quality image other than the good-quality image used during learning may be applied to the learned model to update the learned model through re-learning. The good-quality image may be an image of the inspected surface 211 newly captured using the defect inspection device 3 that is determined by the user to be a good-quality image.

[0046] Although the first storage unit 371 is described as being provided in the processing device 37, it may be provided outside the processing device 37, for example, in a server. In this case, the processing device 37 acquires learning data from the externally provided first storage unit 371 in a wired or wireless manner.

[0047] Also, although the trained model storage unit 373 is described as being provided in the processing device 37, it may be provided outside the processing device 37. In this case, the processing device 37 acquires the trained model from the trained model storage unit 373 provided outside the processing device 37 via a wired or wireless connection.

[0048] According to the configuration of the second embodiment, the processing device 37 includes a first storage unit 371, a learning device 372, a trained model storage unit 373, an inference device 374, and an inspection unit 375. The learning device 372 includes a first data acquisition unit 3721 that acquires a non-defective image from the first storage unit 371 as learning data, and a model generation unit 3722 that uses the learning data to extract features from the non-defective image and generate a trained model that reconstructs and outputs an image similar to the non-defective image from the extracted features. The inference device 374 includes a second data acquisition unit 3741 that acquires an image of the inspected surface 211 captured from the imaging device, and an inference unit 3742 that uses the trained model generated by the model generation unit 3722 to reconstruct and output an image similar to the non-defective image from the image of the inspected surface 211 acquired by the second data acquisition unit 3741. The inspection unit 375 compares the image of the inspected surface 211 acquired by the second data acquisition unit 3741 with an image similar to the non-defective product image reconstructed by the inference unit 3742, and if the two images are not similar, it determines that the inspected surface 211 has a defect.

[0049] Therefore, the processing device 37 inputs an image of the inspected surface 211 captured by the imaging device into a trained model that reconstructs and outputs an image similar to the non-defective image when a non-defective image is input, and compares the output image with the image of the inspected surface 211 captured by the imaging device, thereby determining whether the inspected surface 211 has a defect. In addition, the learning device 372 can also cause the model generation unit 3722 to retrain the trained model, thereby improving the accuracy of the trained model. In addition, the learning device 372 can match one trained model to multiple types of inspected objects 2 or change the trained model in response to a change in the type of inspected object 2 by adding or removing an image of the inspected surface 211 input to the model generation unit 3722.

[0050] Embodiment 3 9 is a configuration diagram of a defect inspection device 4 in the embodiment 3. The configuration and operation of a processing device 47 are different from those in the embodiment 1 or 2. In the present embodiment, the description of the same configuration as in the embodiment 1 or 2 will be omitted as appropriate.

[0051] The processing device 47 in the third embodiment is composed of a second storage unit 471, a learning device 472, a learned model storage unit 473, and an inference device 474. Note that the lighting control unit 41, the power supply unit 42, the lighting 43, the diffusion plate 44, the imaging lens 45, the camera 46, and the display 48 in the third embodiment are the same as the lighting control unit 11, the power supply unit 12, the lighting 13, the diffusion plate 14, the imaging lens 15, the camera 16, and the display 18 in the first embodiment, respectively.

[0052] The second storage unit 471 stores learning data. The learning data includes an image of the inspected surface 211 and a label indicating whether or not a defect exists on the inspected surface 211. Here, the judgment as to whether or not the inspected surface 211 has a defect is made, for example, under the same conditions as those under which the processing device 17 judged whether or not the inspected surface 211 has a defect in the first embodiment. For example, when the processing device 17 judges that the inspected surface 211 has no defect, the user attaches a label indicating that there is no defect to the image of the inspected surface 211 that has been judged to have no defect.

[0053] The learning device 472 is composed of a first data acquisition unit 4721 and a model generation unit 4722. The first data acquisition unit 4721 acquires learning data from the second storage unit 471. The model generation unit 4722 generates a trained model using the learning data acquired by the first data acquisition unit 4721 as training data. Here, supervised learning using training data will be described. Supervised learning refers to a technique in which learning is performed while the learning device is given learning data including labels.

[0054] Fig. 10 is a configuration diagram of a neural network. The configuration of the neural network is the same as that of Fig. 6 in the embodiment 2. In the embodiment 3, the model generation unit 4722 generates a trained model by inputting an image of the inspected surface 211 to the input layer and adjusting the weighting coefficients W1 and W2 so that the result output from the output layer approaches the label.

[0055] The trained model storage unit 473 stores the trained model generated by the model generation unit 4722.

[0056] The inference device 474 includes a second data acquisition unit 4741 and an inference unit 3742. The second data acquisition unit 4741 acquires an image of the inspected surface 211 captured by the camera 46. The inference unit 4742 receives the learned model stored in the learned model storage unit 473 and the image of the inspected surface 211 acquired by the second data acquisition unit 4741. The inference unit 4742 can output a result of inferring whether or not the inspected surface 211 has a defect from the image of the inspected surface 211 by inputting the image of the inspected surface 211 to the learned model.

[0057] Next, the processing performed by the learning device 472 in the third embodiment will be described. FIG. 11 is a flowchart showing the processing performed by the learning device 472. The learning device 472 starts operating when the processing device 47 is powered on. In step S31, the first data acquisition unit 4721 acquires learning data from the second storage unit 471. In step S32, the model generation unit 4722 performs a learning process using the learning data acquired from the second storage unit 471. In step S33, the trained model storage unit 473 stores the trained model generated by the model generation unit 4722. Note that the learning device 472 may start the processing at the timing when a predetermined operation by a user is detected.

[0058] Next, the process performed by the inference device 474 in the third embodiment will be described. FIG. 12 is a flowchart showing the process performed by the inference device 474. The inference device 474 starts operating when the processing device 47 is powered on. In step S34, the second data acquisition unit 4741 acquires an image of the inspected surface 211 captured by the camera 46. In step S35, the inference unit 4742 inputs the image of the inspected surface 211 captured to the learned model. In step S36, the learned model infers whether or not the inspected surface 211 has a defect from the image of the inspected surface 211 and outputs the inference result. When the inference unit 4742 outputs the result of inferring whether or not the inspected surface 211 has a defect from the image of the inspected surface 211, the process ends. Note that the inference device 474 may start the process at the timing when it is detected that the camera 46 has captured an image of the inspected surface 211.

[0059] In addition, the trained model may be updated by re-training by applying training data other than the training data used during training to the trained model. Note that the training data other than the training data used during training may be training data determined by the user.

[0060] Although the second storage unit 471 is described as being provided in the processing device 47, it may be provided outside the processing device 47. In this case, the processing device 47 acquires learning data from the externally provided second storage unit 471 in a wired or wireless manner.

[0061] Also, although the case where the trained model storage unit 473 is provided in the processing device 47 has been described, it may be provided outside the processing device 47. In this case, the processing device 47 acquires the trained model from the trained model storage unit 473 provided outside via a wired or wireless connection.

[0062] According to the configuration of the third embodiment, the processing device 47 includes a second storage unit 471, a learning device 472, a learned model storage unit 473, and an inference device 474. The learning device 472 includes a first data acquisition unit 4721 that acquires, as learning data, an image of the inspected surface 211 and a label indicating whether or not a defect exists on the inspected surface 211 from the second storage unit 471, and a model generation unit 4722 that uses the learning data to generate a learned model that infers whether or not the inspected surface 211 has a defect from the image of the inspected surface 211 and outputs the inference. The inference device 474 also includes a second data acquisition unit 4741 that acquires an image of the inspected surface 211 from an imaging device, and an inference unit 4742 that uses the learned model generated by the model generation unit 4722 to infer whether or not the inspected surface 221 has a defect from the image of the inspected surface 211 acquired by the second data acquisition unit 4741, and outputs the inference result.

[0063] Therefore, when an image of the surface 211 to be inspected is input, the processing device 47 inputs the image of the surface 211 to be inspected captured by the imaging device into a trained model that infers and outputs whether or not the surface 211 to be inspected has a defect, and can obtain the result of inferring whether or not the surface 211 to be inspected has a defect as the inspection result. [Explanation of symbols]

[0064] 1, 3, 4 defect inspection device, 2 object to be inspected, 5 calculation device, 6 storage device, 7 input device, 8 auxiliary storage device, 9 output device, 10 signal line, 11, 31, 41 lighting control unit, 12, 32, 42 power supply device, 13, 33, 43 lighting, 14, 34, 44 diffusion plate, 15, 35, 45 imaging lens, 16, 36, 46 camera, 17, 37, 47 processing device, 18, 38, 48 display, 21 glass plate, 22 resin plate, 23 pattern pattern, 211 surface to be inspected, 371 first storage unit, 372, 472 learning device, 373, 473 learned model storage unit, 374, 474 inference device, 375 inspection unit, 3721, 4721 first data acquisition unit, 3722, 4722 Model generation unit, 3741, 4741 Second data acquisition unit, 3742, 4742 Inference unit

Claims

1. a light source that irradiates only light having a wavelength in the deep ultraviolet region onto a surface to be inspected of a glass member; an image capturing device that receives the light specularly reflected from the inspection surface and captures an image of the inspection surface; a processing device that determines whether the inspected surface has a defect based on the image; A defect inspection device comprising:

2. When an area of ​​a region formed by a plurality of pixels having a luminance or brightness value smaller than a predetermined threshold value in the image of the surface to be inspected captured by the imaging device is equal to or larger than a predetermined value, the processing device determines that the surface to be inspected has a defect. The defect inspection device according to claim 1 .

3. A light source that irradiates only light having a wavelength in the deep ultraviolet region irradiates the light onto a surface to be inspected of a glass member; an imaging device receiving the light specularly reflected from the surface to be inspected and capturing an image of the surface to be inspected; A processing device determines whether the inspected surface has a defect based on the image; A defect inspection method comprising:

4. a data acquisition unit that acquires, as learning data, good-quality images of the inspection surface, which are images of the inspection surface that have no defects or that have defects smaller than a predetermined size, from images of the inspection surface that are captured by an imaging device receiving the light that is specularly reflected from the inspection surface and is irradiated by a light source that irradiates only light in the deep ultraviolet region; A model generation unit that uses the learning data to extract features from the non-defective product image and generate a trained model that reconstructs and outputs an image similar to the non-defective product image from the features; A learning device comprising:

5. a data acquisition unit that acquires an image of the inspection surface of the glass member by irradiating the inspection surface with a light source that irradiates only light with a wavelength in the deep ultraviolet region, and receiving the light specularly reflected from the inspection surface with an imaging device; an inference unit that extracts features from a good-quality image, which is an image of the inspected surface that has no defects or that includes defects smaller than a predetermined size, captured by the imaging device, and reconstructs and outputs an image similar to the good-quality image from the image of the inspected surface acquired by the data acquisition unit using a trained model that has been trained to reconstruct and output an image similar to the good-quality image from the features; an inspection unit that compares the image of the inspected surface acquired by the data acquisition unit with an image similar to the non-defective image reconstructed by the inference unit, and determines that the inspected surface has a defect if the two images are not similar; An inference device comprising:

6. The processing device includes: an inference unit that extracts features from a good-quality image, which is an image of the inspection surface that is free of defects or includes defects smaller than a predetermined size, captured by the imaging device, and reconstructs and outputs an image similar to the good-quality image from the image of the inspection surface captured by the imaging device using a trained model that has been trained to reconstruct and output an image similar to the good-quality image from the features; an inspection unit that compares the image of the inspected surface photographed by the photographing device with an image similar to the non-defective product image reconstructed by the inference unit, and determines that the inspected surface has a defect if the two images are not similar; The defect inspection device according to claim 1 .

7. The processing device includes: a data acquisition unit that acquires the non-defective product images as learning data; A model generation unit that generates the trained model by extracting features from the good product image using the learning data and reconstructing and outputting the image similar to the good product image from the features; The defect inspection device according to claim 6 .

8. A trained model for causing a computer to function to reconstruct and output an image similar to a non-defective image, which is an image of the inspected surface that has no defects or that contains defects smaller than a predetermined size, among images of the inspected surface of a glass member that is irradiated with light from a light source that irradiates only light with a wavelength in the deep ultraviolet region and that is captured by an imaging device receiving the light specularly reflected from the inspected surface, For the image of the surface to be inspected input to the input layer of the trained model, A trained model for causing a computer to function as follows: performing calculations based on weighting coefficients of the trained model, extracting features from the image of the inspected surface, reconstructing an image similar to the non-defective product image from the features, and outputting the image from an output layer of the trained model.

9. a step of acquiring, as learning data, good-quality images of the inspection surface, which are images of the inspection surface that have no defects or that contain defects smaller than a predetermined size, from images of the inspection surface that are captured by an imaging device receiving the light that is specularly reflected from the inspection surface and irradiated by a light source that irradiates only light in the deep ultraviolet region; A step of extracting features from the good product image using the learning data, and generating a trained model that is reconstructed from the features and outputs an image similar to the good product image; A method for generating a trained model consisting of:

10. The model generation unit updates the trained model using the image of the inspection surface that is determined by a user to correspond to the non-defective product image. The learning device according to claim 4.

11. a data acquisition unit that acquires learning data including an image of the inspection surface of the glass member that is irradiated with light from a light source that irradiates only light with a wavelength in the deep ultraviolet region and that is photographed by an imaging device that receives the light that is specularly reflected from the inspection surface, and a label that indicates whether or not the inspection surface has a defect; a model generation unit that generates a trained model using the learning data to infer whether or not the inspection surface has a defect from the image of the inspection surface and outputs the trained model; A learning device equipped with

12. a data acquisition unit that acquires an image of the inspection surface of the glass member by irradiating the inspection surface with a light source that irradiates only light with a wavelength in the deep ultraviolet region, and receiving the light specularly reflected from the inspection surface with an imaging device; an inference unit that infers whether or not the inspected surface has a defect from the image of the inspected surface acquired by the data acquisition unit, using a trained model for inferring whether or not the inspected surface has a defect from the image of the inspected surface, and outputs the inference; An inference device comprising:

13. The processing device includes: an inference unit that infers whether or not a defect exists on the inspection surface from the image obtained by photographing the inspection surface, using a trained model for inferring whether or not a defect exists on the inspection surface from the image of the inspection surface, and outputs the inference; The defect inspection device according to claim 1 .

14. The processing device includes: a data acquisition unit that acquires learning data including the image of the inspection surface captured by the imaging device and a label indicating whether or not the inspection surface has a defect; a model generation unit that generates a trained model that uses the learning data to infer whether or not the inspection surface has a defect from the image of the inspection surface and outputs the trained model; The defect inspection device according to claim 13 ,

15. A trained model for causing a computer to function to output whether or not there is a defect on an inspected surface of a glass member, from an image of the inspected surface captured by an imaging device receiving the light specularly reflected by the inspected surface, wherein the trained model is For the image of the surface to be inspected input to the input layer of the trained model, A trained model for causing a computer to perform calculations based on the weighting coefficients of the trained model and output from the output layer of the trained model whether or not the inspected surface has a defect.

16. a step of acquiring learning data including an image of the inspection surface of the glass member, the image being captured by an imaging device receiving the light specularly reflected by the inspection surface from a light source that emits only light in the deep ultraviolet region, and a label indicating whether or not the inspection surface has a defect; generating a trained model that uses the learning data to infer whether or not the inspected surface has a defect from the image of the inspected surface and outputs the inference; A method for generating a trained model comprising the steps of:

17. The model generation unit updates the trained model using a result of a user checking whether or not there is a defect in the image of the inspected surface acquired by the data acquisition unit and the image of the inspected surface as training data. The learning device according to claim 11.

Images