Learning device, estimation device, defect inspection device, and defect inspection method

Abstract

To provide a learning apparatus, an estimation apparatus, a dimension measurement apparatus, and a dimension measurement method capable of measuring a dimension of an object with high accuracy.SOLUTION: An inspection method includes: a step of inputting defect type information indicating a type of a defect of an object, lighting condition information on a lighting condition for a lighting device to emit light in accordance with the type of the defect of the object, and object information for identifying the object; a step of constructing a model which outputs lighting condition information on an optimal condition for the object when the defect type information and the object information are input, the model being obtained by machine learning using the defect type information, the lighting condition information, and the object information, as training data; a step of inputting the defect type information and the object information to the model, and estimating the optimal lighting condition for the object on the basis of the lighting condition information on the optimal condition for the object, output from the model; a step of controlling the lighting device on the basis of the estimated lighting condition; and a step of inspecting the object.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a learning device, an estimation device, a defect inspection device, and a defect inspection method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a packaging material called an embossed carrier tape for storing minute electronic components has been known (see Patent Document 1 below). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-136545 A Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned embossed carrier tape needs to be inspected for defects such as the adhesion of foreign matter on the surface, and defects such as burrs in the recesses and edges that house electronic components. In order to inspect the appearance defects of the embossed carrier tape with high accuracy, it is necessary to use clear images, but since there are multiple types of defects and embossed carrier tapes come in various shapes, thicknesses, and materials, it is necessary to carefully set the lighting conditions of the lighting device that is optimal for the defect inspection of each embossed carrier tape.

[0005] An object of the present invention is to provide a learning device, an estimation device, a defect inspection device, and a defect inspection method that are capable of inspecting an object having defects with high accuracy. [Means for solving the problem]

[0006] A learning device according to one embodiment of the present invention includes an input unit for inputting defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device suitable for a type of defect to be detected in the object will irradiate light, and object information for identifying the object, and a model construction unit for constructing a model that has been machine-learned using the defect type information, lighting condition information, and object information input by the input unit as training data, and that outputs the lighting condition information regarding conditions optimal for the object when the defect type information and the object information are input.

[0007] An estimation device according to one aspect of the present invention includes an input unit that inputs defect type information indicating a type of defect of an object and object information for identifying the object, and an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using the defect type information, the object information, and lighting condition information related to lighting conditions under which a lighting device corresponding to the type of defect of the object irradiates light, as training data, and estimates optimal lighting conditions for the object based on the lighting condition information related to the optimal conditions for the object output from the model.

[0008] A defect inspection device according to one aspect of the present invention includes an illumination device that irradiates light onto an object, an inspection unit that inspects the object, an input unit that inputs defect type information indicating a type of defect of the object and object information for identifying the object, an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using the defect type information, the object information, and lighting condition information related to the lighting conditions under which a lighting device irradiates light corresponding to the type of defect of the object as training data, and estimates optimal lighting conditions for the object based on the lighting condition information related to the optimal conditions for the object output from the model, and a control unit that controls the illumination device based on the lighting conditions estimated by the estimation unit.

[0009] A defect inspection method according to one aspect of the present invention includes: Defective inspection equipment,inputting defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device corresponding to the type of defect of the object irradiates light, and object information for identifying the object; The defect inspection device, constructing a model that is machine-learned using the input defect type information, the illumination condition information, and the object information as training data, and that outputs the illumination condition information related to the optimum condition for the object when the defect type information and the object information are input; The defect inspection device, a step of inputting the defect type information and the object information for identifying the object into the model when the defect type information and the object information for identifying the object are input, and estimating optimal illumination conditions for the object based on the illumination condition information relating to optimal conditions for the object output from the model; The defect inspection device, controlling the lighting device based on the estimated lighting conditions; The defect inspection device, and inspecting the object. Effect of the Invention

[0010] According to the present invention, it is possible to provide a learning device, an estimation device, a defect inspection device, and a defect inspection method that are capable of inspecting an object having a defect with high accuracy. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a defect inspection system according to an embodiment. [Diagram 2] 1 is a block diagram illustrating an example of a functional configuration of a defect inspection system according to an embodiment. [Diagram 3] FIG. 2 is a block diagram showing an example of a learning unit in the embodiment. [Figure 4] FIG. 1 is a diagram illustrating the relationship between input variables and output variables in a convolutional neural network. [Diagram 5] FIG. 2 is a block diagram showing an example of an estimation unit 330 in the embodiment. [Figure 6] FIG. 2 is a top view showing an example of an arrangement of lighting devices in the embodiment. [Figure 7] FIG. 11 is a top view showing another example of the arrangement of the lighting device in the embodiment. [Figure 8] FIG. 2 is a side view of the lighting device according to the embodiment. [Figure 9] FIG. 11 is another side view of the lighting device according to the embodiment. [Figure 10] 1A and 1B are top views showing an example of an embossed carrier tape 10 according to an embodiment, in which FIG. 1A is an example of an embossed carrier tape 10 without any defects, and FIG. 1B is an example of an embossed carrier tape 10 with a defect. [Figure 11] 11 is a diagram showing a relationship between an illumination condition, a defect type, and correlation data in the embodiment. FIG. [Figure 12] FIG. 2 is a diagram showing another configuration of the defect inspection system 1 according to the embodiment. [Figure 13] FIG. 11 is a diagram for explaining a process of performing an estimation process using a convolutional neural network in the embodiment. [Figure 14] FIG. 1 is a diagram for explaining a learning process using a convolutional neural network in an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of a learning device, an estimation device, a defect inspection device, and a defect inspection method will be described with reference to the drawings.

[0013] [Defect inspection overview] The defect inspection system 1 of the embodiment inspects an object for defects. In the embodiment, the object is, for example, an embossed carrier tape. The embossed carrier tape is made of plastic and has storage pockets (recesses) for storing chip-type electronic components formed continuously. The defect inspection system 1 inspects the storage pockets for defects by acquiring information based on the shape of the embossed carrier tape while irradiating the embossed carrier tape with light. Note that the present embodiment measures the dimensions of shapes such as recesses, but is not limited thereto and can also be applied to applications for measuring shapes other than recesses.

[0014] [Configuration of Defect Inspection System 1] FIG. 1 is a diagram showing an example of a schematic configuration of a defect inspection system 1 in an embodiment. The defect inspection system 1 includes, for example, an illumination device 100 and an imaging device 200 arranged above and below a production line L. The production line L is a moving means for delivering an embossed carrier tape 10 on which recesses 10a are formed. When measuring the dimensions of the embossed carrier tape 10, the defect inspection system 1 images the embossed carrier tape 10 with the imaging device 200 while irradiating the embossed carrier tape 10 with light from the illumination device 100. The defect inspection system 1 measures the dimensions of the recesses 10a based on the captured image or the like.

[0015] 2 is a block diagram showing an example of a functional configuration of the defect inspection system 1 according to the embodiment. The defect inspection system 1 includes, for example, an illumination device 100, an imaging device 200, a defect inspection device 300, an information input unit 400, a teacher data storage unit 500, and a notification unit 600.

[0016] The defect inspection device 300 includes, for example, a defect inspection unit 310, a learning unit 320, an estimation unit 330, and an illumination control unit 340. The functional units, such as the defect inspection unit 310, the learning unit 320, the estimation unit 330, and the illumination control unit 340, are realized by a processor, such as a CPU (Central Processing Unit), executing a program stored in a program memory. In addition, some or all of these functional units may be realized by hardware, such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or may be realized by software and hardware working together. The program may be stored in advance in a storage device (storage device having a non-transient storage medium), such as an HDD (Hard Disc Drive) or a flash memory, of the defect inspection device 300, or may be stored in a removable storage medium, such as a DVD or a CD-ROM, and installed in the HDD or flash memory of the defect inspection device 300 by mounting the storage medium (non-transient storage medium) in a drive device.

[0017] The defect inspection unit 310 inspects the embossed carrier tape 10 for defects using an image captured by the imaging device 200. The defect inspection unit 310 inspects whether or not there is a defect in the embossed carrier tape 10 based on, for example, a correlation value based on a comparison between a reference image and a captured image. The reference image is, for example, an image of a non-defective product. The learning unit 320 constructs an estimation model for estimating optimal lighting conditions for the embossed carrier tape 10. The estimation unit 330 performs calculations using the estimation model to estimate optimal lighting conditions for defect inspection of the embossed carrier tape 10. The lighting control unit 340 controls the lighting device 100 according to the lighting conditions estimated by the estimation unit 330.

[0018] The information input unit 400 is, for example, a user interface device operated by a user. The information input unit 400 receives object information and defect type information based on the user's operation, and outputs the information to the defect inspection device 300.

[0019] The teacher data storage unit 500 is realized by a storage device such as a HDD, a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). The teacher data storage unit 500 may be realized by a SAN (Storage Area Network) or a NAS (Network Attached Storage). The teacher data storage unit 500 stores defect type information, illumination condition information, and object information as teacher data. The defect type information is information indicating the type of defect of the embossed carrier tape 10. The illumination condition information includes at least one illumination condition of the arrangement of the multiple lights constituting the illumination device 100, the light intensity, the light angle, and the light wavelength. The object information is information indicating a part of the embossed carrier tape 10 as the object, but is not limited thereto. The object information may be physical property information including at least one of the shape, color, transmittance, surface property, molding method, and mold shape of the embossed carrier tape 10.

[0020] The notification unit 600 is, for example, a display or a speaker, and notifies, for example, the result of defect inspection, the result of estimating the lighting conditions, the result of controlling the lighting device 100, and the like.

[0021] 3 is a block diagram showing an example of the learning unit 320 in the embodiment. The learning unit 320 includes, for example, a teacher data acquisition unit 321, a model construction unit 322, a Convolutional Neural Network (CNN) 323, and a learning result storage unit 324.

[0022] FIG. 4 is a diagram showing the relationship between input variables and output variables in the convolutional neural network 323. The input variables are defect type information and object information, and the output variables are illumination condition information. In FIG. 4, the input variables and output variables included in each row correspond to one embossed carrier tape 10. For example, the shape information of the embossed carrier tape 10 having product information "1" is a1, the transparency information is b1, the surface property information is c1, and the defect type information is d1, and the arrangement information of the illumination device 100 optimal for inspecting defects in the embossed carrier tape 10 having product information "1" is A1, the wavelength information is B1, and the intensity information is C1.

[0023] The teacher data acquisition unit 321 acquires defect type information, object information, and illumination condition information as teacher data from the teacher data storage unit 500. The model construction unit 322 inputs the defect type information and object information to the convolutional neural network 323, and learns the processing parameters of the convolutional neural network 323 so as to output output variables from the convolutional neural network 323. Specifically, the model construction unit 322 uses the teacher data to recursively calculate (update) the processing parameters so as to reduce the difference between the output variables corresponding to the input variables shown in FIG. 4 and the target output. The model construction unit 322 performs, for example, deep learning to acquire the processing parameters. Deep learning is machine learning using a multi-layered neural network, particularly a neural network with three or more layers. In the embodiment, the convolutional neural network 323 is used as the neural network with the multi-layered structure. The model construction unit 322 stores the updated processing parameters in the learning result storage unit 324.

[0024] FIG. 5 is a block diagram showing an example of the estimation unit 330 in the embodiment. The estimation unit 330 includes an information acquisition unit 331, an estimation processing unit 332, a convolutional neural network 333, and an estimation result storage unit 334. The information acquisition unit 331 acquires defect type information and / or object information from the information input unit 400. The convolutional neural network 333 is an estimation model in which processing parameters stored in the learning result storage unit 324 are set. When the convolutional neural network 333 receives the defect type information and the object information, the convolutional neural network 333 outputs the illumination condition information. The estimation processing unit 332 stores the estimation result output from the convolutional neural network 333 in the estimation result storage unit 334. The estimation result includes information such as the arrangement, wavelength, and intensity of the illumination device 100 as information corresponding to the illumination condition information shown in FIG. 4.

[0025] In the above embodiment, the defect type information and object information are described as input variables, but the learning unit 320 may construct the convolutional neural network 323 so as to use the object information as an input variable and output illumination condition information for inspecting defects that may occur in the object indicated by the object information. The learning unit 320 may also construct the convolutional neural network 323 so as to input a portion of the embossed carrier tape 10 as object information and output illumination condition information for inspecting defects that may occur in the portion.

[0026] FIG. 6 is a top view showing an example of the arrangement of the lighting device 100 in the embodiment. In FIG. 6, for example, one circle indicates one lighting element 100a. The lighting device 100 is configured, for example, by arranging a plurality of lighting elements 100a radially from a virtual central circle. The lighting device 100 is arranged so that the center of the central circle intersects with the production line L. FIG. 7 is a top view showing another example of the arrangement of the lighting device 100 in the embodiment. The lighting device 100 is configured, for example, by arranging a plurality of lighting elements 100a along a plurality of straight lines extending vertically from each side of a virtual quadrangle. The lighting device 100 is arranged so that the center of gravity of the quadrangle intersects with the production line L. The defect inspection system 1 can, for example, switch the arrangement of FIG. 6 or FIG. 7 based on an estimation result according to the object information of the embossed carrier tape 10, adjust the wavelength of the light irradiated from each lighting element 100a, or adjust the intensity of the light irradiated by each lighting element 100a.

[0027] 8 and 9 are side views of the lighting device 100 in the embodiment. As shown in FIG. 8, the lighting device 100 has a plurality of lighting elements 100a arranged along upper and lower arcs with respect to the embossed carrier tape 10 conveyed on the manufacturing line L. Each lighting element 100a may be positioned so that the irradiated light is directed toward a predetermined range of the manufacturing line L, and may be dimmed so that the irradiated light is directed toward the predetermined range of the manufacturing line L. A certain lighting element 100a irradiates the irradiated light to a predetermined range via a mirror 110. The imaging device 200 is arranged at a position to image the predetermined range of the manufacturing line L. As shown in FIG. 9, the defect inspection system 1 can also irradiate the embossed carrier tape 10 with the irradiated light from a part of the lighting elements 100a among the plurality of lighting elements based on the estimation result, for example.

[0028] FIG. 10 is a top view showing an example of an embossed carrier tape 10 of an embodiment, where (a) is an example of an embossed carrier tape 10 without defects, and (b) is an example of an embossed carrier tape 10 with defects. A plurality of recesses 10a and holes 10b are formed in the embossed carrier tape 10. The defect inspection system 1 detects defects such as deformation of the recesses 10a, burrs in the holes 10b, burrs at the ends of the embossed carrier tape 10, foreign matter on the surface of the embossed carrier tape 10, and damage to the surface of the embossed carrier tape 10, as shown in FIG. 10(b). For this purpose, the defect inspection system 1 can estimate optimal lighting conditions for inspecting the type of defect by learning defect type information as an input variable. Furthermore, the defect inspection system 1 can estimate optimal lighting conditions for inspecting the defect in order to inspect the defect that may occur in a portion of the embossed carrier tape 10.

[0029] FIG. 11 shows the relationship between the illumination conditions, the defect types, and the correlation data in the embodiment. The waveform in FIG. 11 is the correlation data, and the horizontal axis of the correlation data is the correlation value, and the vertical axis is the frequency. The correlation data in FIG. 11 includes a waveform corresponding to a non-defective image (right side in the figure) and a waveform corresponding to a defective image (left side in the figure). The non-defective image is an image of an embossed carrier tape 10 without defects. The defective image is an image of an embossed carrier tape 10 with defects. The waveform corresponding to the non-defective image has a peak value corresponding to the correlation value (first correlation value c1) between the reference image and the non-defective image. The waveform corresponding to the defective image has a peak value corresponding to the correlation value (second correlation value c2) between the reference image and the defective image. The reference image is an image of an embossed carrier tape 10 without defects captured in advance under illumination conditions of an arbitrary angle, waveform, and brightness. The first correlation value c1 is a value higher than the second correlation value c2. The more suitable the illumination conditions are for the type of defect, the greater the difference between the first correlation value c1 and the second correlation value c2.

[0030] The correlation data includes a first correlation value range between a reference image and an image of an object without a defect, and a second correlation value range between the reference image and an image of an object with a defect. The first correlation value range and the second correlation value range are narrower as the illumination condition is more suitable for the type of defect of the object. For example, for defect type a, the correlation value range between the reference image and the image of a non-defective product under illumination condition A is narrower than the correlation value range between the reference image and the image of a non-defective product under illumination condition C.

[0031] For example, when the optimal lighting condition for inspecting defect type a is A, the reference image is an image of embossed carrier tape 10 captured under lighting condition A. Defect inspection unit 310 calculates a correlation value between an image of embossed carrier tape 10 without defects captured under lighting condition A and the reference image, and if the calculated correlation value is within a predetermined range from pre-stored correlation value c1, determines that embossed carrier tape 10 does not have a defect of defect type a. Defect inspection unit 310 calculates a correlation value between an image of embossed carrier tape 10 with defect type a captured under lighting condition A and the reference image, and if the calculated correlation value is within a predetermined range from pre-stored correlation value c2, determines that embossed carrier tape 10 has a defect of defect type a.

[0032] From the above, it is desirable that the learning unit 320 constructs the convolutional neural network 323 by using the above-mentioned correlation data so as to output lighting condition information related to lighting conditions that can obtain a low correlation with an image of an object having the defect and a high correlation with a reference image obtained by imaging an object without the defect when the defect type information is input. Also, the estimation unit 330 can output lighting condition information related to lighting conditions that can obtain a high correlation with the reference image when the defect type information is input.

[0033] FIG. 12 is a diagram showing another configuration of the defect inspection system 1 of the embodiment. The invention according to claim 1 of the present application described above may estimate the optimal illumination condition for one defect type and control the illumination device 100 based on the estimated illumination condition, but is not limited thereto. The defect inspection system 1 may include a plurality of pairs of illumination devices 100 and imaging devices 200 (a pair of 100a and 200b, a pair of 100b and 200b, ...). The defect inspection system 1 inspects the embossed carrier tape 10 for defect type a, and then inspects the embossed carrier tape 10 for defect type b. At this time, when the defect inspection system 1 receives object information, it estimates the optimal illumination condition for the defect type a based on the object information, controls the illumination device 100a, and inspects the defect type a based on the image captured by the imaging device 200a, and estimates the optimal illumination condition for the defect type b based on the object information, controls the illumination device 100b, and inspects the defect type a based on the image captured by the imaging device 200b. This allows the defect inspection system 1 to inspect for a plurality of defect types.

[0034] [Convolutional Neural Network] Hereinafter, a description will be given of an example of the convolutional neural network 323 and the convolutional neural network 333 in the embodiment. In this description, the convolutional neural network 323 and the convolutional neural network 333 will be collectively referred to as a "convolutional neural network."

[0035] As described above, the model construction unit 322 sets the teacher data as an input variable to be input to the input layer of the learning convolutional neural network 323, and sets the lighting condition information as an output variable to be output from the output layer. The model construction unit 322 performs machine learning using a learning data set of defect type information, object information, and lighting condition information. The estimation processing unit 332 inputs the object information to the input layer of the trained convolutional neural network 333, and acquires the lighting condition information from the output layer.

[0036] FIG. 13 is a diagram for explaining a process of performing an estimation process using a convolutional neural network in an embodiment. The convolutional neural network includes, for example, a layer L0, a layer L1, a layer L2, a layer Li, and a layer LI. The layer L0 is also called an input layer, the layers L1 to Li are also called intermediate layers or hidden layers, and the layer LI is also called an output layer. In the convolutional neural network, object information is input to the input layer L0. The defect type information and object information are represented by a matrix D11 in which parameters of various information are positions in the matrix. Each element of the matrix D11 includes an R (red) sub-value, a G (green) sub-value, and a B (blue) sub-value as sub-values ​​of parameters corresponding to the positions in the matrix. The first intermediate layer L1 is a layer in which convolution processing (also called filtering processing) and pooling processing are performed.

[0037] (Convolution) An example of the convolution process of the intermediate layer L1 will be described. The convolution process is a process of filtering the original parameters and outputting a feature map. Specifically, the input parameters are divided into a sub-matrix D121 of R, a sub-matrix D122 of B, and a sub-matrix D123 of G. For each of the sub-matrices D121, D122, and D123 (each of which is also referred to as a "sub-matrix D12"), the elements of the sub-matrix are multiplied by the elements of a convolution matrix CM1 (also called a kernel) of s rows and t columns for each sub-matrix of s rows and t columns, and then added together to calculate a first parameter value. The first parameter values ​​calculated in each of the sub-matrixes D12 are multiplied by a weighting coefficient and then added together to calculate a second parameter value. The second parameter values ​​are set as the elements of the convolution matrix D131 as matrix elements corresponding to the positions of the sub-matrix. In each sub-matrix D12, the position of the submatrix is ​​shifted for each element (sub-parameter), so that the second parameter value at each position is calculated, and all matrix elements of the convolution matrix D131 are calculated.

[0038] FIG. 13 is an example of a convolution neural network using a 3-row, 3-column convolution matrix CM (convolution matrix)1, and the first parameter value D1311 is calculated for the 3-row, 3-column submatrix from the 2nd row to the 4th row and the 2nd column to the 4th column of each submatrix D12. A weighting factor is calculated and added to each of the first parameter values ​​of the submatrices D121, D122, and D123, thereby calculating a second parameter value as the matrix element in the 2nd row, 2nd column of the convolution matrix D131. Similarly, the second parameter value of the matrix element in the 3rd row, 2nd column of the convolution matrix D131 is calculated from the submatrix from the 3rd row to the 5th row and the 2nd column to the 4th column. Similarly, the convolution matrices D132, ... are calculated using other weighting factors or other convolution matrices.

[0039] (Pooling) An example of the pooling process of the intermediate layer L1 will be described. The pooling process is a process for reducing a parameter group (matrix) while retaining the characteristics of the defect type information and the target object information. Specifically, for each region PM (pooling matrix) of u rows and v columns in the convolution matrix D131, a representative value of the matrix elements in the region is calculated. The representative value is, for example, the maximum value of the matrix elements in the region. The representative value is set for each element of the CNN matrix D141 as a matrix element corresponding to the position of the region PM. The region in the convolution matrix D131 is shifted for each region PM, so that a representative value at each position is calculated, and all elements of the CNN matrix D141 are calculated.

[0040] FIG. 13 is an example of a convolutional neural network using a region PM of 2 rows and 2 columns, for example, and for the region PM of 2 rows and 2 columns from the 3rd row to the 4th row and the 3rd column to the 4th column of the convolution matrix D131, the maximum value in the region PM is calculated as a representative value. This representative value is set to the matrix element of the 2nd row and the 2nd column of the CNN matrix D141. Similarly, a representative value of the matrix element of the 3rd row and the 2nd column of the CNN matrix D141 is calculated from the submatrix of the 5th row to the 6th row and the 2nd column to the 4th column. Similarly, CNN matrices D142, ... are calculated from the convolution matrices D132, ....

[0041] Each matrix element (N elements) of the CNN matrices D141, D142, ... is generated as a vector X by arranging them in a predetermined order. The vector X is generated by arranging N elements x n (n=1, 2, 3, . . . N). Vector X is a vector u (0) is equivalent to.

[0042] The hidden layer Li represents the i-th (i = 2, ) hidden layer. The hidden layer Li is a vector u (i) Each node in the hidden layer Li receives a vector z (i) The vector z (i) is the vector u (i) is the activation function f(u (i) ) has the values ​​entered in the vector u (i) is the vector z output from the i-1th hidden layer node. (i-1) The weight matrix W (i) and the vector b (i) The vector b has a value obtained by adding (i) is the bias.

[0043] The output layer LI is a vector z (I-1) The output of the output layer LI is M y m (m=1, 2, . . . M). In other words, the output layer LI is a set of M y m A vector Y(y1, y2, y3, y M) is output. As described above, when parameters of defect type information and object information are input as input variables, the convolutional neural network outputs vector Y as an output variable. Vector Y in the embodiment represents illumination condition information.

[0044] Fig. 14 is a diagram for explaining the learning process using a convolutional neural network in the embodiment. The vector output from the first hidden layer L1 for the parameter values ​​of the object information in the learning data set is defined as vector [X]. The vector representing the determined class in the learning data set is defined as vector [Y]. The determined class is the output coefficient (illumination condition information of the arrangement, wavelength, and intensity of the lighting elements) shown in Fig. 4 in the embodiment.

[0045] Weight matrix W (i) An initial value is set for . When a vector [X] is input to the second hidden layer L2 based on the defect type information and object information input to the input layer L0, a vector Y(X) based on the vector [X] is output from the output layer LI. The error E between vector Y(X) and vector [Y] is calculated using a loss function. The gradient ΔE of the i-th layer i is the output from each layer z i and the error signal δ i The error signal δ is calculated using i is the error signal δ i-1 The process of calculating the error signal of the input layer side from the error signal of the output layer side from the output layer side to the input layer side from the output layer side is also called backpropagation. (i) is the gradient ΔE i Similarly, in the first hidden layer L1, the convolution matrix CM or the weighting coefficients are updated.

[0046] [Estimation model settings] The model construction unit 322 sets processing parameters for the convolutional neural network, such as the number of layers, the number of nodes in each layer, the node connection method between each layer, the activation function, the error function, the gradient descent algorithm, the pooling region, the kernel, the weighting coefficient, and the weighting matrix. For example, the model construction unit 322 sets the number of layers to three (I=3). As the number of nodes in each layer (also referred to as the "node number"), the model construction unit 322 sets the number of elements of the vector X (node ​​number N) to 800, the number of nodes in the second intermediate layer (i=2) to 500, and the number of nodes in the output layer (i=3) to 10. However, the embodiment is not limited to this, and the total number may be four or more layers, and a different value may be set for the number of nodes.

[0047] The model construction unit 322 sets 20 convolution matrices CM of 5 rows and 5 columns, and sets a region PM of 2 rows and 2 columns. However, the embodiment is not limited to this, and a different number of matrices or a different number of convolution matrices CM may be set. Also, a region PM of a different number of matrices may be set. The model construction unit 322 may perform more convolution processes or pooling processes.

[0048] The model construction unit 322 sets a full connection as the connection of each layer of the convolutional neural network. However, the embodiment is not limited to this, and the connections of some or all layers may be set to non-full connection. The model construction unit 322 sets a sigmoid function as the activation function of all layers. However, the embodiment is not limited to this, and the activation function of each layer may be other activation functions such as a step function, linear combination, soft sine, soft plus, ramp function, truncated power function, polynomial, absolute value, radial basis function, wavelet, maxout, etc. Also, the activation function of a certain layer may be a different type from that of other layers.

[0049] The model construction unit 322 sets squared loss (mean squared error) as the error function. However, the present invention is not limited to this, and the error function may be cross entropy, τ-quantile loss, Huber loss, or ε sensitivity loss (ε tolerance error function). In addition, the model construction unit 322 sets SGD (stochastic gradient descent) as an algorithm (gradient descent algorithm) for calculating the gradient. However, the present invention is not limited to this, and the gradient descent algorithm may be Momentum (inertia term) SDG, AdaGrad, RMSprop, AdaDelta, Adam (adaptive moment estimation), or the like.

[0050] The model construction unit 322 may set other neural networks such as a perceptron neural network, a recurrent neural network (RNN), a residual network (ResNet), etc. The model construction unit 322 may set a part or all of a supervised learning estimation model such as a decision tree, a regression tree, a random forest, a gradient boosting tree, a linear regression, a logistic regression, or an SVM (support vector machine).

[0051] [Effects of the embodiment] As described above, according to the embodiment, it is possible to realize a learning device (learning unit 320) including: teacher data acquisition unit 321 that inputs defect type information indicating the type of defect in embossed carrier tape 10, lighting condition information related to the lighting conditions under which lighting device 100 irradiates light corresponding to the type of defect in embossed carrier tape 10, and object information for identifying embossed carrier tape 10; and model construction unit 322 that constructs a model that is machine-learned using the defect type information, lighting condition information, and object information input by teacher data acquisition unit 321 as teacher data, and that outputs lighting condition information related to the optimum conditions for embossed carrier tape 10 when the defect type information and / or object information is input. According to this embodiment, it is possible to obtain optimum lighting condition information for the embossed carrier tape 10 to be measured simply by inputting information about the embossed carrier tape 10 to be measured and the defect type into the model, so that the embossed carrier tape 10 can be inspected with high accuracy.

[0052] However, since the accuracy and reproducibility of the inspection change significantly depending on the illumination conditions, it may be difficult to find the optimal illumination conditions. Conventionally, one illumination condition has been improved so that multiple defects that may occur on the embossed carrier tape 10 can be inspected as much as possible, and the improvement of the illumination conditions has been estimated based on the experience of an expert. However, since the optimal illumination conditions differ depending on the type of defect, the material such as the color and transparency of the embossed carrier tape 10, the surface property (reflective, glossy or non-glossy), the shape (flat or uneven), the location, and the characteristics of the embossed carrier tape 10 such as the surface shape and the back shape of the embossed carrier tape 10, it may take a long time to estimate the optimal illumination conditions for multiple inspections with high accuracy. In contrast, according to the embodiment, the estimation time of the illumination conditions can be shortened by constructing the above-mentioned model.

[0053] Moreover, according to the embodiment, the arrangement of the lighting device 100, the light intensity, the light angle, and the light wavelength are used as the lighting condition information, so it is possible to estimate the optimal arrangement of the lighting device 100, the light intensity, the light angle, and the light wavelength. Furthermore, according to the embodiment, the part of the embossed carrier tape 10 is used as the object information, so it is possible to estimate the optimal lighting conditions for defect inspection of the part of the embossed carrier tape 10. Furthermore, according to the embodiment, the physical property information of the embossed carrier tape 10, such as the shape, color, transmittance, surface properties, molding method, and mold shape, is used as the object information, so it is possible to estimate the optimal lighting conditions for defect inspection of the physical property information.

[0054] Moreover, according to the embodiment, it is possible to realize an estimation device (estimation unit 330) including an estimation processing unit 332. According to this embodiment, it is possible to acquire optimal illumination condition information for the embossed carrier tape 10 to be measured simply by inputting information about the type of defect and the embossed carrier tape 10 to be measured into the model, and therefore it is possible to inspect the embossed carrier tape 10 with high accuracy.

[0055] The program of the defect inspection device 300 in one aspect of the present invention may be a program that controls one or more processors such as a CPU (a program that makes a computer function) so as to realize the functions shown in each of the above-mentioned embodiments and modifications related to one aspect of the present invention. Information handled by each of these devices may be temporarily stored in a RAM during processing, and then stored in various storage devices such as a flash memory, SSD, or HDD, and may be read, modified, or written by the CPU as necessary.

[0056] In addition, a part or the whole of the learning unit 320 or the estimation unit 330 included in the defect inspection device 300 in each of the above-mentioned embodiments and modifications may be realized by a computer having one or more processors. In that case, the control function may be realized by recording a program for realizing the control function in a computer-readable recording medium, reading the program recorded in the recording medium into a computer system, and executing the program. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" may include a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in that case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may further be a program that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0057] In addition, a part or the whole of the learning unit 320 or the estimation unit 330 included in the defect inspection device 300 in each of the above-mentioned embodiments and modifications may be realized as an LSI, which is typically an integrated circuit, or may be realized as a chipset. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit and / or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, it is also possible to use an integrated circuit based on that technology.

[0058] Although the embodiments and modifications have been described above in detail with reference to the drawings as one aspect of the present invention, the specific configuration is not limited to the embodiments and modifications, and includes design changes within the scope of the gist of the present invention. In addition, one aspect of the present invention can be modified in various ways within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. In addition, configurations in which elements described in the above embodiments and modifications are replaced with elements that have the same effect are also included.

[0059] For example, one aspect of the present invention may be realized by combining some or all of the above-described embodiments.

[0060] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0061] 1. Defective inspection system 100 Lighting Equipment 200 Imaging device 300 Defective inspection device 310 Defective Inspection Department 320 Learning Department 330 Estimation Department 340 Lighting control section 400 Information input section 500 Teacher data storage unit 600 Notification Department

Claims

1. an input unit for inputting defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device irradiates light corresponding to the type of defect of the object, and object information for identifying the object; a model construction unit that constructs a model by machine learning using the defect type information, the illumination condition information, and the object information input by the input unit as training data, the model outputting the illumination condition information related to the optimum condition for the object when the defect type information and the object information are input; A learning device comprising:

2. the illumination condition information is data indicating a relationship between a type of defect of the object, a condition under which the illumination device irradiates light, and correlation data based on a correlation between a reference image and an image representing a defect state of the object, the model construction unit constructs the model so as to output the illumination condition information relating to an illumination condition that can obtain a high correlation with a reference image when the defect type information is input. The learning device according to claim 1 .

3. The learning device of claim 2, wherein the correlation data includes a first correlation value between a reference image and an image of an object without defects and a second correlation value between the reference image and an image of an object with defects, and is information indicating that the first correlation value is higher than the second correlation value.

4. 3. The learning device of claim 2, wherein the correlation data includes a first correlation value range between a reference image and an image of an object without defects, and a second correlation value range between the reference image and an image of an object with defects, and the first correlation value range and the second correlation value range are information indicating that the lighting conditions are narrow enough to be suitable for a type of defect of the object.

5. The learning device according to claim 1 , wherein the lighting condition information includes at least one of conditions of the arrangement of the lighting device, the intensity of light, the angle of light, and the wavelength of light.

6. The learning device according to claim 1 , wherein the object information indicates a portion of the object, and the illumination condition information indicates an optimum illumination condition for defect inspection of the portion of the object.

7. The learning device of claim 1, wherein the object information indicates physical property information including at least one of the shape, color, transparency, surface properties, molding method, and mold shape of the object, and the lighting condition information indicates optimal lighting conditions for defect inspection of the object indicated by the physical property information.

8. an input unit for inputting defect type information indicating a type of defect in an object and object information for identifying the object; an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using teacher data including the defect type information, the object information, and lighting condition information related to lighting conditions under which a lighting device corresponding to a type of defect of the object irradiates light, and estimates optimal lighting conditions for the object based on the lighting condition information related to optimal conditions for the object output from the model; An estimation device comprising:

9. A lighting device that irradiates light onto an object; An inspection unit that inspects the object; an input unit for inputting defect type information indicating a type of defect of the object and object information for identifying the object; an estimation unit that inputs the defect type information and the object information input by the input unit into a model that has been machine-learned using teacher data including the defect type information, the object information, and lighting condition information related to lighting conditions under which a lighting device corresponding to a type of defect of the object irradiates light, and estimates optimal lighting conditions for the object based on the lighting condition information related to optimal conditions for the object output from the model; a control unit that controls the lighting device based on the lighting condition estimated by the estimation unit; A defect inspection device comprising:

10. A step in which a defect inspection device inputs defect type information indicating a type of defect of an object, lighting condition information regarding lighting conditions under which a lighting device corresponding to the type of defect of the object irradiates light, and object information for identifying the object; constructing a model in which the defect inspection device has machine-learned the input defect type information, the illumination condition information, and the object information using training data, the model outputting the illumination condition information relating to the optimum conditions for the object when the defect type information and the object information are input; a step of inputting the defect type information and object information for identifying the object to the defect inspection device, inputting the defect type information and the object information to the model, and estimating optimal illumination conditions for the object based on the illumination condition information relating to optimal conditions for the object output from the model; The defect inspection device controls the lighting device based on the estimated lighting conditions; The defect inspection device inspects the object; A defect inspection method including:

Images