Inspection device and inspection method

Abstract

To provide an inspection device capable of improving inspection accuracy of the inspection device.SOLUTION: An inspection device comprises: a captured image acquisition section that acquires a captured image of an object to be inspected; a storage device that stores a learning model that is a learning model learned using a normal image indicating a normal object to be inspected to generate a comparison image approximate to the normal image; and an inspection section that determines the presence or absence of anomaly in the object to be inspected using the captured image and the comparison image. The learning model includes a generator that generates a comparison image by input of first noise and second noise. The generator includes a mapping network layer that non-linearly converts the first noise to generate third noise and a synthesis network layer that generates an image using the second noise and the generated third noise and repeats up-sampling the generated image a plurality of times to generate the comparison image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an inspection apparatus and an inspection method. [Background technology]

[0002] A pass / fail judgment device is known that calculates features from each sample image based on the results of associating pass / fail judgment results with the sample image, and generates pass / fail judgment criteria by learning the calculated features using a neural network (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-114843 A Summary of the Invention [Problem to be solved by the invention]

[0004] In inspection devices that use learning models, there is a demand for improved inspection accuracy. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to one embodiment of the present disclosure, an inspection device is provided. The inspection device includes an image acquisition unit that acquires an image of an inspection target, a storage device that stores a learning model that has been trained using a normal image showing a normal inspection target and is used to generate a comparison image that approximates the normal image, and an inspection unit that determines whether or not the inspection target has an abnormality by using the captured image and the comparison image. The learning model includes a generator that generates the comparison image by inputting a first noise and a second noise, the generator including a mapping network layer that performs nonlinear conversion on the first noise to generate a third noise, and a synthesis network layer that generates an image using the second noise and the generated third noise, and generates the comparison image by repeating upsampling of the generated image multiple times. According to the inspection device of this embodiment, the presence or absence of an abnormality can be determined using high-resolution image data, and the inspection accuracy of the inspection device can be improved. (2) In the inspection device of the above aspect, the learning model does not need to include a classifier for discriminating between the captured image and the comparison image. According to the inspection device of this aspect, the captured image and the comparison image can be compared with each other using a simple configuration, and the productivity of the inspection device can be increased. (3) In the inspection device of the above aspect, the learning model may be a generative adversarial network further including a classifier that outputs a numerical value indicating the degree of difference between the comparison image and the captured image. According to the inspection device of this aspect, it is possible to determine the presence or absence of an abnormality by using a captured image and a comparison image, utilizing a known configuration. (4) In the inspection device of the above form, when the inspection unit determines that the captured image and the comparison image are not similar, the inspection unit may repeat a predetermined number of re-evaluations to determine whether or not there is an abnormality in the inspection object using a new comparison image generated by changing at least one of the first noise and the second noise. According to the inspection device of this aspect, by changing the characteristics of the comparison image multiple times, it is possible to prevent abnormalities from being overlooked, and it is possible to improve the inspection accuracy. (5) In the inspection device of the above form, the re-judgment may include a first re-judgment in which the second noise is fixed and the re-judgment is performed a predetermined number of times using the comparison image generated by varying the first noise, and a second re-judgment in which, after the first re-judgment, the first noise is fixed and the re-judgment is performed a predetermined number of times using the comparison image generated by varying the second noise. According to this form of inspection device, by first performing the first re-evaluation, which changes the first noise that significantly changes the features in the image, the abnormality judgment can be completed earlier than when the second re-evaluation is performed first. (6) In the inspection device of the above aspect, the generator may further include a weight adjustment layer that changes the first noise to be input to the mapping network layer so as to reduce the difference between the captured image and the comparison image. According to the inspection device of this aspect, a comparison image that is similar to the captured image can be generated by a simple method of changing the first noise, without training the generator during inspection. (7) In the inspection device of the above form, the inspection unit may determine that there is an abnormality in the inspection object when an accumulated value of the difference between a gradation value of a first pixel included in the captured image and a gradation value of a second pixel at a position corresponding to the first pixel in the comparison image is greater than a predetermined threshold value. According to the inspection device of this aspect, it is possible to prevent the difference value of the RGB gradation values, which is easily recognized by humans, from becoming excessively large, compared to the case where the mean square error is used to calculate the likelihood. The present disclosure can also be realized in various forms other than an inspection device and an inspection method, such as an inspection program, a trained model, a learning method for a learning model, a manufacturing method for a part to be inspected, a manufacturing method for an inspection device, a control method for an inspection device, a learning program for a machine learning model, a non-transitory recording medium on which these computer programs are recorded, and the like. [Brief description of the drawings]

[0007] [Figure 1]FIG. 1 is an explanatory diagram illustrating a schematic overall configuration of an inspection system. [Diagram 2] FIG. 2 is a block diagram showing the internal functional configuration of the inspection apparatus. [Diagram 3] An explanatory diagram conceptually showing the configuration of a machine learning model. [Figure 4] FIG. 1 is an explanatory diagram showing a detailed configuration of a machine learning model. [Diagram 5] 4 is a flowchart showing an inspection method executed by the inspection device. [Figure 6] FIG. 2 is an explanatory diagram showing a connector as an example of an inspection target. [Figure 7] FIG. 13 is an explanatory diagram showing an example of a captured image as a normal image showing a normal connector. [Figure 8] FIG. 13 is an explanatory diagram showing an example of a captured image as an abnormality image showing a connector having an abnormality. [Figure 9] FIG. 11 is an explanatory diagram showing a comparison image generated using a captured image as a normal image. [Figure 10] FIG. 11 is an explanatory diagram showing a comparison image generated using a captured image as an abnormal image. [Figure 11] FIG. 4 is an explanatory diagram showing a difference image generated using the difference in RGB gradation values ​​between a captured image and a comparison image. [Figure 12] FIG. 11 is an explanatory diagram illustrating a method for calculating a difference. [Figure 13] FIG. 11 is an explanatory diagram showing a schematic configuration of a machine learning model provided in an inspection device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: FIG. 1 is an explanatory diagram showing a schematic overall configuration of an inspection system 100. The inspection system 100 includes an inspection device 60 and a camera 70 as a first embodiment of the present disclosure. The inspection device 60 cooperates with the camera 70 to inspect the presence or absence of an abnormality of an inspection object using a captured image of the inspection object processed in the manufacturing line Ln. The inspection device 60 uses a trained machine learning model. The inspection object is, for example, an automobile part WK manufactured in a front-end process PR1. In this embodiment, a connector, which will be described later, will be used as an example of the automobile part WK. The connector is a member used to connect pipes that circulate fuel, refrigerant, and the like in an automobile. The abnormality to be detected may include, for example, a poor connection of a connector, damage to a connector, poor fitting of a locking part of a connector, damage to a pipe, and the like, and may include many types of abnormalities. In this embodiment, a machine learning model for unsupervised learning is used because it is difficult to prepare images of the inspection object having an abnormality.

[0009] The camera 70 captures an image of the inspection target and transmits the captured image of the inspection target to the inspection device 60. The camera 70 includes an imaging unit 72 and a communication unit 74. The imaging unit 72 captures an image of the part WK during transport from the front-end process PR1 to the back-end process PR2. In this embodiment, the imaging unit 72 captures an image of the part WK while it is mounted on the automobile. The captured image captured by the imaging unit 72 is output to the communication unit 74. The camera 70 is not limited to being separate from the inspection device 60, and may be integrated with the inspection device 60.

[0010] In this embodiment, the image captured by the imaging unit 72 is composed of an RGB input image signal composed of each image signal component represented by R (red), G (green), and B (blue). For example, 8 bits are assigned to each image signal, totaling 24 bits of information. In this embodiment, the imaging unit 72 acquires a so-called 12 million pixel image of the entire part WK, which is 4000 pixels horizontally and 3000 pixels vertically. It is preferable that the resolution of the image acquired by the imaging unit 72 is determined in consideration of the inspection range and defect detection accuracy required for the inspection device 60, the size of the defect to be detected, the image size that can be processed by the learning model, and the like. In addition, it is preferable that the image of the inspection target has a high resolution in order to improve the inspection accuracy and expand the inspection range of the inspection device 60. For example, in order to inspect the part WK in a state where it is mounted on an automobile, for example, the inspection target may be imaged from a distant position in order to move the camera 70 away from the automobile to avoid contact between the camera 70 and the automobile. In addition, there are cases where the imaging range is small, and cases where a small defect is detected in a large imaging range. In these cases, it is preferable that the image of the inspection target has a high resolution.

[0011] The communication unit 74 transmits the captured image to the inspection device 60 by wireless communication in accordance with an arbitrary communication protocol. The communication unit 74 may receive an execution command from the inspection device 60.

[0012] In this embodiment, the inspection device 60 and the camera 70 can transmit and receive data to each other by wireless communication, and can also transmit and receive data to the information processing device PC and the terminal devices PD1 and PD2 shown in FIG. 1. For example, the inspection device 60 can transmit inspection results to the information processing device PC and the terminal devices PD1 and PD2 in response to requests from the information processing device PC and the terminal devices PD1 and PD2. The inspection device 60 may be disposed in an area near the production line Ln as a local server, or may be disposed in a location away from the production line Ln as a remote server. When the inspection device 60 is installed as a remote server, the captured image from the camera 70 and the inspection results to the terminal devices PD1 and PD2 and the information processing device PC are transmitted and received by the inspection device 60 via a wireless access point in the factory and a network such as an intranet or the Internet.

[0013] 2 is a block diagram showing the internal functional configuration of the inspection device 60. The inspection device 60 includes a CPU 62 as a central processing unit, a storage device 64, and a communication unit 66. The CPU 62, the storage device 64, and the communication unit 66 are connected to each other via a bus 61 and are capable of bidirectional communication. The inspection device 60 may further include a display unit for displaying the inspection results, such as a liquid crystal display or a touch panel.

[0014] The communication unit 66 is an interface that performs communication control to receive captured images to be used in the inspection between the camera 70 and the communication unit 66 via a network. In this embodiment, the communication unit 66 functions as a captured image acquisition unit that acquires captured images of the inspection target via the camera 70. In this embodiment, the communication unit 66 further transmits the inspection results and display data of the inspection results to the terminal devices PD1 and PD2 and the information processing device PC. The communication unit 66 may further receive command signals requesting the execution of various processes from the terminal devices PD1 and PD2 and the information processing device PC. Note that in the case where the inspection device 60 includes the imaging unit 72, such as when the inspection device 60 is integrated with the camera 70, the imaging unit 72 functions as the captured image acquisition unit.

[0015] The storage device 64 is, for example, a RAM, a ROM, or a hard disk drive (HDD). The HDD or ROM stores various programs for implementing the functions provided in this embodiment. The various programs read from the HDD or ROM are expanded on the RAM and executed by the CPU 62. The storage device 64 includes a learning model storage unit 642 for storing a machine learning model. The storage device 64 temporarily stores image data acquired by the captured image acquisition unit, various images and calculation results generated by the inspection unit 622, and the like. The storage device 64 may be an optical disk, a solid state drive (SSD), a flash memory, or the like.

[0016] The CPU 62 executes a program stored in the storage device 64 to function as an inspection unit 622. The inspection unit 622 uses a machine learning model stored in a learning model storage unit 642 to inspect whether or not an inspection target has an abnormality.

[0017] 3 is an explanatory diagram conceptually showing the configuration of the machine learning model 80 stored in the learning model storage unit 642. In this embodiment, the learning model storage unit 642 stores the machine learning model 80 that has been trained using a normal image that indicates a normal inspection target. The machine learning model 80 includes a weight adjustment layer 82 and a generator 84.

[0018] The generator 84 is in a state of being sufficiently trained using a captured image of a normal test subject (hereinafter also referred to as a "normal image"). The generator 84 generates image data (hereinafter also referred to as a "comparison image") that is similar to the captured image by inputting a first noise Z1 and a second noise Z2 as latent variables. In this embodiment, a machine learning model also called StyleGAN is applied to the generator 84, as described later.

[0019] The first noise Z1 and the second noise Z2 are, for example, latent variables with about 100 dimensions, each element of which is a variable ranging from 0 to 1. The first noise Z1 and the second noise Z2 are determined by the inspection unit 622, for example, by random sampling from a uniform distribution or a normal distribution, and input to the generator 84.

[0020] When the inspection unit 622 acquires the comparison image PR and the captured image PT acquired by the imaging unit 72, it compares the captured image PT with the comparison image PR to determine whether or not there is an abnormality in the inspection object. If the captured image PT and the comparison image PR are similar, the inspection unit 622 determines that there is no abnormality in the inspection object. If the inspection unit 622 determines that the captured image PT and the comparison image PR are not similar, it changes at least one of the first noise Z1 and the second noise Z2 to generate a new comparison image PR and performs a re-determination to determine again whether or not there is an abnormality in the inspection object.

[0021] The weight adjustment layer 82 adjusts the weight of the first noise Z1 input to the generator 84. In this embodiment, the weight adjustment layer 82 applies a fully connected layer using an activation function. The weight adjustment layer 82 changes the first noise Z1 in re-determination so that the difference between the captured image PT and the comparison image PR becomes smaller. As a result, the generator 84 can generate a comparison image PR that is more similar to the captured image PT each time re-determination is performed without new learning during inspection.

[0022] Fig. 4 is an explanatory diagram showing a detailed configuration of the machine learning model 80. The configuration shown in Fig. 4 corresponds to the configuration of the range AR in Fig. 3. As shown in Fig. 4, a StyleGAN model is used for the generator 84. This makes it possible to generate a comparison image PR with a higher resolution than DCGAN, which generates image data with a resolution of, for example, about 64 pixels x 64 pixels.

[0023] The generator 84 includes a mapping network unit 842 and a plurality of synthesis network units 844. The mapping network layer 842 includes a plurality of fully connected layers (FC layers), for example, eight layers. The mapping network layer 842 generates a third noise W1 as a latent space by nonlinearly transforming the first noise Z1 as a latent variable using the eight fully connected layers. The generated third noise W1 is input to the synthesis network layer 844.

[0024] The synthesis network layer 844 generates a comparison image PR from a fixed value CV as a tensor of 4x4x512. The synthesis network layer 844 generates a high-resolution comparison image PR by performing upsampling on the image multiple times and increasing the resolution stepwise. The synthesis network layer 844 is provided with a convolution processing layer and a normalization layer (AdaIN: Adaptive Instance Normalization) AL for each resolution to be upsampled. The synthesis network layer 844 takes in style information and noise information each time upsampling is performed. Specifically, the second noise Z2 is input after the convolution processing layer to take in the noise information that gives a probabilistic fluctuation, and the third noise W1 as a latent space generated by the mapping network layer 842 is taken in as multidimensional style information via the normalization layer AL. The third noise W1 as a latent space and the first noise Z1 for generating the third noise W1 are factors that can affect major features in the image, and the second noise Z2 as noise information is a factor that can affect detailed features.

[0025] 5 is a flowchart showing an inspection method executed by the inspection device 60. This flow starts when the part WK delivered from the previous process PR1 reaches the inspection process. Whether the part WK has reached the inspection process or not can be determined, for example, by performing image recognition of the part WK that has reached the inspection process using a camera 70 or the like.

[0026] In step S10, the inspection device 60 acquires a captured image PT. In this embodiment, the imaging unit 72 of the camera 70 acquires the captured image PT by capturing an image of the part WK to be inspected. The captured image PT acquired by the imaging unit 72 is transmitted to the inspection device 60 via the communication unit 74. The inspection device 60 acquires the captured image PT via the communication unit 66.

[0027] Fig. 6 is an explanatory diagram showing a connector 30 as an example of an inspection target. As shown in Fig. 6, the connector 30 has a main body 36. The main body 36 has a fitting portion 38. With the first pipeline 32 and the second pipeline 34 inserted into the main body 36, the fitting portion 38 is pushed toward the connection portion between the first pipeline 32 and the second pipeline 34, whereby the first pipeline 32 and the second pipeline 34 are fixed in a connected state.

[0028] FIG. 7 is an explanatory diagram showing an example of a captured image PTg as a normal image showing a normal connector 30. FIG. 8 is an explanatory diagram showing an example of a captured image PTn as an abnormal image showing a connector 30 having an abnormality. As shown in FIG. 7, the captured image PTg shows the connector 30 in a normal state in which the fitting portion 38 is completely pushed toward the connection portion between the first pipe 32 and the second pipe 34, and the first pipe 32 and the second pipe 34 are fixed. As shown in FIG. 8, the captured image PTn shows the connector 30 in a state in which the fitting portion 38 is not completely pushed in and protrudes from the main body 36. In the captured image PTn, a protruding portion P1 of the fitting portion 38 protruding from the main body 36 and a cutout portion P2 at a position different from that of the connector 30 in a normal state are captured. In the following description, an example will be described in which the inspection device 60 determines that the fitting portion 38 of the connector 30 is not fully pushed in as an abnormality.

[0029] In step S20, the inspection unit 622 inputs the randomly sampled first noise Z1 and second noise Z2 to the generator 84 of the machine learning model 80 to generate a comparison image PR. The first noise Z1 is input to the generator 84 via the weight adjustment layer 82 and nonlinearly converted to a third noise W1 as a latent space via the mapping network layer 842 shown in FIG. 4. The synthesis network layer 844 generates an image from a fixed value CV, and while incorporating the third noise W1 as style information and the second noise Z2 as noise information, repeats upsampling of the image to generate a comparison image PR of 1024 pixels x 1024 pixels.

[0030] FIG. 9 is an explanatory diagram showing a comparison image PRg generated using a captured image PTg as a normal image. FIG. 10 is an explanatory diagram showing a comparison image PRn generated using a captured image PTn as an abnormal image. As shown in FIG. 9, it can be seen that the comparison image PRg is generated in a state similar to the normal image. The comparison image PRn shown in FIG. 10 is an example of a comparison image generated when re-determination is performed until the upper limit number of times is reached. As shown in FIG. 10, it can be seen that the comparison image PRn has a generation part P3 generated so that the protruding part P1 in the captured image PTn shown in FIG. 8 is similar to the normal image, and a generation part P4 generated so that the notch part P2 is similar to the normal image.

[0031] In step S30, the inspection unit 622 compares the captured image PT with the generated comparison image PR. In this embodiment, the inspection unit 622 performs the comparison using an integrated value of the difference between the sum of the RGB gradation values ​​of the pixels of the captured image PT and the sum of the RGB gradation values ​​of the pixels of the comparison image PR at positions corresponding to the pixels of the captured image PT. The pixels of the captured image PT are also called "first pixels," and the pixels of the comparison image PR are also called "second pixels."

[0032] Fig. 11 is an explanatory diagram showing a difference image CP generated using the difference in RGB gradation values ​​between the captured image PTn and the comparison image PRn. From Fig. 11, it can be seen that the difference image CP has, as areas with large differences in RGB gradation values, for example, a difference portion P5 which is the difference between the protruding portion P1 and the generated portion P3, and a difference portion P6 which is the difference between the notched portion P2 and the generated portion P4.

[0033] FIG. 12 is an explanatory diagram that shows a schematic method of calculating the difference between the captured image PTn and the comparative image PRn. In FIG. 12, in order to facilitate understanding of the technique, the number of pixels is shown in a state in which it is smaller than the number of pixels of the actual image. Table TB1 shown in FIG. 12 conceptually shows the RGB gradation value of each pixel included in the captured image PTn. Table TB2 conceptually shows the RGB gradation value of each pixel included in the comparative image PRn, and Table TB3 conceptually shows the RGB gradation value of each pixel included in the difference image CP. Each value shown in Table TB1 indicates the sum of the RGB gradation value of each pixel of the captured image PTn. Similarly, each value shown in Table TB2 indicates the sum of the RGB gradation value of each pixel of the comparative image PRn. Each value shown in Table TB3 indicates the difference value of the sum of the RGB gradation value for each pixel between Table TB1 and Table TB2.

[0034] As shown in Table TB3, the inspection unit 622 calculates the difference between the sum of the RGB gradation values ​​of the captured image PTn and the sum of the RGB gradation values ​​of the generated comparison image PRn for each pixel. If the captured image PT is an abnormal image, the comparison image is regenerated by the generator 84 so as to approximate the captured image PTn as an abnormal image by repeating re-determination. Therefore, as shown in Table TB3, for example, the difference between the captured image and the comparison image remains as a difference in RGB gradation values. The inspection unit 622 accumulates the calculated RGB gradation values ​​for each pixel for all pixels. In the example of Table TB3 in FIG. 12, the accumulated value of the RGB gradation values ​​is 11.

[0035] Returning to Fig. 5, in step S40, the inspection unit 622 judges whether or not there is an abnormality in the inspection object by using the calculated integrated value of the RGB gradation values ​​of all pixels. In this embodiment, the inspection unit 622 judges whether or not there is an abnormality in the inspection object by comparing the calculated integrated value of the RGB gradation values ​​of all pixels with a predetermined threshold value stored in the storage device 64. If the integrated value is lower than the threshold value (S40: YES), the process proceeds to step S42, where the inspection unit 622 judges that there is no abnormality in the inspection object and ends the process. If the integrated value is equal to or greater than the threshold value (S40: NO), the process proceeds to step S50.

[0036] In step S50, the inspection unit 622 judges whether the number of inspections N has reached a predetermined upper limit. The number of inspections corresponds to the number of re-determinations plus 1. The upper limit of the number of inspections N can be set arbitrarily based on the takt time required for the inspection device 60, and is set to 100 in this embodiment. If the number of inspections N has reached 100 (S50: YES), the process proceeds to step S52, where the inspection unit 622 judges that there is an abnormality in the inspection object and ends the process. The case where the number of inspections N has reached 100 corresponds to the case where it is judged that none of the 100 comparison images PR generated by changing the first noise Z1 and the second noise Z2 is similar to the captured image PT. If the number of inspections N is less than 100 (S50: NO), the process proceeds to step S60.

[0037] In step S60, the inspection unit 622 judges whether the number of inspections N has reached a predetermined number of switchings. The number of switchings is a threshold value used to judge whether to switch the combination method of the first noise Z1 and the second noise Z2. The number of switchings can be set arbitrarily, and in this embodiment, the number of switchings is set to 50, which is half the upper limit. If the number of inspections N is less than 50 (S60: YES), the process proceeds to step S62, where the inspection unit 622 fixes the second noise Z2 to the value input in step S20, and newly determines the first noise Z1 by random sampling. If the number of inspections N is 50 or more (S60: NO), the process proceeds to step S64, where the inspection unit 622 fixes the first noise Z1 to the value at the time when the number of inspections N is 50, and newly determines the second noise Z2 by random sampling. Among the re-determinations, the re-determination using a comparison image PR newly generated by fixing the second noise Z2 and varying the first noise Z1 is also referred to as a “first re-determination,” and the re-determination using a comparison image PR newly generated by fixing the first noise Z1 and varying the second noise Z2 is also referred to as a “second re-determination.” When the first noise Z1 and the second noise Z2 are newly determined in step S62 or step S64, the inspection unit 622 increments the inspection count N by 1 and returns to step S20.

[0038] As described above, the inspection device 60 of this embodiment includes a captured image acquisition unit that acquires a captured image PT of an inspection target, a memory device 64 that stores a machine learning model 80 that has been trained using a normal image that indicates a normal inspection target and is used to generate a comparison image PR that approximates the normal image, and an inspection unit 622 that determines whether or not an abnormality exists in the inspection target using the captured image PT and the comparison image PR. The machine learning model 80 includes a generator 84 that generates a comparison image PR by inputting a first noise Z1 and a second noise Z2. The generator 84 includes a mapping network layer 842 that performs nonlinear conversion on the first noise Z1 to generate a third noise W1, and a synthesis network layer 844 that generates an image using the second noise Z2 and the generated third noise W1, and generates a comparison image PR by repeating upsampling of the generated image multiple times. The generator 84 of the machine learning model 80 includes a mapping network layer 842 that uses StyleGAN to nonlinearly convert the input first noise Z1 to generate a third noise W1, and a synthesis network layer 844 that generates an image using the second noise Z2 and the third noise W1 and generates a comparison image PR by repeating upsampling multiple times to increase the resolution of the generated image. Therefore, in the inspection device 60 that uses the machine learning model 80, by adopting the network structure of StyleGAN for the generator 84, it is possible to determine the presence or absence of an abnormality using high-resolution image data.

[0039] According to the inspection device 60 of this embodiment, the machine learning model 80 does not include a classifier 86 for distinguishing between the captured image PT and the comparison image PR. Therefore, the captured image PT and the comparison image PR can be compared with each other with a simple configuration, and the productivity of the inspection device 60 can be increased.

[0040] According to the inspection device 60 of this embodiment, when the inspection unit 622 determines that there is no abnormality in the inspection target, the inspection unit 622 repeats a predetermined number of times a re-determination of the presence or absence of an abnormality in the inspection target using a new comparison image PR generated by changing at least one of the first noise Z1 and the second noise Z2. By changing the characteristics of the comparison image PR multiple times, it is possible to suppress overlooking an abnormality and improve the inspection accuracy.

[0041] According to the inspection device 60 of this embodiment, the re-determination performed by the inspection unit 622 includes a first re-determination in which the first noise Z1 is fixed, the second noise Z2 is changed, and a re-determination is performed a predetermined number of times using the comparison image PR generated, and a second re-determination in which the second noise Z2 is fixed, and the first noise Z1 is changed, and a re-determination is performed up to a predetermined upper limit number of times after the first re-determination. By changing only one of the first noise Z1 and the second noise Z2 to generate a new comparison image PR and performing the re-determination, it is possible to suppress a large deviation between the characteristics of the captured image PT and the characteristics of the comparison image PR at the time of the re-determination. In addition, by performing the first re-determination in which the first noise Z1 that changes the large features in the image is changed first, and then performing the second re-determination in which the second noise Z2 that changes the detailed features is changed, the possibility of determining that there is no abnormality within the number of switching times is increased, and the abnormality determination can be terminated early, thereby increasing the productivity of the inspection process.

[0042] According to the inspection device 60 of this embodiment, the generator 84 further includes a weight adjustment layer 82 that changes the first noise Z1 input to the mapping network layer 842 so as to reduce the difference between the captured image PT and the comparison image PR. Therefore, during the inspection process by the inspection device 60, a comparison image PR that approximates the captured image PT can be generated by a simple method of changing the first noise Z1 without training the generator 84 during the inspection.

[0043] According to the inspection device 60 of this embodiment, the inspection unit 622 calculates the difference between the RGB gradation value of a first pixel included in the captured image PT and the RGB gradation value of a second pixel at a position corresponding to the first pixel in the comparison image PR, and determines that there is an abnormality in the inspection object when the integrated value of the calculated RGB gradation value differences is greater than a predetermined threshold value. Therefore, compared to the case where the difference value is calculated using the mean squared error (MSE) of the RGB gradation values ​​at each pixel between the captured image PT and the comparison image PR, for example, it is possible to prevent the difference value of the RGB gradation values ​​that are easily recognized by humans from becoming excessively large, and to determine an abnormality in a state in which the change in color of the RGB gradation values ​​is properly taken into consideration, and it is possible to obtain inspection results that are approximately equivalent to those of an artificial inspection.

[0044] B. Second embodiment: FIG. 13 is an explanatory diagram showing a schematic configuration of a machine learning model 80b provided in the inspection device 60 according to the second embodiment. As shown in FIG. 13, the machine learning model 80b is a model that can be used as an anomaly detection method using Generative Adversarial Networks (GAN). The machine learning model 80b may also be called Anomaly Detection with Generative Adversarial Networks (AnoGAN). The machine learning model 80b differs from the machine learning model 80 shown in the first embodiment in that it further includes a discriminator 86 and a determiner 88. In this embodiment, the inspection unit 622 determines the presence or absence of an anomaly using the output result from the determiner 88.

[0045] The classifier 86 is in a state of being sufficiently trained to be able to distinguish between a normal image showing a normal inspection target and the generated comparison image PR. The classifier 86 outputs a likelihood indicating the degree of difference between the comparison image PR and the captured image PT. When the classifier 86 acquires the comparison image PR and the captured image PT acquired by the imaging unit 72, the classifier 86 compares the captured image PT with the comparison image PR and outputs the likelihood, which is a numerical value indicating the degree of difference between the comparison image PR and the captured image PT, to the determiner 88. In this embodiment, the likelihood is a value indicating the probability that the captured image PT has an abnormality, and for example, when the likelihood is a high value, it can be determined that the inspection target has an abnormality. In this embodiment, the classifier 86 calculates, for example, the sum of a first difference value, which is the difference between the captured image PT and the comparison image PR, and a second difference value, which is the difference between the output value of the intermediate layer when the comparison image PR is input to the classifier 86 and the output value of the intermediate layer when the captured image PT is input to the classifier 86, and outputs the result as the likelihood. However, when the likelihood indicates a probability value of a normal product, a low value may be detected as an abnormality.

[0046] The determiner 88 determines whether or not the inspection target has an abnormality by using the likelihood output from the discriminator 86. In step S40, if the likelihood is lower than a predetermined threshold, the determiner 88 may determine that the captured image PT and the comparison image PR are similar, and proceed to step S42, and if the likelihood is equal to or higher than the threshold, may determine that the captured image PT and the comparison image PR are not similar, and proceed to step S50. The determination result of the abnormality by the determiner 88 of the machine learning model 80b is output to the inspection unit 622, and the inspection unit 622 determines whether or not the inspection target has an abnormality based on the result of the determiner 88. In the machine learning model 80b, the generator 84 and the discriminator 86 do not need to perform learning while the inspection is being performed.

[0047] According to the inspection device 60 of this embodiment, the machine learning model 80b is a generative adversarial network further including a classifier 86 that outputs a numerical value indicating the degree of difference between the captured image PT and the comparison image PR. Therefore, by utilizing a known configuration, it is possible to determine the presence or absence of an abnormality using the captured image PT and the comparison image PR.

[0048] C. Other embodiments: (C1) In the above first embodiment, the inspection object is described as an example of a connector as an automobile part WK. In contrast, the inspection object is not limited to a connector used in an automobile, but may be various parts included in an automobile, such as the body or parts of the automobile. The inspection object is not limited to only automobile parts, but may be parts used in various moving bodies such as railroad cars, aircraft, and ships, or may be various manufactured products other than moving bodies.

[0049] (C2) In the above first embodiment, the inspection unit 622 may calculate the difference between the sum of the RGB gradation values ​​of the captured image PTn and the sum of the RGB gradation values ​​of the generated comparison image PRn for each pixel. In contrast, the inspection unit 622 may compare the difference between the sum of the RGB gradation values ​​of the captured image PTn and the sum of the RGB gradation values ​​of the generated comparison image PRn with a predetermined threshold, integrate the number of pixels where the difference is greater than or equal to the threshold, and determine that there is an abnormality when the number of pixels is greater than or equal to a predetermined value, and determine that there is no abnormality when the number of pixels is less than the predetermined value. In addition, the inspection unit 622 may calculate the difference between each of the R, G, and B gradation values ​​of the RGB gradation values ​​for each pixel between the captured image PT and the comparison image PR, and then use a value obtained by integrating the calculated differences between each of the R, G, and B gradation values. The inspection unit 622 may calculate the likelihood by using the mean squared error (MSE) of the RGB gradation values ​​for each pixel between the captured image PT and the comparison image PR.

[0050] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention column can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0051] 30...connector, 32...first pipe, 34...second pipe, 36...main body, 38...fitting portion, 60...inspection device, 61...bus, 62...CPU, 64...storage device, 66...communication unit, 70...camera, 72...imaging unit, 74...communication unit, 80, 80b...machine learning model, 82...weight adjustment layer, 84...generator, 86...classifier, 88...determinator, 100...inspection system, 622...inspection unit, 642...learning model storage unit, 842...mapping network layer, 844...synthesis network network layer, AL... normalization layer, AR... range, CP... difference image, CV... fixed value, Ln... production line, P1... protruding part, P2... cutout part, P3, P4... generation part, P5, P6... difference part, PC... information processing device, PD1, PD2... terminal device, PR, PRg, PRn... comparison image, PR1... previous process, PR2... subsequent process, PT, PTg, PTn... captured image, TB1 to TB3... table, W1... third noise, WK... part, Z1... first noise, Z2... second noise

Claims

1. An inspection device comprising: an image acquisition unit that acquires an image of an object to be inspected; A storage device that stores a learning model that has been trained using a normal image that shows a normal inspection target, and is used to generate a comparison image that is similar to the normal image; an inspection unit that determines whether or not an abnormality exists in the inspection object by using the captured image and the comparison image, The learning model is A generator for generating the comparison image by inputting a first noise and a second noise, a mapping network layer that nonlinearly transforms the first noise to generate a third noise; a synthesis network layer that generates an image using the second noise and the generated third noise, and generates the comparison image by repeating upsampling of the generated image a number of times; When it is determined that the captured image and the comparison image are not similar to each other, the inspection unit repeats a re-determination of the presence or absence of an abnormality in the inspection object a predetermined number of times using a new comparison image generated by changing at least one of the first noise and the second noise. Inspection equipment.

2. 2. The inspection device according to claim 1, The learning model does not include a classifier for discriminating between the captured image and the comparison image. Inspection equipment.

3. 2. The inspection device according to claim 1, The re-determination is a first re-determination in which the second noise is fixed and the re-determination is performed a predetermined number of times using the comparison image generated by changing the first noise; and after performing the first re-determination, performing the re-determination a predetermined number of times using the comparison image generated by fixing the first noise and varying the second noise. Inspection equipment.

4. 2. The inspection device according to claim 1, the inspection unit determines that the inspection object is abnormal when an integrated value of a difference between a gradation value of a first pixel included in the captured image and a gradation value of a second pixel at a position corresponding to the first pixel in the comparison image is greater than a predetermined threshold value. Inspection equipment.

5. The inspection device according to claim 1, The learning model is The generator; and a classifier that outputs a likelihood indicating a degree of difference between the captured image and the comparison image; a judger that judges the presence or absence of an anomaly in the inspection target by using the likelihood output from the classifier, The inspection unit determines whether or not the inspection target has an abnormality based on the result of the determiner. Inspection equipment.

6. The inspection device according to claim 1, The generator further includes a weight adjustment layer that changes the first noise input to the mapping network layer so that a difference between the captured image and the comparison image is reduced. Inspection equipment.

7. 1. A testing method comprising: Acquire an image of the inspection object; generating a comparison image that is similar to the normal image using a learning model that has been trained using a normal image that shows a normal object; determining whether or not an abnormality exists in the inspection target using the captured image and the comparison image; The learning model is A generator for generating the comparison image by inputting a first noise and a second noise, a mapping network layer that nonlinearly transforms the first noise to generate a third noise; a synthesis network layer that generates an image using the second noise and the generated third noise, and generates the comparison image by repeating upsampling of the generated image a number of times; when it is determined that the captured image and the comparison image are not similar to each other, a re-determination is performed a predetermined number of times to determine the presence or absence of an abnormality in the inspection object using a new comparison image generated by changing at least one of the first noise and the second noise. Testing method.

Images