Inspection method and inspection device

Abstract

The invention provides an inspection method and an inspection device, which can restrain workload and cost for judging whether an inspection object is normal or abnormal. The inspection apparatus includes: an image generator storage unit in which an image generator constructed by machine learning is stored, the image generator generating an image complemented with a defect region from an image having a defect region in a part; an image storage unit that stores an image to be inspected; a defect image generation unit that generates, from the image to be examined, a defect image in which a region to which a window of a pre-specified shape and size is applied is damaged; a complemented image generation unit that generates a complemented image in which the region is complemented by inputting the defect image to the image generator; a difference acquisition unit that obtains a difference between the image to be inspected and the complemented image; and a determination unit that determines whether the region is normal or abnormal by comparing the difference with a predetermined reference.

Description

Technical Field

[0001] The present invention relates to an inspection technique for determining whether an inspection object is normal or abnormal based on an image captured of the inspection object, and more particularly to an inspection method and an inspection device. Background Art

[0002] When products manufactured in a factory are shipped, an outgoing inspection is performed to confirm whether there are any abnormalities in the products. In such an inspection, for example, an inspector visually inspects the captured images of the products to confirm whether the products contain defective parts or whether foreign substances are mixed in, and thus determines (good or bad determination) whether the products are normal or abnormal. However, if an inspector attempts to visually inspect a large number of product images manufactured in a factory, there will be problems such as overlooking defective products, high labor costs, and limited inspection speed.

[0003] Therefore, an attempt is made to perform machine learning by using a large number of images of products that have been determined to be normal or abnormal as teaching data, and thus use the constructed learned model as an identifier to perform the good or bad determination of the products from the product images of the inspection object.

[0004] [Prior Art Documents]

[0005] [Patent Documents]

[0006] [Non-Patent Document 1] "RUTILEA Develops Defective Product Image Generation Technology Using AI - Solving the Shortage of Teaching Images for Appearance Inspection -", [Online], May 11, 2023, RUTILEA Co., Ltd., [Searched on November 10, 2023], Internet <URL:https: / / rutilea.com / info / 2278 / >

[0007] [Non-Patent Document 2] "Learning of Generative Adversarial Networks (GANs)", [Online], The MathWorks, Inc., [Searched on November 13, 2023], Internet <URL:https: / / jp.mathworks.com / help / deeplearning / ug / train-generative-adversarial-network.html>

[0008] [Non-Patent Document 3] "Mechanism of Image Generation AI [Part 2] What Shape is the Paintbrush of AI? Understanding the 'Image Generator'", [Online], March 30, 2023, Gijutsu-Hyoron Co., Ltd., [Searched on November 13, 2023], Internet <URL:https: / / gihyo.jp / article / 2023 / 03 / how-ai-image-generator-work-02> Summary of the Invention

[0009] [Problems to be Solved by the Invention]

[0010] The abnormal forms of products are not uniform. Therefore, the accuracy of the recognizer using the learned model constructed by machine learning depends on the variation of the teaching data used in the machine learning, specifically, on the variation of the abnormal part to be recognized. To cope with diverse abnormal forms, it is necessary to use product images with more diverse abnormal forms including these as teaching data for machine learning. Generally, most of the products manufactured in factories are normal, and it is difficult to obtain product images with various abnormal forms. Although it is possible to produce products with various abnormal forms intentionally or use dedicated software to generate product images with abnormalities as described in Non-Patent Document 1, these tasks have problems of large workload and high cost.

[0011] Here, taking the inspection of products manufactured in factories as an example, similar problems also exist when determining whether an inspection object of a non-industrial product is normal or abnormal.

[0012] The problem to be solved by the present invention is to suppress the workload and cost of determining whether an inspection object is normal or abnormal.

[0013] [Technical Means for Solving the Problem]

[0014] An inspection method according to one aspect of the present invention completed to solve the above problems includes:

[0015] An image generator preparation process of preparing an image generator by machine learning, the image generator generating an image that fills in a defective area from an image having a defective area;

[0016] An inspection image preparation process of preparing an inspection object image;

[0017] A defective image generation process of specifying a window with a predetermined shape and size and generating a defective image in which the area to which the window is applied is defective from the inspection object image;

[0018] A completed image generation process of generating a completed image that fills in the area by inputting the defective image into the image generator;

[0019] A difference acquisition process of obtaining the difference between the inspection object image and the completed image; and

[0020] A determination process of determining whether the area is normal or abnormal by comparing the difference with a predetermined reference.

[0021] Another inspection device according to the present invention, which is completed to solve the above problems, includes:

[0022] An image generator storage unit that stores an image generator constructed by machine learning, and the image generator generates an image that completes the defective area from an image having a defective area in part;

[0023] An image storage unit that stores an inspection object image;

[0024] A defective image generation unit that generates a defective image in which an area where a window of a predetermined shape and size is applied is defective from the inspection object image;

[0025] A completed image generation unit that generates a completed image that completes the area by inputting the defective image into the image generator;

[0026] A difference acquisition unit that obtains the difference between the inspection object image and the completed image; and

[0027] A determination unit that determines whether the area is normal or abnormal by comparing the difference with a predetermined reference.

[0028] [Advantages of the Invention]

[0029] In the present invention, an image generator is prepared in advance by machine learning, and the image generator generates an image that completes the defective area from an image having a defective area in part. The image generator used in the present invention is called image generation AI, and it has a function of completing the defective area by inferring a normal image that should contain the defective area included in the image from the image around the defective area. The image generator is constructed by using images of normal objects (preferably inspection objects) for machine learning, and completes the images of normal objects (inspection objects) in the defective areas of the inspection object images.

[0030] In the present invention, a window of a predetermined shape and size is applied to an image of an inspection object (inspection object image) to make the internal area defective, and then the image generator completes the image of the defective area (defective area). As described above, since the image generator completes the image of the normal inspection object in the defective area, when the inspection object image is an image of a normal inspection object, there is no difference between the inspection object image and the completed image. On the other hand, when the area where the window is applied in the inspection object image contains an abnormality, by obtaining the difference between the inspection object image and the completed image (for example, the difference of each pixel value), a significant difference will appear. By the above predetermined reference, this significant difference that appears is extracted, thereby determining that the area where the window is applied contains an abnormality.

[0031] In the conventional method, since an identifier is used to identify whether an inspection object image contains an abnormality, in order to accurately identify the normality and abnormality of the inspection object, a large number of inspection object images including various forms of abnormalities are required as teaching data for machine learning. On the other hand, the present invention uses an image generator that fills in the defective area inside the window applied to the inspection object image with an image of a normal inspection object. Whether the inspection object image contains an abnormality is determined only by a simple process of comparing the difference between the pixel values of the inspection object image and the filled image with a predetermined reference, and this process does not require an identifier. The present invention only needs to use an image generator constructed by machine learning using images of normal objects. Therefore, compared with the case of using an identifier that identifies the normality and abnormality of the inspection object from the image of the inspection object as in the conventional method, the workload and cost can be reduced to construct the image generator and determine whether the inspection object is normal or abnormal. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 FIG. is a main structural diagram of an inspection system including an embodiment of the inspection device of the present invention.

[0033] Figure 2 FIG. is a diagram illustrating the structure of a generative adversarial network.

[0034] Figure 3 FIG. is a flowchart illustrating an embodiment of the inspection method of the present invention.

[0035] Figure 4 FIG. is a diagram illustrating the setting of a window in this embodiment.

[0036] Figure 5 FIG. is a diagram illustrating a defective image, a filled image, and a difference image in this embodiment.

[0037] Figure 6 FIG. is a preprocessed image in an inspection example of a storage battery.

[0038] Figure 7 FIG. are preprocessed images of a storage battery before and after a charge-discharge test.

[0039] Figure 8 FIG. are a preprocessed image, a defective image, a filled image, and a difference image in an inspection example of a storage battery.

[0040] Figure 9 FIG. are a preprocessed image, a defective image, a filled image, and a difference image in another inspection example of a storage battery.

[0041] DESCRIPTION OF REFERENCE NUMERALS

[0042] 1: Inspection system

[0043] 10: X-ray CT apparatus

[0044] 20: Control and processing device

[0045] 30: Storage unit

[0046] 31: Image generator storage unit

[0047] 32: Image storage unit

[0048] 40: Inspection software

[0049] 41: Imaging control unit

[0050] 42: Image preprocessing unit

[0051] 43: Window setting unit

[0052] 44: Defect image generation unit

[0053] 45: Completion image generation unit

[0054] 46: Difference image acquisition unit

[0055] 47: Determination unit

[0056] 48: Inspection result display processing unit

[0057] 51: Input unit

[0058] 52: Display unit

[0059] 61: Post-preprocessing image

[0060] 62: Window

[0061] 63: Defect image

[0062] 64: Completion image

[0063] 65: Difference image Detailed implementation manners

[0064] The implementation manners of the inspection method and inspection apparatus of the present invention will be described below with reference to the accompanying drawings.

[0065] <Structure of inspection system 1>

[0066] Figure 1 The main structure of the inspection system 1 including the inspection apparatus of the present embodiment is shown. The inspection system 1 of the present embodiment includes an X-ray CT apparatus 10 and a control and processing device 20. The control and processing device 20 corresponds to an embodiment of the inspection apparatus of the present invention.

[0067] The X-ray CT apparatus 10 irradiates an object to be inspected with X-rays and captures tomographic images of its interior. In addition, the X-ray CT apparatus 10 is capable of performing dual-energy imaging, that is, irradiating X-rays of multiple different energies and capturing tomographic images at each energy.

[0068] The control and processing device 20 includes a storage unit 30. The storage unit 30 is provided with an image generator storage unit 31 and an image storage unit 32. The image generator storage unit 31 stores an image generator constructed from a learned model created by machine learning. The image storage unit 32 stores tomographic images of the object to be inspected captured by the X-ray CT apparatus 10. In addition, reference information for determining whether the object to be inspected is normal or abnormal from the images of the object to be inspected (in this embodiment, the threshold of the standard deviation of the pixel values of each pixel of the difference image 65 (see Figure 5 described later)) is also stored in the storage unit 30. Such a reference may be determined based on values obtained by performing a series of processes described later on one or more images of normal objects to be inspected as inspection target images.

[0069] The control and processing device 20 further includes a shooting control unit 41, an image preprocessing unit 42, a window setting unit 43, a defective image generation unit 44, a completion image generation unit 45, a difference image acquisition unit 46, a determination unit 47, and an inspection result display processing unit 48 as functional modules. The control and processing device 20 may be constituted by, for example, an ordinary personal computer, and these functional modules are implemented by a processor executing a program of dedicated inspection software 40 installed in advance. In addition, the control and processing device 20 is connected to an input unit 51 constituted by a keyboard and a mouse, etc., and a display unit 52 constituted by a liquid crystal display, etc.

[0070] In this embodiment, before determining normality and abnormality based on the images of the object to be inspected, an image generator is first created and stored in the image generator storage unit 31. The image generator of this embodiment is constructed by a generative adversarial network (GAN: Generative Adversarial Networks), which is a method of unsupervised machine learning that can learn features without giving correct answer data. For example, as described in Non-Patent Document 2, the generative adversarial network itself is a known method, so only a brief description will be given here.

[0071] Figure 2Shows the structure of a generative adversarial network. A generative adversarial network has two neural networks. One is the Generator, which takes a random value (latent input) vector as input and generates data with the same structure as the learning data. The other is the Discriminator, which is given a data batch containing observations of both the learning data and the data generated by the Generator, and attempts to classify these observations to determine whether they are real or generated.

[0072] When learning a GAN, the performance of both networks is usually maximized by simultaneously learning the two networks. Specifically, for the Generator, by inputting noise equivalent to the seed of the generated data features, mapping this noise to be close to the desired data, and performing machine learning to generate data that is judged to be real by the Discriminator; for the Discriminator, machine learning is performed to identify the images generated by the Generator and the real images. In this machine learning, it is preferably to have the Generator generate images of the same type as the object to be inspected as the generated data. In this way, a Generator that has learned more features of the object to be inspected can be created. In addition, in a GAN, only an image generator that generates images of normal objects to be inspected needs to be constructed, so there is no need to perform machine learning on images of abnormal objects to be inspected as in traditional methods of creating a Discriminator. Further, in this embodiment, only the Generator generated by the GAN needs to be used as the image generator, and there is no need to perform machine learning on the Discriminator. Therefore, in this embodiment, the Discriminator with a certain recognition ability can be kept unchanged (without further machine learning), and only the Generator is subjected to machine learning through the GAN.

[0073] <Method of Inspection Using Inspection System 1>

[0074] Next, with reference to Figure 3 the flowchart, the steps of determining the normality and abnormality of the object to be inspected using the inspection device of this embodiment will be described.

[0075] The user first prepares the tomographic image (inspection object image) of the object to be inspected (step 1). In the case where the tomographic image has already been acquired, the tomographic image of the object to be inspected is read from the image storage unit 32. In the case where the tomographic image has not been acquired, the object to be inspected is placed at the designated position of the X-ray CT device 10, and an instruction to take a picture is given. When an instruction to take a tomographic image is given, the imaging control unit 41 irradiates X-rays of multiple different energies respectively and takes the respective tomographic images. The taken tomographic images are associated with information for identifying the object to be inspected (such as an ID number, etc.) and stored in the image storage unit 32.

[0076] Next, the image preprocessing unit 42 performs predetermined preprocessing on the tomographic image of the object to be inspected (step 2). The preprocessing of the image includes, for example, processing for normalizing the size of the tomographic image, processing for removing artifacts in the image, etc. The normalization of the size of the tomographic image refers to the processing of making the number of pixels in the horizontal and vertical directions of the captured image consistent with a predetermined number of pixels. This predetermined number of pixels is preferably equivalent (preferably the same) to the number of pixels of the image generated by the generator in the machine learning of the image generator used for construction. In this way, when generating a completion image (described later), it is possible to more easily reflect the features learned by the image generator. Furthermore, removing artifacts in the image refers to, for example, based on tomographic images obtained by irradiating X-rays with different energies, extracting only the part corresponding to the part to be inspected in the object to be inspected, and at the same time removing the artifacts unique to the images taken by the X-ray CT apparatus 10. For example, in the case of determining the normality and abnormality of a battery electrode, it is a process of distinguishing the electrode and other components from the above-mentioned plurality of tomographic images, identifying the electrode part or removing unnecessary parts. In addition, the preprocessing of the image is not an essential process, but by performing such preprocessing, it is possible to generate a higher-precision generated image by the generator and prevent misjudgment caused by image artifacts.

[0077] Then, the window setting unit 43 sets a window 62 of a specified size at the initial position (e.g., the upper left corner) of the preprocessed image (preprocessed image 61) (step 3. See Figure 4 upper left). The window 62 used in the present embodiment is rectangular. In the present embodiment, the lengths of both sides of the window 62 are 10% of the lengths of both sides of the preprocessed image.

[0078] After setting the window 62, the defect image generation unit 44 generates a defect image 63 that eliminates the pixel data of the part specified by the window 62 in the preprocessed image 61 (step 4. See Figure 5 ).

[0079] After generating the defect image 63, the completion image generation unit 45 uses the image generator stored in the image generator storage unit 31 to generate a completion image 64 that completes the image of the defect area of the defect image 63 (step 5. See Figure 5 ).

[0080] After generating the completion image 64, the difference image acquisition unit 46 creates a difference image 65 that is the difference between the preprocessed image 61 and the completion image 64 (step 6. See Figure 5 ).

[0081] After creating the differential image 65, the determination unit 47 calculates the standard deviation of the differential image 65 (step 7). Specifically, the differences in the pixel values of each pixel within the region of the set window in the differential image 65 are obtained, and the standard deviation thereof is calculated. The determination unit 47 compares the calculated standard deviation value with the threshold value stored in the storage unit 30 (step 8). Then, if the value of the standard deviation is less than or equal to the threshold value (step 8 is NO), it is determined that the region of the application window 62 is normal (step 9), and if the value of the standard deviation exceeds the threshold value (step 8 is YES), it is determined that the region of the application window 62 contains an abnormality (step 10).

[0082] After completing the determination of whether the region of the application window 62 is normal or abnormal, the window setting unit 43 determines whether windows 62 have been set at all positions of the preprocessed image 61 (step 11). Here, since the window 62 is only set at the initial position, it is determined that there are still positions where the window 62 has not been set.

[0083] When it is determined that there are still positions where the window 62 has not been set, the window setting unit 43 resets the window 62 at a position shifted from the previously set position of the window 62 (here, the initial position) in a predetermined direction and by a predetermined number of pixels (for example, 1 to several pixels in the right direction) (step 12). As Figure 4 shown, the window setting unit 43 successively repeats the setting of the window 62 on the preprocessed image 61 using, for example, a method called a sliding window used in the field of object detection and the like. In addition, this embodiment is an embodiment for a case where the abnormal part is not clear. If the abnormal part has been anticipated, the window may not be moved, but the user may specify the anticipated area, and only the specified area may be processed.

[0084] After resetting the window 62, the process returns to step 4, and the same processes as steps 4 to 11 described above are executed again. Then, in step 11, after determining that the windows 62 have been set at all positions, the inspection result display processing unit 48 overlaps the positions of the windows 62 determined to contain abnormalities on the image of the preprocessed image 61 and displays the result as an inspection result on the display unit 52 (step 13).

[0085] <Example 1>

[0086] Here, as a specific example, the results of inspecting the electrodes of a storage battery are described. In this example, images after repeatedly performing charge and discharge tests on a cylindrical lithium-ion battery (a battery in which a sheet formed by stacking a positive electrode, a separator, and a negative electrode is wound and housed inside a cylindrical container. 18650-type LIB) are used to check whether there are any abnormalities in the electrodes. Here too, as in the above-described embodiment, a sliding window is used to sequentially set windows on the preprocessed image 61, and it is determined whether there are any abnormalities at each window position through the processing of steps 4 to 10 above.

[0087] Figure 6 This is the preprocessed image 61. In this example, a preprocessed image 61 of 700×700 pixels is used, and a square window of 70×70 pixels is applied. Figure 7 On the left is a partially enlarged view of the tomographic image before the test (Image A), Figure 7 and on the right is a partially enlarged view of the tomographic image after the test (Image B). An electrode abnormality (a state of being wrinkled and bent) appears at the position marked by a circle in the tomographic image B. In addition, in this example, the abnormality of the electrode appears as a wrinkle, but the abnormality of the electrode may appear in various forms.

[0088] Figure 8 The preprocessed image 61, the defect image 63, the completed image 64, and the difference image 65 in one window where an abnormality is found are shown. As described above, the image generator of the present embodiment is constructed by machine learning that generates an image that completes the defective region from an image having a defective region in part. Therefore, when a region including the abnormal part (the region where the window is set) is made defective and a completed image 64 that completes the defective region with the image generator is generated, the defective region is completed into an image of a normal electrode. Therefore, when the region where the window is applied is normal, there is no significant difference between the preprocessed image 61 and the completed image 64. On the other hand, when the region where the window is applied includes an abnormality, as Figure 8 shown in the partially enlarged view, there is a significant difference between the preprocessed image 61 and the completed image 64, and fluctuations in pixel values occur in each pixel of these difference images. Therefore, by calculating the standard deviation representing the fluctuations in the pixel values of each pixel of the difference image and comparing it with a predetermined threshold value, it is possible to determine whether the region includes an abnormality.

[0089] <Example 2>

[0090] Figure 9This is a result example of checking for the presence or absence of electrode abnormalities before and after repeatedly charging and discharging a cylindrical lithium-ion battery, which is the same as in Example 1. Example 1 uses the inspection result of the tomographic image of the horizontal cross-section (the cross-section perpendicular to the direction in which the electrodes extend) of the lithium-ion battery, while Example 2 uses the inspection result of the tomographic image of the vertical cross-section (the cross-section parallel to the direction in which the electrodes extend and including the central axis of the wound electrodes) of the lithium-ion battery. In Example 2 here, the positions where electrode abnormalities occur also clearly appear in the difference image, from which it is known that the pixel values of each pixel fluctuate (the standard deviation becomes larger).

[0091] Thus, in the present embodiment and examples, a complementary image is generated by an image generator of machine learning that generates an image of a normal object (preferably the object to be inspected), and the subsequent determination of normal and abnormal is performed by a simple process of obtaining the standard deviation of the difference image between the pre-processed image (original image) and the complementary image. Therefore, there is no need to construct an identifier for identifying the normality and abnormality of the object to be inspected by machine learning as in the conventional method, nor is it necessary to learn a large number of images of the object to be inspected that contain abnormalities appearing in various forms. Compared with the conventional method, the workload and cost of determining the normality and abnormality of the object to be inspected can be suppressed.

[0092] The above-described embodiment and examples are only illustrative, and appropriate changes can be made according to the gist of the present invention.

[0093] In the above embodiment, a tomographic image of an object to be inspected (such as a storage battery) is obtained using the X-ray CT apparatus 10, and the normality and abnormality of the storage battery electrodes are determined based on the tomographic image. However, the type of the object image to be inspected is arbitrary, and appropriate images can be used according to the purpose of the inspection. For example, when performing an appearance inspection, an image of the appearance of the object to be inspected can be used. In addition, the object to be inspected is not limited to objects such as storage batteries. For example, various images such as an image of a human tissue (organ) taken using the X-ray CT apparatus 10 or an image showing the distribution of a target compound inside a specimen obtained by an imaging mass spectrometer can be used as the object image to be inspected.

[0094] In the above embodiment, an example of performing dual-energy imaging to obtain multiple images in order to remove image artifacts is described. However, in addition to this, imaging methods such as photon-counting CT and phase imaging can be adopted according to the characteristics of the object to be inspected to obtain images for distinguishing the composition of the object to be inspected and determining the inspected part.

[0095] In the above-described embodiments and examples, it is configured to determine the normality and abnormality of the region where the window is set by comparing the standard deviation obtained by calculating the difference image of the region of the set window with a threshold value. However, the determination of the normality and abnormality of the region can be made based on a quantity (feature quantity) representing the characteristics of the difference image of the region, or a quantity other than the standard deviation can be used. For example, the average value, variance value, eigenvalue, representative value of the norm, etc. of the pixel values of each pixel in the difference image of the region of the set window can be used. Further, in the above-described embodiments and examples, the normality and abnormality of the inspection object are determined based on the monochromatic image of the inspection object, but a color image can also be used. In the case of using a color image, the RGB value can be used as the pixel value.

[0096] In the above-described embodiments and examples, it has been described that the image generator generates a difference image for the defective image using only one completed image. However, an image generator constructed by machine learning (so-called generative AI) will, in most cases, generate multiple images and their accuracies, and present the one with the highest accuracy as the generated image. In the above-described embodiment, only one completed image presented by the image generator is used to generate the difference image, but the image generator's judgment of accuracy is not always correct. For example, in the case of generating the completed image 64 from the defective image 63 in which the edge portion of the pre-processed image 61 or the image of the inspection object with low periodicity is defective, the accuracy of the completed image 64 generated by the image generator is low, and thus the accuracy is also low. In this case, difference images 65 can be created for all the completed images 64 generated by the image generator (or a predetermined number of completed images 64 sorted by accuracy), and the normality and abnormality can be determined based on the feature quantity (such as the standard deviation in the above-described embodiments and examples) calculated from these multiple difference images 65. In this way, even in the case where it is difficult for the image generator to generate a highly accurate completed image, an abnormality can be detected more reliably.

[0097] In the above-described embodiment, only one size of window is used, but multiple windows of different sizes can also be used. For example, when the abnormal portion included in the inspection object is small, if a large window is used for the abnormal portion, the pixel values indicating abnormality will appear only in a very small number of pixels within the window, and there will be no significant difference in other pixels, and the value of the standard deviation may not exceed the threshold. In this case, by using multiple windows of different sizes, it is possible to detect abnormal portions of any size using a window of a size suitable for the size of the abnormal portion.

[0098] In the above-described embodiment, the window is set at multiple different positions and a part of the windows overlap. However, if the position where the abnormality appears in the inspection object is known in advance, the window can be set only at that position.

[0099] In the above-described embodiments, an image generator created by a generative adversarial network is used, but the image generator can also be created by other methods, and an image generator created by these methods can also be used. For example, as described in Non-Patent Document 3, an image generator created by a diffusion model, VAE (Variational AutoEncoder), flow-based model, autoregressive model, etc. can also be used.

[0100] [Mode]

[0101] It is obvious to those skilled in the art that the above-described exemplary embodiments are specific examples of the following modes.

[0102] (Item 1)

[0103] An inspection method according to one mode of the present invention includes:

[0104] An image generator preparation step of preparing an image generator by machine learning, the image generator generating an image that fills in a defective area from an image having a defective area in a part;

[0105] An inspection image preparation step of preparing an inspection target image;

[0106] A defective image generation step of specifying a window having a predetermined shape and size and generating a defective image in which the area to which the window is applied is defective from the inspection target image;

[0107] A filled image generation step of generating a filled image that fills in the area by inputting the defective image into the image generator;

[0108] A difference acquisition step of obtaining a difference between the inspection target image and the filled image; and

[0109] A determination step of determining whether the area is normal or abnormal by comparing the difference with a predetermined reference.

[0110] (Item 2)

[0111] The inspection device according to Item 2 includes:

[0112] An image generator storage unit that stores an image generator constructed by machine learning, the image generator generating an image that fills in a defective area from an image having a defective area in a part;

[0113] An image storage unit that stores an inspection target image;

[0114] A defective image generation unit that generates a defective image in which the area to which a window having a predetermined shape and size is applied is defective from the inspection target image;

[0115] The completed image generation unit generates a completed image that fills in the region by inputting the defective image into the image generator;

[0116] The difference acquisition unit calculates the difference between the inspection target image and the completed image; and

[0117] The determination unit determines whether the region is normal or abnormal by comparing the difference with a predetermined reference.

[0118] (Item 3)

[0119] The inspection device according to Item 3 is the inspection device according to Item 2,

[0120] wherein the image generator is composed of a generator used together with a recognizer in adversarial learning.

[0121] In the inspection method according to Item 1 and the inspection device according to Item 2, an image generator is prepared in advance by machine learning, and the image generator can generate an image that fills in the defective region from an image having a defective region in part. As such an image generator, for example, an image generator composed of a generator used together with a recognizer in adversarial learning as used in the inspection device according to Item 3 can be used. The image generator used in the inspection method according to Item 1 and the inspection device according to Item 2 is called image generation AI, and it has a function of filling in the defective region by inferring a normal image including the defective region that should be included in the image from the image around the defective region. The image generator is constructed by machine learning using images of normal objects (preferably inspection target objects), and fills in the image of a normal object (inspection target object) in the defective region of the inspection target image.

[0122] In the inspection method according to Item 1 and the inspection device according to Item 2, a window having a predetermined shape and size is applied to the image of the inspection target object (inspection target image) to make the internal region defective, and then the image generator fills in the image of the defective region (defective region). As described above, since the image generator fills in the image of a normal inspection target object in the defective region, there is no difference between the inspection target image and the completed image when the inspection target image is an image of a normal inspection target object. On the other hand, when the region where the window is applied in the inspection target image contains an abnormality, a significant difference will occur by calculating the difference between the inspection target image and the completed image (for example, the difference in each pixel value). By the above-mentioned predetermined reference, such a significant difference that appears is extracted, and thus it is determined that the region where the window is applied contains an abnormality.

[0123] In the conventional method, since an identifier is used to identify whether an inspection object image contains an abnormality, in order to accurately identify the normality and abnormality of the inspection object, a large number of inspection object images containing various forms of abnormalities are required as teaching data for machine learning. On the other hand, in the inspection method of Item 1 and the inspection apparatus of Item 2, an image generator is used, and the image generator fills in the defective area inside the window applied to the inspection object image with an image of a normal inspection object. Whether the inspection object image contains an abnormality is determined only by a simple process of comparing the difference between the pixel values of the inspection object image and the filled image with a predetermined reference, and this process does not require an identifier. In the inspection method of Item 1 and the inspection apparatus of Item 2, only an image generator constructed by machine learning using an image of a normal inspection object needs to be used. Therefore, compared with the case of using an identifier that identifies the normality and abnormality of the inspection object from the image of the inspection object as in the conventional method, the workload and cost can be suppressed to construct an image generator, and it can be determined whether the inspection object is normal or abnormal.

[0124] (Item 4)

[0125] The inspection apparatus of Item 4 is an inspection apparatus according to any one of Items 2 or 3,

[0126] wherein the windows are respectively set at a plurality of different positions of the inspection object image such that a part of the windows overlaps with each other.

[0127] In the inspection apparatus of Item 4, regardless of which part of the inspection object image contains an abnormality, the abnormality can be detected based on the difference obtained when the window is applied at the position.

[0128] (Item 5)

[0129] The inspection apparatus of Item 5 is an inspection apparatus according to any one of Items 2 to 4,

[0130] wherein the filled image generation unit generates a plurality of filled images for one defective image,

[0131] and the difference obtaining unit obtains the differences between the inspection object image and each of the plurality of filled images.

[0132] For example, in the case of generating a filled image from a defective image in which an edge portion of the inspection object image or an image of an inspection object with low periodicity is defective, the accuracy of the image generated by the image generator may be low. In the inspection apparatus of Item 5, since the differences are respectively obtained for the plurality of filled images generated by the image generator, even in the case where it is difficult for the image generator to generate a highly accurate filled image, the abnormality can be detected more reliably.

Claims

1. An inspection method, characterized in that: include: An image generator preparation step, preparing an image generator through machine learning, wherein the image generator generates an image that fills the defective area from an image having a defective area; Inspection image preparation process, preparing inspection object images; a defect image generating step of designating a window of a predetermined shape and size and generating a defect image in which a region to which the window applies is defective from the inspection object image; a completed image generating step of generating a completed image that completes the region by inputting the defective image into the image generator; A difference acquisition step of obtaining a difference between the inspection object image and the complemented image; as well as The determination step determines whether the area is normal or abnormal by comparing the difference with a predetermined reference.

2. An inspection device, characterized in that: include: An image generator storage unit stores an image generator constructed by machine learning, wherein the image generator generates an image that fills the defective area from an image having a defective area; An image storage unit storing an image of an inspection object; a defect image generating unit that generates a defect image in which a region to which a window of a predetermined shape and size is applied is defective from the inspection object image; a complement image generating unit, which generates a complement image that complements the region by inputting the defective image into the image generator; a difference acquisition unit for obtaining a difference between the inspection object image and the complemented image; as well as The determination unit determines whether the region is normal or abnormal by comparing the difference with a predetermined reference.

3. The inspection device according to claim 2, characterized in that: The image generator consists of a generator used with a recognizer in adversarial learning.

4. The inspection device according to claim 2, characterized in that: The windows are respectively set at a plurality of different positions of the inspection object image so that parts of the windows overlap with each other.

5. The inspection device according to claim 2, characterized in that: The complement image generating unit generates a plurality of complement images for one defective image. The difference acquisition unit obtains a difference between the inspection target image and each of the plurality of complementation images.

Images