Appearance inspection system and appearance inspection method

Abstract

Provided is a visual inspection system capable of generating a learned model using data for improving the learned model among data obtained by visual inspection and inferring a defect.SOLUTION: The appearance inspection apparatus 10 includes the annotation unit 5 that associates the annotation information with the input image 2, the annotation information indicating whether the defect of the structure is normal or not, the annotation information being determined by the user based on the input image 2 used for the appearance inspection of the structure, the learning unit 8 that generates the learned model by deep learning using the annotation information and the input image 2 with which the annotation information is associated with respect to the initially set learned model, and the learning unit 8 that, using the learned model generated by the learning unit 8, and a defect inference unit 3 configured to output an inference result obtained by inferring a defect based on the input image 2, in which the annotation unit 5 associates annotation information with an input image for which the inference result is different from a result determined by a user for the input image, and causes the learning unit to relearn the learned model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an appearance inspection system and an appearance inspection method. [Background technology]

[0002] Conventionally, inspections of the inside of nuclear power plants have been performed by visual inspection, where inspectors visually check images captured by cameras to determine whether or not there are any flaws. Inside the reactor, darkness and noise on the screen caused by radiation make it difficult to distinguish images, placing a heavy burden on inspectors. Therefore, machine learning is used to detect and identify defects, and the location of flaws is displayed to inspectors, reducing the burden on them.

[0003] Patent Document 1 states that "the machine learning device provided in the visual inspection device comprises a state observation unit that observes state variables including normal product image data showing an image of a normal product and comparison image data showing an image of a product to be compared with the normal product; a label data acquisition unit that acquires label data including classification data showing the label of the image of the product to be compared (normal product or defective product); and a learning unit that uses the state variable S and the label data to learn to classify the differences between the image of the normal product and the image of the product to be compared (normal product or defective product)." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-095217 Summary of the Invention [Problem to be solved by the invention]

[0005] Typically, training a neural network to classify objects in images requires a huge number of images and the labor required to label all of those images for defects. Furthermore, the shapes of the structures to be inspected in nuclear power plants vary widely. Therefore, detecting suspected defects requires training for each inspection object, which takes a long time. Furthermore, defect identification is required to determine whether a detected suspected defect is an actual defect or a pattern such as a weld mark. Defect identification requires separate learning of defect shapes, which further increases the training time.

[0006] One advantage of using machine learning is that it allows for progressive learning. Progressive learning is possible by using flaw detection images obtained through actual flaw detection as training data for machine learning. However, incorporating all of the obtained data into the training data not only increases the learning time, but also raises the risk of over-learning, which could lead to a decrease in defect detection accuracy.

[0007] The present invention has been made in consideration of these circumstances, and aims to generate a trained model using data obtained from visual inspection that improves the trained model. [Means for solving the problem]

[0008] The visual inspection system of the present invention includes an annotation unit that links annotation information indicating whether a defect in a structure is normal or not, as determined by a user based on an input image used in the visual inspection of the structure, to the input image; a learning unit that generates a learned model by deep learning using the annotation information and the input image to which the annotation information is linked for an initially set learned model; and a defect inference unit that outputs an inference result of inferring a defect based on the input image using the learned model generated by the learning unit, and the annotation unit links the annotation information to an input image for which the inference result differs from the result determined by the user for the input image, and causes the learning unit to re-learn the learned model. [Effects of the Invention]

[0009] According to the present invention, a trained model can be generated using data obtained from visual inspection that improves the trained model. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the overall configuration of a visual inspection system according to a first embodiment of the present invention and an overview of processing; [Figure 2] 1 is a block diagram showing an example of the internal configuration of a visual inspection apparatus according to a first embodiment of the present invention. [Figure 3] FIG. 2 is a block diagram showing a detailed example of the internal configuration of a defect detection unit according to the first embodiment of the present invention. [Figure 4] 3 is a schematic diagram of an image restoration process and a difference determination process performed in the defect detection unit according to the first embodiment of the present invention. FIG. [Figure 5] FIG. 2 is a block diagram showing a detailed example of the internal configuration of a defect identification unit and an annotation unit according to the first embodiment of the present invention. [Figure 6] 3A to 3C are diagrams illustrating an example of a defect identification and inference process performed by an identification and inference unit according to the first embodiment of the present invention. [Figure 7] FIG. 4 is a diagram showing an example of an inspector determination process performed by an annotation unit according to the first embodiment of the present invention. [Figure 8] 5 is a flowchart showing an example of a visual inspection process performed by the visual inspection apparatus according to the first embodiment of the present invention. [Figure 9] 1 is a block diagram showing an example of the hardware configuration of a computer according to a first embodiment of the present invention. [Figure 10] FIG. 10 is a block diagram showing an example of the internal configuration of a visual inspection apparatus according to a second embodiment of the present invention. [Figure 11] FIG. 10 is a block diagram showing an example of the internal configuration of a defect detection unit according to a second embodiment of the present invention. [Figure 12]10A and 10B are schematic diagrams illustrating image restoration processing, difference determination processing, and coincidence calculation processing performed in a defect detection unit according to a second embodiment of the present invention. [Figure 13] FIG. 10 is a block diagram showing an example of a detailed internal configuration of a defect identification unit and an annotation unit according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] [First embodiment] FIG. 1 is a diagram showing an example of the overall configuration of a visual inspection system 100 according to the first embodiment and an overview of the processing.

[0013] The visual inspection system 100 includes an imaging unit 1 and a visual inspection device 10. The imaging unit 1 outputs image data of an image or video captured of the exterior of a structure of a nuclear power plant, for example. The image data is input to the visual inspection device 10 as an input image 2. The input image 2 may include flaw detection data including images of defects. The input image 2 may also be stored in a database not shown in FIG. 1 and read out as needed from the visual inspection device 10. The visual inspection device 10 inspects whether the input image 2 includes images of cracks in the structure, etc., and evaluates the input image 2.

[0014] In the visual inspection device 10, an AI judgment process (S1) is performed by the defect inference unit 3 using AI (Artificial Intelligence). The AI ​​judgment process (S1) is a process in which the defect inference unit 3 judges the presence or absence of a defect based on the input image 2. The defect inference unit 3 uses a trained model generated by the learning unit 8 to infer a defect based on the input image 2 and outputs an inference result.

[0015] The AI ​​that performs the AI ​​judgment process (S1) uses a trained model that has previously learned training data 7 through machine learning process (S3). For example, a different trained model is used for each structure that is the subject of visual inspection. Therefore, before the visual inspection, the inspector 9 selects, via the model selection unit 9, a trained model that has been trained for the structure that is the subject of visual inspection. The model selection unit 9 selects a defect detection model and a defect identification model to be used for the visual inspection for each structure.

[0016] When the AI ​​judgment process (S1) using the trained model selected by the model selection unit 9 is completed, the output result 4 is output as an example of an inference result. Examples of data for the output result 4 include the result of the defect inference unit 3 determining whether or not there is a defect, an image in which the defective portion is emphasized in the input image 2, etc.

[0017] Inspector 6 is an example of a user who performs a visual inspection of a structure using visual inspection device 10. Inspector 6 performs a defect presence / absence determination process (S2) to visually determine the presence or absence of a defect based on output result 4 and input image 2. Annotation unit 5 links annotation information indicating whether or not a defect in the structure is normal, determined by the user based on input image 2 used in the visual inspection of the structure, to input image 2. Therefore, defect presence / absence determination process (S2) is a process in which annotation unit 5 operated by inspector 6 links annotation information to input image 2. The determination result of inspector 6 in defect presence / absence determination process (S2) becomes the final determination result of the presence or absence of a defect.

[0018] A part of the results of the defect presence / absence determination process (S2) becomes training data 7 to be used for re-learning in the machine learning process (S3). Here, the input image 2 in which the output result 4 of the presence / absence of a defect in the input image 2 by the AI ​​determination process (S1) differs from the output result 4 of the presence / absence of a defect in the input image 2 determined by the inspector 6 by the defect presence / absence determination process (S2) is used as training data 7.

[0019] The learning unit 8 generates a trained model by deep learning using the annotation information and the input image 2 linked to the annotation information for the initially set trained model. Therefore, in the machine learning process (S3), the learning unit 8 performs re-learning using the teacher data 7 and saves the trained model. The learning unit 8 can improve the trained model by re-learning using the teacher data 7 including annotation information that improves the trained model. The defect inference unit 3 performs the AI ​​judgment process (S1) again using the trained model after re-learning.

[0020] In this way, the AI ​​judgment process (S1), the defect presence / absence judgment process (S2), and the machine learning process (S3) are repeatedly performed in the visual inspection device 10. As a result, the AI ​​judgment process (S1) can be performed using a trained model that has undergone progressive learning in the machine learning process (S3).

[0021] Furthermore, the annotation unit 5 associates annotation information with an input image where the inference result of the output result 4 differs from the result of the user's judgment of the input image, and causes the learning unit 8 to re-learn the learned model.

[0022] Note that when the visual inspection device 10 is initially set up, there is no trained model, so a trained model must be generated. To do this, an inspector 6 visually inspects the input image 2, and the annotation unit 5 links annotation information to the input image 2. The learning unit 8 performs machine learning processing (S3) based on the input image 2 linked with the annotation information to generate a trained model. The defect inference unit 3 then uses the generated trained model to perform AI judgment processing (S1).

[0023] Next, an example of the internal configuration and operation of the visual inspection apparatus 10 according to the first embodiment will be described with reference to FIGS. FIG. 2 is a block diagram showing an example of the internal configuration of the visual inspection device 10. As shown in FIG.

[0024] The visual inspection device 10 includes an input image acquisition unit 11, an image storage unit 12, a defect detection unit 30, a defect identification unit 40, an annotation unit 5, and a selection unit 60. As shown in the figure, the defect detection unit 30 performs a defect detection process (S11), the defect identification unit 40 performs a defect identification process (S12), the annotation unit 5 performs an inspector judgment process (S13), and the selection unit 60 performs a selection process (S14). The defect detection process (S11) and the defect identification process (S12) are performed by AI. The inspector judgment process (S13) and the selection process (S14) are performed by an inspector 6.

[0025] First, the input image acquisition unit 11 acquires the input image 2 from the imaging unit 1 shown in FIG. The image storage unit 12 is a database that stores a plurality of input images, and is an example of an input image storage unit. The image storage unit 12 stores the input images 2 acquired by the input image acquisition unit 11 so that they can be read by the defect detection unit 30. The image storage unit 12 classifies and stores the input images 2 for each major configuration of the nuclear power plant. The image storage unit 12 classifies and stores the input images 2 for each nuclear power plant in a different region.

[0026] The defect detection unit 30 reads one input image 101 (see FIG. 3, which will be described later) from the multiple input images 2 stored in the image storage unit 12, and performs a defect detection step (S11) in which a trained model detects defects in the structure based on the input image 101. The input image 101 is also called a flaw detection image because it is used to identify the presence or absence of defects such as scratches. In the defect detection step (S11), whether or not the input image 101 contains a defect is identified by inference processing using the defect detection model.

[0027] However, the input image 101 also contains images that are not necessarily defects, such as shadows, glare, and scratches that do not affect quality. For this reason, the defects detected by the defect detection unit 30 must be identified by the defect identification unit 40 in the next step to determine what kind of defect they actually are. Details of the processing by the defect detection unit 30 will be described later with reference to FIGS. 3 and 4.

[0028] The defect identification unit 40 performs a defect identification step (S12) in which the trained model identifies the presence or absence of a defect based on the input image 101 determined to contain a defect by the defect detection unit 30. In the defect identification step (S12), whether or not the defect is actually a defect is identified by an inference process using the defect identification model. The defect identification result is output to the annotation unit 5. Details of the processing by the defect identification unit 40 will be described later with reference to FIGS. 5 and 6.

[0029] The annotation unit 5 performs an inspector judgment step (S13) in which an inspector 6 visually inspects the structure by visually inspecting the input image 101. In the inspector judgment step (S13), it is determined whether or not the defect identification result from the defect identification step (S12) is an actual defect. If the result of the inspector 9 recognizing the input image 101 and linking the presence or absence of a defect differs from the defect identified by the defect identification unit 40, the annotation unit 5 links the annotation information to the input image 101. The judgment result by the inspector 6 is reflected in the learning process in the defect detection unit 30 and the defect identification unit 40. Details of the processing by the annotation unit 5 will be described later with reference to FIG. 7.

[0030] The selection unit 60 selects an output destination for reflecting the judgment result by the inspector 6 in the annotation unit 5 in the learning process of at least one of the defect detection unit 30 and the defect identification unit 40. The processing of the selection unit 60 is performed not only by the inspector 6 but also by an engineer 61 who understands the characteristics of AI. For this reason, based on the annotation information linked to the input image 101, the selection unit 60 selects the input image 101 to be learned by at least one of the feature learning unit 34 and the discrimination learning unit 42, and the feature learning unit 34 and the discrimination learning unit 42 that will learn the input image 101, in accordance with instructions from the engineer 61.

[0031] FIG. 3 is a block diagram showing an example of a detailed internal configuration of the defect detection unit 30. As shown in FIG.

[0032] The defect detection unit 30 includes a feature amount inference unit 31, a feature amount learning unit 34, and a defect detection model storage unit 35. The feature amount inference unit 31 includes a restoration unit 32 and a difference determination unit 33. In the defect detection step (S11), the feature amount inference unit 31 evaluates the presence or absence of a suspected defect in the input image 101 using the restoration unit 32 and the difference determination unit 33. Then, the defect detection unit 30 detects defects using a defect detection model that detects defects as a trained model.

[0033] The feature inference unit 31 infers the feature of the input image 101 read from the image storage unit 12 shown in FIG. 2. A feature is a feature that appears in the appearance of a structure, such as the pattern of the structure. The feature inference unit 31 uses a defect detection model output by a feature learning unit 34 that has been trained on healthy parts of each structure (parts that have been confirmed to contain no defects, etc.). Multiple defect detection models, which are examples of trained models trained for each structure, are stored in the defect detection model storage unit 35. The feature inference unit 31 reads out a defect detection model tailored to the structure whose features are to be inferred from the defect detection model storage unit 35 and uses it to detect defects in the structure.

[0034] As a method for evaluating defects by the feature inference unit 31, for example, a method in which the restoration unit 32 outputs a restored image 103 (see Figure 4 described later) using image restoration processing such as an autoencoder, and a method in which the difference determination unit 33 takes the difference between the original input image 101 before the restoration processing and the restored image 103 after the restoration processing are used.

[0035] Here, an example of the image restoration process by the restoration unit 32 and the difference determination process by the difference determination unit 33 will be described with reference to FIG. 4 is a schematic diagram of the image restoration process and difference determination process performed by the defect detection unit 30 according to the first embodiment. The restoration unit 32 and the difference determination unit 33 are both capable of performing the defect detection step (S11) in combination with a defect detection model.

[0036] The input image 101 includes, for example, a pattern 101a such as a weld line and a crack 101b. The pattern 101a may be present in the structure, but the crack 101b must not be present in the structure. However, even if the crack 101b appears to be a defect, it may be a reflection of light when the image is captured by the imaging unit 1.

[0037] The defect detection unit 30 outputs a restored image 103 restored from the input image 101 based on healthy part data for training data that includes healthy parts without defects. For this reason, the restoration unit 32 performs a restoration process (S111) by machine learning using, for example, an autoencoder. In the machine learning restoration process (S111), a pattern 101a, such as a weld line, at each captured part indicated by training data 102 is restored in advance as training data 102 to restore the pattern in the input image 101. At this time, the training data 102 does not include information about defects.

[0038] The restored image 103 is an example of an output image after performing the restoration process (S111) on the input image 101. The machine learning for the restoration process (S111) uses training data 102 that does not contain defect information. The training data 102 includes an image 102a showing a horizontal weld line in the figure and an image 102b showing a vertical weld line. Both images 102a and 102b show healthy parts of the structure.

[0039] In the machine learning of the restoration process (S111), training data 102 that does not include defect information is used, so images of defects are not restored. Therefore, images of defects are not included in the restored image 103. Therefore, the difference determination unit 33 compares the input image 101 with the restored image 103 to determine the difference, thereby making it possible to extract images that are suspected of being defective.

[0040] The difference determination unit 33 performs a difference determination process (S112) to determine the difference between the restored image 103 output by the restoration unit 32 and the input image 101 before the image restoration process. The difference determination unit 33 determines that there is a defect D1 if there is a difference in the image (crack 101b shown in the figure), and determines that there is no defect D2 if there is no difference. The feature amount inference unit 31 detects defects from the input image 101 based on the difference.

[0041] Returning to Figure 3, the explanation continues. If the difference determination unit 33 determines that there is no defect D2, it is determined that there is a normal judgment D3 and the appearance inspection is terminated. However, as a conservative inspection, the input image 101 that has been determined to be normal D3 may be displayed on the screen, and the inspector 6 may check again for the presence or absence of patterns 101a and cracks 101b in the input image 101. The input image 101 that the defect detection unit 30 has determined to have a defect D1 and the restored image 103 are output to the defect identification unit 40 and evaluated by the defect identification unit 40.

[0042] The feature learning unit 34 uses the healthy area data D4 for teacher data selected by the selection unit 60 to learn the feature amounts of the healthy areas and generate a defect detection model. The healthy area data D4 for teacher data is, for example, the teacher data 102 shown in FIG. 4. If there is an existing defect detection model for the structure, the feature learning unit 34 updates the existing defect detection model. The defect detection model is updated, for example, by changing the weights between layers in the neural network.

[0043] That is, the learning unit 8 shown in FIG. 1 includes a feature learning unit 34 that uses an input image 101 with annotation information that the defect detection model has not yet learned as healthy part data for training data, and causes the defect detection model to learn the features of the healthy part.

[0044] The defect detection model storage unit 35 is an example of a database, and stores the defect detection models generated by the feature amount learning unit 34. As described above, the defect detection models stored in the defect detection model storage unit 35 are read out as needed by the feature amount inference unit 31 and used for defect detection.

[0045] FIG. 5 is a block diagram showing an example of the detailed internal configuration of the defect identifying section 40 and the annotation section 5. As shown in FIG.

[0046] First, an example of the internal configuration of the defect identification unit 40 and the defect identification inference process (S121) will be described. The defect identification unit 40 includes an identification inference unit 41, an identification learning unit 42, and a defect identification model storage unit 43. The defect identification unit 40 detects defects using a defect identification model that identifies defects as a trained model.

[0047] The discrimination and inference unit 41 identifies defects in the input image 101 that have been determined to have a defect, based on the defect part data for training data that includes a defective part, and associates annotation information indicating the presence or absence of a defect with the input image 101. For example, the discrimination and inference unit 41 determines whether the image of the suspected defect part correctly represents a defect, based on the input image 101 that the difference determination unit 33 has determined to have a defect D1, as shown in Figure 3. Therefore, the discrimination and inference unit 41 determines whether the crack 101b, which is the difference part shown in Figure 4, is a defect.

[0048] An input image 101 that the defect identification unit 40 has determined to be free of defects D11 is output to the annotation unit 5, where it is visually evaluated by the inspector 6 at a first labeling unit 51 of the annotation unit 5. Similarly, an input image 101 that the defect identification unit 40 has determined to be free of defects D12 is output to the annotation unit 5, where it is visually evaluated by the inspector 6 at a second labeling unit 52 of the annotation unit 5.

[0049] Here, an outline of the processing performed by the identification and inference unit 41 of the defect identification unit 40 will be described. FIG. 6 is a diagram showing an example of the defect identification and inference process (S121) performed by the identification and inference unit 41. As shown in FIG.

[0050] 2, an emphasis process is performed on a portion of the input image 101 that is suspected of being a defect (referred to as a suspicious portion). The emphasis process is, for example, a process in which the feature amount inference unit 31 adds an emphasis portion 105 indicated by a frame around the image of a crack that is a suspected defect portion.

[0051] In the defect identification inference process (S121), the identification and inference unit 41 uses a defect identification model read from the defect identification model storage unit 43 to identify whether or not a suspected defect portion is an actual defect. The identification and inference unit 42 uses defect image training data 104 to learn defect images in advance. The defect image training data 104 includes images of only defects, such as defect images 104a and 104b that show cracks of different shapes, etc. The defect identification model is updated as the identification and inference unit 42 learns the defect images. Multiple defect identification models, which are examples of trained models learned for each structure, are stored in the defect identification model storage unit 35 (see FIG. 3). The identification and inference unit 41 performs the defect identification and inference process (S121) on the suspected defect portion in the input image 101 and outputs the identification result 106.

[0052] The identification result 106 represents the result of the identification and inference unit 41 identifying a defect in a suspected defect portion through the defect identification and inference process (S121). The defect identification model learns only the shape of the defect. Therefore, if the image of the suspected defect portion has a shape different from the learned defect shape, such as a scratch or a shadow caused by deposits, the identification and inference unit 41 will classify the input image 101 as having no defect D11. For an image of an actual crack, the defect identification and inference unit 41 will classify the input image 101 as having a defect D12, because the defect identification model has already been learned.

[0053] When the discrimination and inference unit 41 discriminates a suspected defect portion D11 through the discrimination process of the suspected defect portion performed by the discrimination and inference unit 41 using the defect discrimination model, the discrimination and inference unit 41 may output the input image 101 from which the emphasis portion 105 has been deleted to the annotation unit 5. Alternatively, the discrimination and inference unit 41 may use a color different from the emphasis color applied to the defective portion and perform emphasis processing on this portion as a suspected defect portion identified by the discrimination and inference unit 41. In this case, the discrimination and inference unit 41 highlights the position where there is a difference between the original input image 101 and the restored image 103 determined to be free of defects as a suspected defect portion.

[0054] On the other hand, if the discrimination and inference unit 41 discriminates that there is a defect D12, the input image 101 with the emphasis portion 105 remaining may be output to the annotation unit 5. Regardless of the discrimination result 106 by the discrimination and inference unit 41, the input image 101 with the emphasis portion 105 remaining may be output to the annotation unit 5.

[0055] Returning to the explanation of Figure 5. The discrimination learning unit 42 uses the defect part data D13 for teacher data selected by the selection unit 60 to learn the feature quantities of the defect image and generate a defect discrimination model. The defect part data D13 for teacher data is data input from the selection unit 60, and is data that uses only images of defects (called defects) such as cracks as teacher data. If there is a defect discrimination model for an existing structure, this defect discrimination model is updated. The defect discrimination model is updated, for example, by changing the weights between layers in the neural network.

[0056] That is, the learning unit 8 shown in FIG. 1 includes an identification learning unit 42 that uses an input image 101 with annotation information that the defect identification model has not yet learned as defect part data for training data, and causes the defect identification model to learn the feature quantities of the defect part.

[0057] The defect identification model storage unit 43 is an example of a database, and stores the defect identification model generated by the identification learning unit 42. As described above, the defect identification model stored in the defect identification model storage unit 43 is read out as needed by the identification inference unit 41 and used for defect identification.

[0058] Next, an example of the internal configuration of the annotation unit 5 and the inspector judgment process (S131) ​​will be described. The inspector 6 visually inspects the input image 101 that is the basis for defect detection and defect identification, and makes a final judgment on the presence or absence of defects.

[0059] Here, the inspector determination process (S131) ​​will be described. FIG. 7 is a diagram showing an example of the inspector determination process (S131) ​​performed by the annotation unit 5. As shown in FIG.

[0060] In the inspector judgment process (S131), an input image 101 is used in which an emphasis portion 105 is added to the original input image 101. The input image 101 with the emphasis portion 105 added is displayed on a display device or the like, and an inspector 6 visually checks the emphasis portion 105 added to the input image 101 to judge whether or not there is a defect. Then, a judgment result 107 is output.

[0061] The judgment result 107 includes any of the defect-free D21, D25, and defect-containing D23, D27 that have been added by the annotation unit 5 as described below. Depending on the defect-free D21, D25, or defect-containing D23, D27, the input image 101 is sent to the selection unit 60 and sorted for use in re-learning.

[0062] Returning to the explanation of Figure 5. The annotation unit 5 includes a first labeling unit 51 and a second labeling unit 52. The first labeling unit 51 and the second labeling unit 52 receive an operation input from an inspector 6 while checking an input image 101 displayed on a screen to attach a label indicating the result of a defect determination, and generate a determination result 107 (see FIG. 7 described later).

[0063] The first labeling unit 51 performs labeling processing to attach a label to the input image 101, which is the result of the inspector 9's judgment of the presence or absence of a defect based on recognition of the input image 101 that has been classified as "no defect" D11 by the discriminative inference unit 41. In this labeling processing, the result of the inspector 6's visual judgment is attached to the input image 101. The input image 101 that the inspector 6 judged as "no defect" D21 by the first labeling unit 51 matches the result of the discriminative inference unit 41's judgment of "no defect" D11. Therefore, the discriminative inference unit 41 has correctly judged the "no defect" D11 and makes a normal judgment D22. In this case, it is considered that the feature learning unit 34 and the discriminative learning unit 42 are using a correctly trained model, and therefore re-learning in the feature learning unit 34 and the discriminative learning unit 42 is not necessary.

[0064] On the other hand, the input image 101 determined by the first labeling unit 51 to be defective D23 differs from the result determined by the discriminative inference unit 41 to be non-defective D11. That is, the discriminative inference unit 41 erroneously determined the input image 101 to be non-defective D11. The discriminative inference unit 41 made the erroneous determination because the input image 101 was an unlearned image D24, not an image learned by the discriminative learning unit 42. When the discrimination results of the feature inference unit 31 and the discriminative inference unit 41 differ from the content of the label assigned by the inspector 6, the feature learning unit 34 and the discriminative learning unit 42 were not able to learn correctly because there was no image of the corresponding location in the training data. Therefore, the first labeling unit 51 outputs the input image 101, to which the trained model has assigned a label (an example of annotation information) indicating that the input image is non-learned D24, to the selection unit 60.

[0065] The second labeling unit 52 determines whether or not a defect exists by having the inspector 9 recognize the input image 101 that has been identified by the discriminative inference unit 41 as having a defect D12, and assigns a label representing this determination result to the input image 101. In this labeling process, the visual determination result of the inspector 6 is also assigned to the input image 101. The input image 101 that has been identified by the second labeling unit 52 as having no defect D25 differs from the result of the discriminative inference unit 41 identifying the input image as having a defect D12. In other words, the discriminative inference unit 41 erroneously determined the input image 101 to have a defect D12. The discriminative inference unit 41 made the erroneous determination because the input image 101 was an unlearned image D26 that was not learned by the discriminative learning unit 42. In this way, when the identification results of the feature inference unit 31 and the discriminative inference unit 41 differ from the content of the label assigned by the inspector 6, the feature inference unit 34 and the discriminative learning unit 42 did not learn correctly because there was no image of the corresponding location in the training data. Therefore, the second labeling unit 52 outputs to the selection unit 60 the input image 101 to which a label (an example of annotation information) indicating that the learned model is unlearned D26 has been added.

[0066] On the other hand, the input image 101 judged by the inspector 6 as having a defect D27 by the second labeling unit 52 is the same as the result judged as having a defect D12 by the discriminative inference unit 41. Therefore, the discriminative inference unit 41 has correctly judged that there is a defect D12, and judges the image as normal D28. In this case, it is considered that the feature learning unit 34 and the discriminative learning unit 42 are using a model that has been trained correctly, and therefore re-learning by the feature learning unit 34 and the discriminative learning unit 42 is not necessary.

[0067] The selection unit 60 selects whether to add the input image 101 to either or both of the training data used for training by the feature learning unit 34 and the discrimination learning unit 42, based on the judgment of the engineer 61 shown in FIG. 3. For example, if the input image 101 is determined to have a defect D1 by the defect detection unit 30, and is then determined to have a defect D12 by the defect discrimination unit 40, and is determined to have no defect D25 by the annotation unit 5, the defect detection model and defect discrimination model for this input image 101 are untrained D26. Therefore, the selection unit 60 adds this input image 101 to the healthy part data for training data D4 and the defective part data for training data D13.

[0068] Furthermore, if the input image 101 that is determined to have a defect D1 by the defect detection unit 30 is determined to have no defect D11 by the defect identification unit 40 and determined to have a defect D23 by the annotation unit 5, the determination results of the defect detection unit 30 and the annotation unit 5 are the same. However, the determination results of the defect identification unit 40 and the annotation unit 5 are different. Therefore, the selection unit 60 adds this input image 101 to the defect part data D13 for training data.

[0069] The input image 101 is input as training data to the feature amount training unit 34 and the discriminative training unit 42 selected for re-learning by the selection unit 60. Then, the feature amount training unit 34 and the discriminative training unit 42 perform progressive re-learning.

[0070] Fig. 8 is a flowchart showing an example of the appearance inspection process performed by the appearance inspection apparatus 10 according to the first embodiment. The appearance inspection process shown in Fig. 8 is one form of the appearance inspection method according to the present invention, and the details of the process will be described mainly with reference to Figs. 1, 2, 3, and 5.

[0071] First, in order to generate a trained model, an inspector 6 visually determines whether or not there is a defect in the input image 2, and the defect annotation unit 5 associates annotation information with the input image 2 (S21). The input image 2 associated with the annotation information is used as training data 7 shown in FIG. 1.

[0072] Next, the learning unit 8 reads the teacher data 7 and generates a trained model through deep learning (S22). In the case of the defect detection unit 30, the feature learning unit 34 reads the teacher data healthy part data D4 and generates a defect detection model. In the case of the defect discrimination unit 40, the discrimination learning unit 42 reads the teacher data defective part data D13 and generates a defect discrimination model.

[0073] Next, the defect inference unit 3 uses the learned model to infer defects in the input image 2 (S23). In the defect inference unit 3, the processes of the defect detection unit 30 and the defect identification unit 40 are performed.

[0074] The feature inference unit 31 of the defect detection unit 30 restores the input image 101 using the defect detection model using the restoration unit 32 as shown in FIG. 4, and determines the difference between the input image 101 and the restored image 103 using the difference determination unit 33 (S24).

[0075] Next, it is determined whether the difference determination unit 33 has determined that there is a defect (S25). If the difference determination unit 33 has determined that there is no defect (NO in S25), this process ends. If the difference determination unit 33 has determined that there is a defect (YES in S25), the identification and inference unit 41 of the defect identification unit 40 identifies and infers a defect in the input image 101 using the defect identification model as shown in FIG. 5 (S26).

[0076] Next, an inspector 6 visually determines whether or not there is a defect in the input image 2, and the defect annotation unit 5 associates annotation information with the input image 2 (S27).

[0077] Next, the engineer 61 selects the input image 101 to be used for re-learning and the trained model to be re-learned (S28) using the selection unit 60, and ends this process. The input image 101 to be used for re-learning becomes the training data 7 shown in FIG. 1. The trained model to be re-learned is a defect detection model in the defect detection unit 30, and a defect discrimination model in the defect discrimination unit 40.

[0078] <Example of computer hardware configuration> Next, the hardware configuration of the computer 70 that constitutes the visual inspection device 10 will be described.

[0079] 9 is a block diagram showing an example of the hardware configuration of a calculator 70. The calculator 70 is an example of hardware used as a computer that can operate as the visual inspection apparatus 10 according to this embodiment. In the visual inspection apparatus 10 according to this embodiment, each functional block is configured by the calculator 70 (computer) executing a program, and the functional blocks work together to realize the visual inspection method according to this embodiment.

[0080] The computer 70 includes a CPU (Central Processing Unit) 71, a ROM (Read Only Memory) 72, and a RAM (Random Access Memory) 73, each connected to a bus 74. The computer 70 further includes a display device 75, an input device 76, a non-volatile storage 77, and a network interface 78.

[0081] The CPU 71 reads out program code of software that realizes each function according to this embodiment from the ROM 72, loads it into the RAM 73, and executes it. Variables, parameters, etc. that arise during the calculation processing of the CPU 71 are temporarily written to the RAM 73, and these variables, parameters, etc. are read out as appropriate by the CPU 71 to realize the processing of each functional unit according to the first embodiment. However, an MPU (Micro Processing Unit) or a GPU (Graphics Processing Unit) may be used instead of the CPU 71, or the CPU 71 and a GPU (Graphics Processing Unit) may be used together.

[0082] The display device 75 is, for example, a liquid crystal display monitor, and displays the results of the processing performed by the computer 70 and the data that was the basis of the processing (input image 101, restored image 103, identification result 106, judgment result 107, etc.) to the inspector 6. The input device 76 is, for example, a keyboard, a mouse, etc., and allows the inspector 6 to input predetermined operations and give instructions.

[0083] The nonvolatile storage 77 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. The nonvolatile storage 77 stores an operating system (OS), various parameters, and programs for operating the computer 70. The nonvolatile storage 77 also stores a defect detection model storage unit 35, a defect identification model storage unit 43, and the like. The ROM 72 and the nonvolatile storage 77 store programs, data, and the like required for the CPU 71 to operate. That is, the ROM 72 and the nonvolatile storage 77 are used as examples of computer-readable, non-transitory storage media that store programs executed by the computer 70.

[0084] For example, a NIC (Network Interface Card) or the like is used as the network interface 78. The network interface 78 is capable of transmitting and receiving various types of data between devices via a LAN (Local Area Network), a dedicated line, or the like connected to a terminal of the NIC.

[0085] In the visual inspection apparatus 10 according to the first embodiment described above, the learning unit 8 shown in Fig. 1 is divided into two units, the feature amount learning unit 34 and the discrimination learning unit 42, and it is possible to select whether to have learning be performed by either the feature amount learning unit 34 or the discrimination learning unit 42. As a result, for an input image 101 containing a structure, defect detection using a defect detection model that has been subjected to machine learning in advance, and defect classification using a defect classification model are automatically performed.

[0086] Conventionally, inspectors 6 visually check a large number of input images 101, but the appearance inspection device 10 according to the first embodiment automatically selects only those input images 101 that contain suspected defect portions from the large number of input images 101. The inspectors 6 then need only visually check only the selected input images 101 and determine whether or not there are defects, significantly reducing the burden on the inspectors 6. Such an appearance inspection device 10 can be incorporated as part of an appearance inspection system that is useful for assisting the inspectors 6 in their visual inspections.

[0087] Furthermore, an input image 101 for which the defect detection and identification results by the AI ​​and the defect presence / absence determination results by the inspector 6 differ is selected as a target for re-learning. Therefore, the feature amount learning unit 34 and the identification learning unit 42 re-learn, respectively, thereby progressively updating the defect detection model and the defect identification model. The defect detection and identification accuracy of the feature amount inference unit 31 and the identification inference unit 41, which use the defect detection model and the defect identification model, can be improved compared to conventional defect detection models and defect identification models. This allows the visual inspection apparatus 10 to obtain highly reliable defect detection and identification results.

[0088] [Second embodiment] Next, an example of the configuration and operation of a visual inspection apparatus 10A according to a second embodiment of the present invention will be described with reference to FIGS. The visual inspection apparatus 10A according to the second embodiment is an invention that adds a coincidence determination section 37 (see FIG. 11) to the defect detection section 30 according to the first embodiment, and aims to more efficiently perform progressive learning in the feature learning section 34 and the discrimination learning section 42. The differences from the first embodiment are described below.

[0089] FIG. 10 is a block diagram showing an example of the internal configuration of the visual inspection device 10A. The visual inspection device 10A includes an input image acquisition unit 11, an image storage unit 12, a defect detection unit 30A, a defect identification unit 40, and an annotation unit 5. The visual inspection device 10A does not include the selection unit 60 according to the first embodiment, and is configured to automatically input learning data to be re-learned into the defect detection unit 30A and the defect identification unit 40.

[0090] 11 is a block diagram showing an example of the internal configuration of the defect detection section 30 A. The defect detection section 30 A includes a coincidence determination section 37 in addition to the components of the defect detection section 30 according to the first embodiment.

[0091] As shown in FIG. 3, an input image 111 determined to have a defect D1 after being processed by the restoration unit 32 and difference determination unit 33 of the feature inference unit 31 is input to the coincidence determination unit 37. The input image 111 is one image read out from the plurality of input images 2 stored in the image storage unit 12. The coincidence determination unit 37 determines the coincidence between the input image 111 determined to have a defect D1 and a restored image 112 (see FIG. 12 described later). The coincidence determination unit 37 determines an input image 101 whose coincidence is less than a threshold (e.g., 20%) as unlearned, and uses the input image 101 determined to be unlearned as healthy part data for training data to train the defect detection model on the features of healthy parts. On the other hand, the coincidence determination unit 37 determines an input image 101 whose coincidence is equal to or greater than the threshold as learned, and outputs the input image 101 determined to have learned to the defect identification unit 40.

[0092] Here, the matching determination process (S113) will be described with reference to FIG. 12 is a schematic diagram of the image restoration process, difference determination process, and coincidence calculation process performed by the defect detection unit 30A according to the second embodiment. The restoration unit 32, difference determination unit 33, and coincidence determination unit 37 are all capable of performing the defect detection step (S11) in combination with a defect detection model.

[0093] The input image 111 is a flaw detection image that is not included in the training data 102. The input image 111 includes images of a pattern 111a, a crack 111b, and the like.

[0094] The restoration unit 32 performs restoration processing (S111) by machine learning using, for example, an autoencoder or the like. As a result of performing restoration processing on the image 102a included in the training data 102, the restoration unit 32 outputs a restored image 112 including a weld line pattern. The image of the pattern 111a included in the input image 111 has not yet been learned by the feature learning unit 34. For this reason, the restored image 112 output after the restoration processing (S111) is significantly different from the original input image 111.

[0095] The difference determination unit 33 performs a difference determination process (S112) to determine the difference between the restored image 112 output by the restoration unit 32 and the input image 111 before the image restoration process. If there is a difference in the image, the difference determination unit 33 determines that there is a defect D1, and if there is no difference, it determines that there is no defect D2. In this example, the difference between the input image 111 and the restored image 112 is shown as a pattern 111a or a crack 111b, and the presence or absence of a defect is determined.

[0096] The coincidence determination unit 37 performs a coincidence calculation process (S113) to calculate the degree of coincidence by comparing the restored image 112 determined to have a defect D1 with the input image 111 before the image restoration process, and to determine whether the feature learning unit 34 has already learned the structure of the input image 111. The coincidence can be determined, for example, by converting each image to monochrome, calculating the number of coincident luminance values ​​for each pixel, and calculating the ratio of the number of pixels to the entire image. The coincidence process can be set, for example, so that a ratio of 20% or more is high and a ratio of less than 20% is low. This threshold is set internally.

[0097] The degree of match will be high if the teacher data 102 of the captured image of each structure includes the structure shown in the input image 111, and low if the structure is not included. Alternatively, the degree of match will be high if the pattern 102a shown in the teacher data 102 of each structure matches the pattern 111a shown in the input image 111, and low if they do not match. In the example of Figure 12, it is assumed that the input image 111 is a flaw detection image of a structure not included in the teacher data 102, so a result showing low match is output.

[0098] If the coincidence determination unit 37 determines that the coincidence is low, it determines that the input image 111 is an unlearned determination D6. Because the input image 111 is unlearned, the feature learning unit 34 adds the input image 111 to the training data 102 of the feature learning unit 34. On the other hand, the coincidence determination unit 37 passes the input image 111, for which the learned determination D5 has been made in the coincidence determination, to subsequent processing (each step (S12, S13) according to the first embodiment). Note that the learned determination D5 is the input image 111 labeled as "learned," and if this input image 111 is additionally learned by the feature learning unit 34, there is a risk of overlearning.

[0099] As described with reference to FIG. 5, the input image 111 is subjected to a defect identification step (S12) and an inspector judgment step (S13).

[0100] FIG. 13 is a block diagram showing an example of the detailed internal configuration of the defect identifying section 40 and the annotation section 5. As shown in FIG. The respective blocks of the defect identification unit 40 and annotation unit 5 according to the second embodiment are the same as the respective blocks of the defect identification unit 40 and annotation unit 5 according to the first embodiment shown in Fig. 5. In the defect identification process (S12) and the inspector judgment process (S13), if the judgment result by the identification and inference unit 41 differs from the judgment result by the inspector 6, additional learning is performed in each learning unit.

[0101] For example, after the discrimination and inference unit 41 has determined that there is no defect D11, the first labeling unit 51 of the annotation unit 5 determines that there is a defect D23, and the input image 111 labeled as unlearned D24 contains a defect. This input image 111 is added to the defect data D13 for training data in the discrimination learning unit 42, and is learned by the discrimination learning unit 42.

[0102] On the other hand, an input image 111 that is determined to have a defect D12 by the discrimination and inference unit 41 and then determined to have no defect D25 by the second labeling unit 52 of the annotation unit 5 is a healthy part and is unlearned data. Therefore, the input image 111 labeled as unlearned D26 is added to the healthy part data D4 for training data shown in FIG. 11 and is learned by the feature learning unit 34.

[0103] This method enables efficient progressive learning by the feature learning unit 34 and the discrimination learning unit 42. Note that in the visual inspection apparatus 10A according to the second embodiment, step S28 is removed from the flowchart shown in Fig. 8. Therefore, a description of the flowchart for the visual inspection apparatus 10A will be omitted.

[0104] In the visual inspection apparatus 10A according to the second embodiment described above, the input image 111 determined as unlearned (D6) by the match determination unit 37 included in the defect detection unit 30A is subject to re-learning by the feature learning unit 34. The input image 111 determined as learned (D5) by the match determination unit 37 is subject to re-learning by the corresponding feature learning unit 34 or discrimination learning unit 42 when the defect presence / absence determination result by the defect identification unit 40 differs from the defect presence / absence determination result by the annotation unit 5. The inspector 6 does not need to decide whether to load the input image 111 to be re-learned into the feature learning unit 34 or discrimination learning unit 42. This eliminates the need for the inspector 6 to select the input image 111 to be re-learned, thereby reducing the burden on the inspector 6 and shortening the time required for progressive learning.

[0105] Furthermore, the input image 111 to be re-learned is unlearned data, which allows the feature learning unit 34 and the discriminative learning unit 42 to prevent overlearning using already learned data.

[0106] [Variations] In the above-described embodiments, the visual inspection devices 10 and 10A are used for visual inspection of structures in a nuclear power plant. However, the visual inspection devices 10 and 10A can also be used in other applications besides nuclear power plants, such as hydroelectric power plants and thermal power plants, as well as for railway track maintenance work.

[0107] Furthermore, although the visual inspection apparatuses 10 and 10A have been described as a single apparatus, the functional units constituting the visual inspection apparatuses 10 and 10A may be configured as different devices. For example, the defect detection unit 30, the defect identification unit 40, and the annotation unit 5 may be configured as different devices or cloud applications, and a visual inspection system may be configured in which these functional units are integrated into one.

[0108] In addition, the first labeling section 51 and the second labeling section 52 configured in the annotation section 5 may be replaced with, for example, a defect determination model that has undergone machine learning, so that they can operate without the intervention of the inspector 6.

[0109] Furthermore, by using the visual inspection apparatus 10A according to the second embodiment, the defect identification model and its performance may change during the inspection period in which the visual inspection is being performed, and the output results of each model may also change before and after the performance change. For this reason, during the inspection period in which the visual inspection is being performed using the visual inspection apparatus 10 according to the first embodiment, the selection unit 60 waits for the execution of the selection step (S14). After the inspection period ends, the selection step (S14) shown in FIG. 2 is executed, thereby progressively training the defect identification model and its performance. The re-trained defect identification model and its performance may be used for visual inspection in a location separate from the structure in which the previous visual inspection was performed.

[0110] It should be noted that the present invention is not limited to the above-described embodiments, and various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiments have described the configuration of the device and system in detail and specifically in order to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of the embodiments described here with the configuration of other embodiments, and it is also possible to add the configuration of one embodiment to the configuration of another embodiment. Furthermore, it is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0111] 1...imaging unit, 2...input image, 3...defect inference unit, 4...output result, 5...annotation unit, 6...inspector, 7...teaching data, 8...learning unit, 9...engineer, 10...visual inspection device, 11...input image acquisition unit, 12...image storage unit, 30...defect detection unit, 31...feature inference unit, 32...restoration unit, 33...difference determination unit, 34...feature learning unit, 35...defect detection model storage unit, 40...defect identification unit, 41...identification inference unit, 42...identification learning unit, 43...defect identification model storage unit, 51...first labeling unit, 52...second labeling unit, 60...selection unit

Claims

1. an annotation unit that associates annotation information, which indicates whether a defect in a structure is normal or not, determined by a user based on an input image used in visual inspection of the structure, with the input image; a learning unit that generates an initially set trained model by deep learning using the annotation information and the input image linked to the annotation information; and a defect inference unit that outputs an inference result obtained by inferring the defect based on the input image using the trained model generated by the training unit, The annotation unit associates the annotation information with the input image for which the inference result and the user's judgment of the input image differ, and causes the learning unit to re-learn the trained model. Visual inspection system.

2. The defect inference unit includes a defect detection unit that detects defects in the structure using the trained model based on the input image; a defect identification unit that identifies the presence or absence of the defect using the trained model based on the input image in which the defect detection unit has determined that the defect is present; The annotation unit associates the annotation information with the input image when the result of the user recognizing the input image and associating the presence or absence of the defect differs from the defect identified by the defect identification unit. The visual inspection system according to claim 1 .

3. The defect detection unit includes a restoration unit that outputs a restored image restored from the input image based on healthy part data for training data that includes the healthy parts without defects, and a difference determination unit that determines a difference between the input image and the restored image, and is equipped with a feature amount inference unit that detects the defect from the input image based on the difference. The visual inspection system according to claim 2 .

4. the defect detection unit detects the defect using a defect detection model that detects the defect as the trained model; The learning unit includes a feature amount learning unit that uses the input image with the annotation information added, for which the defect detection model has not yet learned, as healthy part data for training data, and causes the defect detection model to learn feature amounts of the healthy part. The visual inspection system according to claim 3 .

5. The defect identification unit includes an identification and inference unit that identifies the defect in the input image determined to have the defect based on defect portion data for teacher data including the defect portion having the defect, and associates the annotation information indicating the presence or absence of the defect with the input image. The visual inspection system according to claim 4 .

6. the defect identification unit detects the defect by using a defect identification model that identifies the defect as the trained model; The learning unit includes a discrimination learning unit that uses the input image with the annotation information added, which has not yet been learned by the defect identification model, as defect part data for teacher data, and causes the defect identification model to learn feature quantities of the defect part. The visual inspection system according to claim 5 .

7. The annotation portion includes: a first labeling unit that assigns a label to the input image that has been identified as having no defect by the identification and inference unit, the label being a result of the user's recognition of the presence or absence of the defect; a second labeling unit that assigns a label to the input image that has been identified as having a defect by the identification and inference unit, the label being a result of the user's recognition of the presence or absence of the defect; the first labeling unit associates the input image determined to have a defect with the annotation information that the trained model has not yet trained; The second labeling unit associates the annotation information that the trained model has not yet learned with the input image that has been determined to have no defect. The visual inspection system according to claim 6 .

8. a selection unit that allows the user to select the input image to be learned by at least one of the feature amount learning unit and the discriminative learning unit, and the feature amount learning unit and the discriminative learning unit that will learn the input image, based on the annotation information linked to the input image; The visual inspection system according to claim 7 .

9. The defect detection unit includes a coincidence determination unit that determines a degree of coincidence between the input image determined by the feature amount inference unit to have a defect and the restored image, determines the input image for which the degree of coincidence is less than a threshold to be unlearned, uses the input image determined to be unlearned as healthy part data for training data, and causes the defect detection model to learn feature amounts of the healthy part, determines the input image for which the degree of coincidence is equal to or greater than the threshold to be learned, and outputs the input image determined to be learned to the defect identification unit. The visual inspection system according to claim 4 .

10. an input image acquisition unit that acquires the input image from an imaging unit that images the structure; an input image storage unit that stores the plurality of input images acquired by the input image acquisition unit so that the input images can be read by the defect detection unit; a model selection unit that selects the defect detection model and the defect identification model to be used in the appearance inspection for each of the structures, the defect detection unit includes a defect detection model storage unit that stores a plurality of the defect detection models; The defect identification unit includes a defect identification model storage unit that stores a plurality of the defect identification models. The visual inspection system according to any one of claims 4 to 9.

11. a step of associating annotation information, which indicates whether a defect in the structure is normal or not, determined by a user based on an input image used for visual inspection of the structure, with the input image; generating the trained model by deep learning using the annotation information and the input image linked to the annotation information for the initially set trained model; outputting an inference result of inferring the defect based on the input image using the generated trained model; and associating the annotation information with the input image for which the inference result and the user's judgment on the input image differ, and re-training the trained model. Visual inspection method.

Images