Appearance analysis system, appearance analysis method, and program

Abstract

To provide a technique for performing efficient learning in appearance analysis utilizing machine learning.SOLUTION: An appearance analysis system includes at least one processor and a memory resource. The processor performs: obtaining a learning dataset including a plurality of learning images indicating an appearance of an article; dividing the learning images into a plurality of regions of interest by using a division model; generating, for each of the regions of interest, an evaluation model for evaluating the region of interest and outputting a result of evaluating the region of interest by each evaluation model; and performing learning of the division model by using a value determined using the evaluation result and the learning data set.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an appearance analysis system, an appearance analysis method, and a program. [Background technology]

[0002] 2. Description of the Related Art In the manufacture of industrial products, appearance inspection is widely carried out to evaluate the appearance of an article by determining shape defects, assembly defects, adhesion of foreign matter, etc. based on an inspection image.

[0003] With regard to the appearance inspection device disclosed in Patent Document 1, paragraph

[0054] states, "The appearance of the welded portion 201 of the workpiece 200 is inspected by the shape measurement unit 21 of the appearance inspection device 20 (step S1)." Paragraph

[0059] states, "By executing step S5, the presence or absence of a shape defect and the type of the shape defect are identified in the acquired image data. Based on this result, the learning data set is reviewed, recreated, or newly created (step S6), and the judgment model is re-trained using the learning data set created in step S6 (step S7)." Paragraph

[0061] states, "By performing the routine shown in Figure 5 an appropriate number of times and frequency as necessary, the accuracy of the judgment model for determining whether the shape of the welded portion is good or bad can be improved, and the accuracy of determining the presence or absence and type of shape defect that is important for determining whether the shape of the welded portion is good or bad can be improved." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 129617 Summary of the Invention [Problem to be solved by the invention]

[0005] Traditionally, most visual inspections have been performed visually by inspectors, but due to the increased workload on inspectors and the resulting variability in inspection results, a method of visual inspection using an evaluation engine based on machine learning is now being used.When using a single evaluation engine for the visual inspection of items with diverse structures, a huge amount of learning is required to achieve high inspection performance, making it inefficient.

[0006] The present invention has been made in consideration of the above points, and aims to provide a technology for efficiently learning in appearance analysis using machine learning. [Means for solving the problem]

[0007] The present application includes a number of means for solving at least part of the above problems, examples of which are as follows.

[0008] In order to solve the above problem, the appearance analysis system of the present invention is an appearance analysis system comprising at least one processor and memory resources, wherein the processor: * acquires a training dataset including a plurality of training images showing the appearance of an article; * uses a segmentation model to segment the training images into a plurality of regions of interest; * generates an evaluation model for each region of interest that evaluates the region of interest, and outputs evaluation results for the region of interest using each of the evaluation models; and * trains the segmentation model using values determined using the evaluation results and the training dataset. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a technology for efficiently learning in appearance analysis using machine learning.

[0010] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a processing sequence of the entire appearance analysis system. [Figure 2] FIG. 10 is a diagram illustrating an example of a processing method in a learning phase of the appearance analysis system. [Figure 3] FIG. 10 is a diagram illustrating an example of a processing method in an inspection phase of the appearance analysis system. [Figure 4] FIG. 10 is a diagram showing an example of a processing method in a learning phase of the appearance analysis system in the first modified example. [Figure 5] FIG. 10 is a diagram showing an example of a processing method in an inspection phase of the appearance analysis system in the first modified example. [Figure 6] FIG. 10 is a diagram showing an example of a processing method in a learning phase of an appearance analysis system in a second modified example. [Figure 7] FIG. 10 is a diagram showing an example of a processing method in an inspection phase of the appearance analysis system in the second modified example. [Figure 8] FIG. 11 is a diagram showing an example of a processing method in a learning phase of an appearance analysis system in a third modified example. [Figure 9] FIG. 10 is a diagram illustrating an example of specifying a division condition. [Figure 10] FIG. 13 is a diagram showing an example of a processing method in a learning phase of an appearance analysis system in a fourth modified example. [Figure 11] FIG. 10 is a diagram for explaining an example of a division method using design data. [Figure 12] FIG. 13 is a diagram showing an example of a processing method in an inspection phase of an appearance analysis system in a fourth modified example. [Figure 13] FIG. 1 illustrates an example of a hardware configuration of an appearance analysis system. DETAILED DESCRIPTION OF THE INVENTION

[0012] Visual inspections are widely performed for many industrial products, including those in machinery, metals, chemicals, food, and textiles, based on inspection images to evaluate various aspects of workmanship, such as shape defects, assembly defects, foreign matter adhesion, internal defects and criticality, and surface scratches, spots, and dirt. Traditionally, most of these visual inspections have been performed by inspectors through visual judgment. However, with increasing demands for mass production and quality improvement, inspection costs and the burden on inspectors are increasing.

[0013] Furthermore, sensory testing, which relies on human senses, requires particularly high levels of experience and skill. Individuality and reproducibility, such as different evaluation criteria depending on the inspector and different results each time the test is conducted, are also issues. To address these issues of cost, skill, and individuality, there is a strong demand for automation of testing.

[0014] In recent years, the performance of machine learning has improved dramatically with the proposal of deep network models, such as Convolutional Neural Networks (CNNs). Various methods have been proposed for machine learning-based evaluation engines, but one well-known method is "ensemble learning," which improves evaluation performance by using multiple evaluation engines rather than a single one.

[0015] There are several methods for ensemble learning. For example, parallel processing can be used, in which the same test object is evaluated using multiple evaluation engines and the average or majority vote of the evaluation results is taken, or serial processing can be used, in which the test object is evaluated using a first evaluation engine, and if the evaluation is not successful, a second evaluation engine is used to evaluate it.

[0016] In training the evaluation engine, an image of the object to be inspected for training (training image) is input, and the internal parameters of the evaluation engine (network weights, biases, etc.) are updated so that the difference between the estimated evaluation value output by the evaluation engine and the correct evaluation value instructed by the inspector becomes smaller.

[0017] Regarding the timing of updating the internal parameters, rather than learning all the training images at once, it is possible to divide the training images into several sets called mini-batches and update the internal parameters for each mini-batch. This is called mini-batch learning, and once all the mini-batches have been learned, all the training images have been used for learning. Learning all these mini-batches once is called one epoch, and by repeating epochs multiple times, the internal parameters are optimized. It is also possible to shuffle the training images included in the mini-batch for each epoch.

[0018] When performing visual inspection using machine learning on objects with diverse structures, such as industrial products, high inspection performance may not be achieved due to the diversity of structures and appearances of the objects, as well as the variety of defect types that occur.

[0019] In contrast, a method of visual inspection using a single evaluation engine with a high degree of freedom that can judge many pattern variations is envisioned. Here, an evaluation engine with a high degree of freedom refers to an evaluation engine with many internal parameters that must be determined through learning, such as a deep network model with many layers.

[0020] In this case, a huge number of learning images are required to appropriately determine many internal parameters, but as a practical matter, it may be difficult to collect many learning images in advance on a production line, etc. In this embodiment, by using multiple evaluation engines with a small learning load, highly accurate inspection can be performed even on industrial products with diverse structures.

[0021] <Outline of Appearance Analysis System 1> The following describes examples of embodiments of the present invention with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0022] 1 is a diagram showing an example of the overall processing sequence of the appearance analysis system 1. The appearance analysis system 1 executes a learning phase and an inspection phase. The learning phase is a phase in which an inspection engine is trained using learning images, and the inspection phase is a phase in which the inspection images are evaluated using the inspection engine trained in the learning phase.

[0023] First, as a preliminary step, the appearance analysis system 1 acquires an image of the object to be inspected (step S0). The image shows the appearance of the item and is acquired by capturing an image of the surface or interior of the object to be inspected as a digital video using an imaging device such as a CCD (Charge Coupled Device) camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, or an X-ray inspection device.

[0024] The appearance analysis system 1 may acquire an image of the inspection target by receiving an image captured by another system (not shown) and storing the image in a storage resource 102 (described later) of the appearance analysis system 1. The image of the object is used as a learning image in the learning phase and as an inspection image in the inspection phase.

[0025] Next, in the learning phase, a correct evaluation value is assigned to each learning image (step S1). Specifically, the appearance analysis system 1 assigns a correct evaluation value determined using a predetermined evaluation standard to each learning image. The evaluation value is an index for evaluating various aspects of the inspection target, such as shape defects, assembly defects, adhesion of foreign matter, internal defects and criticality of the target, scratches, spots, and dirt on the surface, and the evaluation method is not limited. For example, the criticality of the defect or the surface condition may be quantified and used as the evaluation value, or a label classifying the type of defect that has occurred may be used as the evaluation value.

[0026] As an example, the appearance analysis system 1 determines a correctness evaluation value for a certain evaluation criterion based on input operations made by an inspector's visual judgment or numerical values analyzed by other inspection devices or means, and stores the value in association with each training image. Note that the appearance analysis system 1 in this embodiment can perform training of the inspection engine, i.e., update internal parameters, for each predetermined number of training images. Therefore, one correctness evaluation value is assigned for each training image. As an example, the correctness evaluation value is assigned for each mini-batch. In this case, one unit is a plurality of training images that make up a mini-batch, and the inspection engine performs training for each mini-batch.

[0027] The learning images acquired in step S0 and the correct answer evaluation values assigned in step S1 are used as a learning data set.

[0028] Next, the appearance analysis system 1 performs learning of the inspection engine (step S2). The inspection engine is an estimator that receives an inspection image as input and outputs an estimated evaluation value. The inspection engine of this embodiment has an image segmentation engine Ec as a segmentation model and an evaluation engine {Ed_i} as an evaluation model, each of which is optimized by learning.

[0029] The image segmentation engine Ec and the evaluation engine {Ed_i} can use various existing machine learning engines, such as deep neural networks represented by Convolutional Neural Networks (CNNs), Support Vector Machines (SVMs) / Support Vector Regression (SVRs), k-nearest neighbor (k-NN), etc. These inspection engines can handle region segmentation, classification problems, and regression problems.

[0030] In this step, the appearance analysis system 1 optimizes the internal parameters of the inspection engines (image segmentation engine Ec and evaluation engines {Ed_i}) so that, when the learning image acquired in step S0 is input, an estimated evaluation value close to the correct evaluation value assigned in step S1 is output. In the case of a neural network, the internal parameters include "hyperparameters" such as the network structure, activation function, learning rate, and learning termination condition, as well as "model parameters" such as the weights (coupling coefficients) and biases between network nodes.

[0031] A rule-based engine can also be used for the image segmentation engine Ec and the evaluation engine {Ed_i}. In this case, the internal parameters are image processing parameters such as filter coefficients for various image processes and judgment thresholds. In addition, the image segmentation engine Ec and the evaluation engine {Ed_i} may be a combination of a machine learning engine and a rule-based engine.

[0032] The internal parameters of the image segmentation engine Ec and the internal parameters of the evaluation engine {Ed_i} obtained as a result of the learning in this step are used in the inspection phase.

[0033] In the inspection phase, an automatic inspection is performed by the inspection engine (step S3). The appearance analysis system 1 performs an inspection on the inspection image acquired in step S0. Specifically, the appearance analysis system 1 inputs the internal parameters of the image segmentation engine Ec and the internal parameters of the evaluation engine {Ed_i} obtained through learning to the image segmentation engine Ec and the evaluation engine {Ed_i}, respectively.

[0034] The appearance analysis system 1 divides an inspection image into regions of interest using an image division engine Ec. The appearance analysis system 1 evaluates the divided regions of interest using an evaluation engine {Ed_i}, and outputs an estimated evaluation value for the inspection image. The processing of the image division engine Ec and the evaluation engine {Ed_i} will be described in detail later. If necessary, an inspector may check the inspection results output by the inspection engine.

[0035] <How to process during the learning phase> Next, the processing of the appearance analysis system 1 in the learning phase will be described. Fig. 2 is a diagram showing an example of a processing method in the learning phase of the appearance analysis system 1. First, the appearance analysis system 1 acquires a training image group 2 (step S21). Specifically, as shown in step S0 of Fig. 1, the appearance analysis system 1 acquires a training image group 2 which is a plurality of training images taken of an object to be inspected. The appearance analysis system 1 divides the training image group 2 into training images 21 each serving as a unit.

[0036] Furthermore, the appearance analysis system 1 assigns a correctness evaluation value to each unit of training image 21. The assignment of the correctness evaluation value is the same as the process performed in step S1 of FIG. 1. The training image group 2 and the correctness evaluation value are treated as a training data set. The appearance analysis system 1 passes each unit of training image 21 and the correctness evaluation value to the image segmentation engine Ec. Below, an example will be described in which one training image 21 is input to the image segmentation engine Ec as one unit of training image 21. Note that, as mentioned above, the number of training images 21 per unit is not limited to this example.

[0037] The appearance analysis system 1 performs the processes from step S22 to step S24 for each unit of learning image 21, thereby optimizing the internal parameters of the image segmentation engine Ec and the evaluation engine {Ed_i}.

[0038] Next, the appearance analysis system 1 uses the image segmentation engine Ec to segment the training image 21 (step S22). Specifically, the appearance analysis system 1 segments the training image 21 into a plurality of regions of interest {R_i} (i=1, ..., NR, NR: number of regions of interest). Note that, as will be described in detail later, the image segmentation engine Ec performs the kth iterative learning using the result of the k-1th iterative learning. That is, based on the result of the k-1th iterative learning, the kth segmentation is performed on the kth unit of the training image 21.

[0039] FIG. 2 shows an example in which one learning image 21 is divided into three regions of interest (region of interest R_1(k), region of interest R_2(k), and region of interest R_3(k)). In region of interest R_i, white pixels indicate the part of interest. For example, in region of interest R_1(k), the gear region indicated by white pixels is the part of interest, and in step S23 described below, the evaluation engine Ed_1 masks (excludes) black pixels before processing. Note that although the number of regions of interest R_i shown in FIG. 2 is three, the number of regions of interest and the method of division can change depending on the learning results of the image division engine Ec.

[0040] Note that the determination of the region of interest by the image segmentation engine Ec may be performed by labeling each pixel in the learning image 21 to which region of interest R_i it belongs (semantic segmentation), as shown in Fig. 2. Alternatively, the learning image 21 may be divided into regions of a certain size in a grid pattern (not shown), and each region may be classified as a region of interest R_i.

[0041] Next, the appearance analysis system 1 evaluates the learning image 21 using the evaluation engines Ed_i (step S23). Specifically, the appearance analysis system 1 generates evaluation engines Ed_i specialized for the attention regions R_i(k), and evaluates each attention region R_i(k) using each evaluation engine Ed_i. That is, in the appearance analysis system 1, the number of evaluation engines Ed_i changes depending on the number of attention regions R_i.

[0042] In the example shown in FIG. 2, the appearance analysis system 1 evaluates each region of interest R_i(k) using an evaluation engine Ed_1 generated to evaluate the region of interest R_1(k), an evaluation engine Ed_2 generated to evaluate the region of interest R_2(k), and an evaluation engine Ed_3 generated to evaluate the region of interest R_3(k). In the example shown in the figure, the evaluation engine Ed_i calculates an individual evaluation value for the region of interest R_i by calculating an evaluation value for each pixel of the region of interest R_i, but the evaluation method of the evaluation engine Ed_i is not limited to this. Each evaluation engine Ed_i outputs an individual evaluation value for the region of interest R_i as the evaluation result.

[0043] The appearance analysis system 1 obtains an estimated evaluation value of the training image 21 by integrating the individual evaluation values. The appearance analysis system 1 determines the test accuracy rate using the estimated evaluation value of the training image 21 and the correct evaluation value assigned to the training image 21 as a training data set. Note that the appearance analysis system 1 may determine the test accuracy rate of the training image 21 by dividing the correct evaluation value according to the region of interest and comparing and integrating the individual evaluation values.

[0044] The test accuracy rate can be said to be an evaluation value for the learning image 21. In addition, the evaluation value for the learning image 21 is not limited to the test accuracy rate, but may be any value that indicates the relationship between the learning image 21 and the accuracy evaluation value.

[0045] Furthermore, the appearance analysis system 1 can repeat the process of step S23 for each unit of learning image 21, thereby optimizing the internal parameters of each evaluation engine Ed_1.

[0046] Next, the appearance analysis system 1 executes learning of the image segmentation engine Ec (step S24). First, as a background for learning the image segmentation engine Ec, it is necessary to determine what kind of regions of interest R_i the inspection image should be appropriately divided into, and to determine internal parameters for dividing the image into such regions of interest R_i.

[0047] Based on the accuracy rate of the inspection results (inspection accuracy rate) obtained by integrating the individual evaluation values of each trained evaluation engine Ed_i, the appearance analysis system 1 updates the division method of the attention region {R_i[k+1]} in the (k+1)th iterative learning so as to improve the inspection accuracy rate, and learns the internal parameters of the image segmentation engine Ec so that such division can be performed. For example, the appearance analysis system 1 integrates or divides the attention region {R_i[k]} during the kth iterative learning using the inspection accuracy rate of the training image 21 as an evaluation value, and determines the optimal division method of the attention region {R_i}. Thereafter, the appearance analysis system 1 performs the (k+1)th processing from step S22 to step S24 on the next training image 21.

[0048] In other words, the appearance analysis system 1 evaluates the region of interest {R_i} obtained by dividing one unit of the training image 21, and uses the evaluation value determined as a result to train the image division engine Ec and the evaluation engine {Ed_i}. The appearance analysis system 1 divides another unit of the training image using the internal parameters obtained as a result of the training.

[0049] There are no limitations on the conditions for ending the process shown in Fig. 2. For example, the appearance analysis system 1 may end the process in this figure when a predetermined number of epochs of learning have been executed, or may end the process by performing validation using validation data.

[0050] As described above, in this embodiment, an inspection image is divided into multiple regions of interest R_i by region segmentation using the image division engine Ec, and inspection is performed using a dedicated evaluation engine Ed_i specialized for the region of interest R_i. In other words, the evaluation engine is switched depending on the region of interest R_i. By determining the region of interest R_i so that the structure, appearance, and types of defects occurring in the object to be inspected are limited as much as possible, the pattern variations handled by each evaluation engine Ed_i are reduced. This reduces the learning load, and even if an evaluation engine with a low degree of flexibility is used, it is expected that the number of required learning images will be reduced and inspection performance will be improved.

[0051] In this embodiment, the image segmentation engine Ec and the evaluation engine Ed_i are optimized by repeatedly learning them using the inspection accuracy rate as an evaluation value. That is, a high inspection accuracy rate can be achieved across the entire inspection engine, which has multiple evaluation engines Ed_i. In other words, because multiple evaluation engines Ed_i share the responsibility for a variety of inspection targets and inspection items, high inspection performance can be achieved for industrial products with diverse structures.

[0052] <Processing method during the inspection phase> FIG. 3 is a diagram showing an example of a processing method in the inspection phase of the appearance analysis system 1.

[0053] First, the appearance analysis system 1 acquires an inspection image 3 showing the appearance of an article (step S31). The processing in this step is similar to the processing performed in step S0 in FIG.

[0054] Next, the appearance analysis system 1 uses the image segmentation engine Ec to segment the inspection image 3 acquired in step S31 (step S32). In this step, the image segmentation engine Ec segments the inspection image 3 using the internal parameters obtained in the learning phase shown in Fig. 2. The inspection image 3 is segmented into multiple regions of interest R_i based on the internal parameters after learning.

[0055] Next, the appearance analysis system 1 uses the evaluation engine Ed_i to evaluate the attention region R_i obtained by division in step S32 (step S33). In this step, the evaluation engine Ed_i evaluates the attention region R_i using the internal parameters obtained in the learning phase shown in Fig. 2. Note that the evaluation engines Ed_i used for evaluation in this step are generated as many times as the number of attention regions R_i obtained by division in step S32. In other words, an evaluation result is output for each attention region R_i obtained as a result of division.

[0056] Next, the appearance analysis system 1 outputs the inspection result (step S34). The appearance analysis system 1 integrates the individual evaluation values, which are the evaluation results output by each evaluation engine Ed_i in step S33, to output an estimated evaluation value of the inspection image 3 as the inspection result.

[0057] <Processing method in the learning phase in the first modified example> Next, an appearance analysis system 1 in a first modified example will be described. Differences from the above-described embodiment will be described below. In the example of the learning phase shown in FIG. 2, the evaluation method of the evaluation engine Ed_i is not limited, and it has been explained that an evaluation value can be calculated for each pixel of the attention area R_i, for example. In the learning phase, the appearance analysis system 1 in this modified example evaluates the attention area R_i using a local area.

[0058] 4 is a diagram showing an example of a processing method in the learning phase of the appearance analysis system 1 in the first modified example. The processing performed by the appearance analysis system 1 in steps S41 and S42 is similar to the processing performed in steps S21 and S22 in FIG. 2, and therefore a description thereof will be omitted.

[0059] Next, the appearance analysis system 1 uses the evaluation engine Ed_i to evaluate the learning image 41 (step S43). The appearance analysis system 1 generates an evaluation engine Ed_i specialized for the region of interest R_i(k) and evaluates each region of interest R_i(k) using each evaluation engine Ed_i, similar to step S23 in FIG. 2.

[0060] In this modified example, the appearance analysis system 1 divides each region of interest R_i into multiple local regions f_i(m) (m=1, ..., NF_i, NF_i: number of local regions into which the region of interest R_i is divided), and evaluates each local region f_i(m) using an evaluation engine Ed_i, thereby evaluating the region of interest R_i composed of the local regions f_i(m). Note that the method for dividing the local regions f_i(m) is not limited, and may be, for example, a grid of a predetermined size.

[0061] Similar to the example shown in FIG. 2, evaluation engines Ed_1 are generated for each of the regions of interest R_i divided in step S42. For example, each evaluation engine Ed_1 evaluates the local regions f_i(m) that make up the corresponding region of interest R_i, and integrates the obtained values to output an individual evaluation value for the region of interest R_i. In other words, the appearance analysis system 1 outputs an individual evaluation value for the region of interest R_i using information indicating the relationship between the local region f_i(m) and the region of interest R_i, i.e., information indicating the region of interest R_i to which the local region f_i(m) belongs, and an evaluation of the local region f_i(m). Note that the information indicating the relationship between the local region f_i(m) and the region of interest R_i is not limited to this example.

[0062] The process executed by the appearance analysis system 1 in step S44 is the same as that in step S24 in FIG.

[0063] <Processing method in the inspection phase in the first modified example> 5 is a diagram showing an example of a processing method in the inspection phase of the appearance analysis system 1 in the first modified example. The processing performed by the appearance analysis system 1 in steps S51 and S52 is similar to the processing performed in steps S31 and S32 in FIG. 3, and therefore a description thereof will be omitted.

[0064] Next, the appearance analysis system 1 uses the evaluation engine Ed_i to evaluate the inspection image 5 (step S53). In the inspection phase, the evaluation engine Ed_i of the appearance analysis system 1 in the first modified example divides each attention region R_i into local regions and evaluates them, as in the learning phase shown in Fig. 4. The process executed by the appearance analysis system 1 in step S54 is the same as the process in step S34 in Fig. 3, and therefore description thereof will be omitted.

[0065] <Processing method in the learning phase in the second modified example> The appearance analysis system 1 in the second modified example will be described below. Differences from the above embodiment will be described below. The appearance analysis system 1 in this modified example evaluates an image by setting a degree of belonging to a local region.

[0066] 6 is a diagram showing an example of a processing method in the learning phase of the appearance analysis system 1 in the second modified example. The processing performed by the appearance analysis system 1 in steps S61 and S62 is similar to the processing performed in steps S21 and S22 in FIG. 2, and therefore a description thereof will be omitted.

[0067] Next, the appearance analysis system 1 divides the training image 61 into local regions {f_j} (j = 1, ..., Nr, Nr: number of divided regions) using the image division engine Ec, and estimates the degree of belonging {a_i(j)} (i = 1, ..., NR, NR: number of regions of interest) to the region of interest R_i for each local region f_j (step S63). The degree of belonging {a_i(j)} is an index indicating the likelihood that the local region f_j belongs to each region of interest R_i.

[0068] In this modified example, as in the above-described embodiment, the basic process is to divide the inspection image into multiple regions of interest R_i by the image segmentation engine Ec, and to perform inspection using a different evaluation engine Ed_i for each region of interest R_i. However, there are cases where it is difficult to perform reliable region segmentation using the image segmentation engine Ec for some inspection objects.

[0069] That is, it is difficult to determine whether a local region f_j in the inspection image (which may be a region obtained by dividing the image into a grid, or may be in pixel units) belongs to the first region of interest R_1 or the second region of interest R_2. If this determination is incorrect, the inspection will be performed using an evaluation engine different from the evaluation engine Ed_i that was originally used, and therefore the performance of dividing into regions of interest R_i will affect the overall inspection performance.

[0070] Therefore, if it is difficult to determine, the local region f_j is not assigned to one region of interest R_i, but is assigned to multiple regions of interest R_i that are more likely to be assigned. For example, if there is a high probability that the local region f_1 belongs to three regions of interest R_1, R_2, and R_3, the region of interest R_i to which the local region f_1 belongs is determined to be R_i={R_1, R_2, R_3}.

[0071] When performing region segmentation, the image segmentation engine Ec can output the likelihood that a region belongs to the region of interest R_i, rather than a binary value indicating whether or not a region belongs to the region of interest R_i. The degree of belonging {a_i} may also be calculated based on the shape of the segmented region. For example, if a region of interest R_1 contains a small region classified as region of interest R_2, this small region may be noise, and it may be more appropriate to classify it as the surrounding region of interest R_1. In this case, the image segmentation engine Ec can calculate the degree of belonging of the small region as {a_1, a_2} = {0.4, 0.6}, for example. This means that there is a 40% chance that the small region belongs to region of interest R_1, and a 60% chance that it belongs to region of interest R_2.

[0072] In addition, if the shape of the area classified as the attention area R_1 is significantly distorted, the reliability of the result is also considered to be low. In such a case, the image segmentation engine Ec can lower the degree of belonging to the attention area R_1 and increase the degree of belonging to the other attention areas.

[0073] Next, the appearance analysis system 1 uses the evaluation engines Ed_i to evaluate the learning images 61 (step S64). The appearance analysis system 1 generates evaluation engines Ed_i specialized for the regions of interest R_i(k) and evaluates each region of interest R_i(k) using each evaluation engine Ed_i, similar to step S23 in FIG. 2.

[0074] The appearance analysis system 1 in this modified example uses evaluation engines Ed_i to evaluate local regions {f_j}. Specifically, the appearance analysis system 1 obtains evaluation values using evaluation engines Ed_i corresponding to regions of interest R_i to which each local region f_j may belong. For example, if local region f_1 may belong to regions of interest R_1, R_2, or R_3, the appearance analysis system 1 inspects local region f_1 using evaluation engines Ed_1, Ed_2, and Ed_3 to obtain three evaluation values: an evaluation value when local region f_1 belongs to region of interest R_1, an evaluation value when local region f_1 belongs to region of interest R_2, and an evaluation value when local region f_1 belongs to region of interest R_3.

[0075] Next, the appearance analysis system 1 integrates each evaluation value based on the degree of belonging {a_i} to obtain an individual evaluation value, which is the final inspection result of the local region f_j. For example, the integration can be performed by weighting each evaluation value based on the magnitude of the degree of belonging {a_i}, but the integration method is not limited to this. The appearance analysis system 1 uses the final inspection result of the local image {f_j} to obtain an estimated evaluation value of the training image 61.

[0076] Next, the appearance analysis system 1 executes learning of the image segmentation engine Ec (step S65). This process is similar to the process of step S24 in FIG. 2, and therefore a description thereof will be omitted.

[0077] As described above, in this modified example, highly reliable inspection performance can be obtained by performing processing without limiting the region of interest R_i to which the local region f_j belongs to one.

[0078] <Processing method in the inspection phase in the second modified example> 7 is a diagram showing an example of a processing method in the inspection phase of the appearance analysis system 1 in the second modified example. The processing performed by the appearance analysis system 1 in steps S71 and S72 is similar to the processing performed in steps S31 and S32 in FIG. 3, and therefore a description thereof will be omitted.

[0079] Next, the appearance analysis system 1 uses the image segmentation engine Ec to estimate the degree of belonging {a_i(j)} of the local region {f_j} to each region of interest R_i (step S73). In the second modified example, the appearance analysis system 1 also estimates the degree of belonging {a_i(j)} of each local region f_j to each region of interest R_i to which it may belong in the inspection phase, as in the learning phase shown in Fig. 6. The method of estimating the degree of belonging is the same as in step S63 of Fig. 6.

[0080] Next, the appearance analysis system 1 uses the evaluation engine Ed_i to evaluate the inspection image 7 (step S74). In the inspection phase as well, the appearance analysis system 1 in the second modified example obtains individual evaluation values of the local regions f_j based on the degrees of belonging, as in step S64 of the learning phase shown in FIG. 6, and obtains an estimated evaluation value of the inspection image 7 based on the individual evaluation values.

[0081] The process executed by the appearance analysis system 1 in step S75 is similar to the process in step S34 in FIG. 3, and therefore a description thereof will be omitted.

[0082] <Processing method in the learning phase in the third modified example> The appearance analysis system 1 in the third modified example will be described below. The appearance analysis system 1 in this modified example has a GUI (Graphical User Interface) that accepts a user's designation to integrate or divide the regions of interest R_i. In the learning step, the appearance analysis system 1 optimizes the internal parameters of the image segmentation engine Ec based on the user's designation so that the image segmentation engine Ec determines how to divide the regions of interest R_i. The following describes the differences from the above-described embodiment.

[0083] 8 is a diagram showing an example of a processing method in the learning phase of the appearance analysis system 1 in the third modified example. The processing performed by the appearance analysis system 1 in step S81 is similar to the processing performed in step S21 in FIG. 2, and therefore a description thereof will be omitted.

[0084] Next, the appearance analysis system 1 accepts an input operation of the division conditions (step S82). Specifically, the appearance analysis system 1 accepts an input operation of the division method of the learning image 81 by the user as the division conditions.

[0085] As in the above-described embodiment, in this modification, the appearance analysis system 1 determines the type of region segmentation the image segmentation engine Ec will perform (the type of regions of interest R_i to divide into) through learning using the inspection accuracy rate as an evaluation value. When the inspection object is complex or there are only a small number of learning images, appropriate learning may not be performed. In this modification, if the user has prior knowledge about how to divide the regions of interest R_i, this knowledge can be incorporated into the image segmentation engine Ec as a rule.

[0086] 9 is a diagram illustrating an example of specifying a division condition. For example, suppose that the image division engine Ec during the kth iterative learning divides the learning image 81 into regions of interest R_1[k] to R_4[k] (images 811 to 814) (region division method 1). On the other hand, if it is known from the characteristics of the inspection object that the defect types occurring in the regions of interest R_1[k] (image 811) and R_3[k] (image 813) are the same, the region of interest R_3[k] can be merged with the region of interest R_1[k], and the combination of the two can be reset as the region of interest R_1[k] (image 815) (region division method 2). That is, the appearance analysis system 1 accepts an input operation to the GUI indicating that the region of interest R_3[k] is to be merged with the region of interest R_1[k].

[0087] If the original regions of interest R_1[k] and R_3[k] handle the same defect type, it is possible to inspect them using the same evaluation engine Ed_i. Furthermore, when training the evaluation engine Ed_i, combining the images of both can make training more efficient.

[0088] Furthermore, during the k-th iterative learning, it is also possible to specify that the image segmentation engine Ec should subdivide a region that belonged to one region of interest R_1[k] into two or more regions. If the region of interest R_1[k] contains multiple types of structures to be inspected and it is determined that these should be inspected using different evaluation engines, the image segmentation engine Ec can be trained to divide the region of interest.

[0089] For example, consider a case where the learning image 81 has been divided into regions of interest R_1[k] to R_4[k] (images 811 to 814) (region division method 1). If two gears, one large and one small, exist in the region of interest R_1[k] (image 811), and it is known that different defect types occur in the two gears due to the characteristics of the object of inspection, the region division method is specified to divide the region of interest R_1[k] (image 811) into two regions of interest R_1[k] (image 818) and R_2[k] (image 819) (region division method 3). That is, the appearance analysis system 1 accepts an input operation to the GUI indicating that the region of interest R_1[k] is to be divided.

[0090] As a result, the regions of interest R_1[k] (image 818) and R_2[k] (image 819) are inspected using different evaluation engines Ed_1 and Ed_2. Note that it is expected that the regions of interest R_i will also be divided in the inspection phase according to the division criteria for the regions of interest R_i specified in the learning phase.

[0091] Additionally, the appearance analysis system 1 may set regions that do not require inspection based on an input operation of the division conditions. The learning image 81 in FIG. 8 is divided into four regions of interest R_1, R_2, R_3, and R_4, and the region of interest R_4 is determined to be a region of interest that does not require inspection based on an input operation. For example, the appearance analysis system 1 excludes white pixels that are a focus of the region of interest R_4 that does not require inspection from the evaluation targets of the evaluation engine Ed_i.

[0092] The processing performed in steps S83 to S85 is the same as the processing performed in steps S22 to S24 in Fig. 2, and therefore description thereof will be omitted. Note that in step S83, the appearance analysis system 1 uses the image segmentation engine Ec to segment the learning image 81 so as to satisfy the segmentation conditions input in step S82.

[0093] In addition, in one example of the processing method according to FIG. 8, similarly to FIG. 2, each evaluation engine Ed_i performs evaluation for each pixel of each attention region R_i, for example. However, similarly to the evaluation in the first modified example shown in FIG. 4, the evaluation may be performed using a local region {f_i}, or similarly to the evaluation in the second modified example shown in FIG. 6, the evaluation may be performed using a local region {f_i} and a degree of belonging {a_i(j)} obtained by dividing the learning image 81.

[0094] As described above, by incorporating the user's domain knowledge of the inspection target, it becomes possible to optimize the inspection engine more appropriately.

[0095] <Processing method in the learning phase in the fourth modified example> The appearance analysis system 1 in the fourth modified example will be described below. In the image division step, the appearance analysis system 1 in this modified example receives input of design data (e.g., CAD (Computer-Aided Design) data) of the object to be inspected, and divides the inspection image into a plurality of regions of interest {R_i} based on region information in the design data. The following describes the differences from the above-described embodiment.

[0096] 10 is a diagram showing an example of a processing method in the learning phase of the appearance analysis system 1 in the fourth modified example. The processing performed by the appearance analysis system 1 in step S91 is similar to the processing performed in step S21 in FIG. 2, and therefore a description thereof will be omitted.

[0097] Next, the appearance analysis system 1 acquires design data of the article to be inspected (step S92). In this modification, the appearance analysis system 1 can utilize design data such as CAD, which is design information of the inspection object displayed in the learning image 91.

[0098] Next, the appearance analysis system 1 divides the training image 91 using the image division engine Ec (step S93). Similar to step S22 shown in Fig. 2, the appearance analysis system 1 divides the training image 91 into a plurality of regions of interest {R_i} (i = 1, ..., NR, NR: number of regions of interest). When dividing the training image 91, the appearance analysis system 1 uses the correspondence between the design data and the training image 91 acquired in step S92.

[0099] 11 is a diagram illustrating an example of a division method using design data. Design data 900 is an example of the design data acquired in step S92, and lines 901 to 904 indicate the contour lines of the design shape of the object to be inspected. Since structural information of the object to be inspected is known from the design data, by comparing an inspection image 905 with the design data 900, the structural information of the inspection image 905 can be grasped without using image recognition. Therefore, the region of interest {R_i} can be divided by referring to this structural information.

[0100] Generally, the imaging range of an inspection image is only a portion of the object being inspected, and imaging deviations occur, so by comparing the inspection image with the design data, the correspondence between the two can be determined. The dash-dotted line 901 indicates the matching position of the inspection image 905 in the design data 900, and the image 906 is drawn by overlapping the dotted lines 902 to 904 in the design data 900 onto the inspection image 905.

[0101] The division into regions of interest {R_i} may be performed based solely on structural information provided by the design data, or may be performed in conjunction with region segmentation using image processing. Consider dividing an inspection image 905 into three regions of interest R_1 to R_3 (images 907 to 909). It can be seen that it is difficult to recognize the structure corresponding to the dotted line 904, which is the contour line in the design data, in the inspection image 905. Therefore, it is effective to divide the regions of interest R_2 and R_3 based on the dotted line 904.

[0102] On the other hand, because the design data is merely a design shape, there may be deviations from the actual shape. For example, as shown in image 906, the rough positions of the two gears (dotted lines 902 and 903) between the design data and the inspection image match, but the shapes deviate slightly. Therefore, when dividing the region of interest R_1 (image 907), it is possible to extract the general shape from the design data, but the exact shape requires region segmentation using image processing.

[0103] In this modification, the design data is input to the image segmentation engine Ec and can be used to segment the learning image 91 into regions of interest. Note that in the inspection phase, which will be described later, the design data is also input to the image segmentation engine Ec and can be similarly used to segment the inspection image into regions of interest.

[0104] The description will return to Fig. 10. The processes performed in steps S94 and S95 are similar to the processes performed in steps S23 and S24 in Fig. 2, and therefore will not be described further. In addition, in one example of the processing method according to Fig. 10, similar to Fig. 2, each evaluation engine Ed_i performs evaluation for each pixel of each attention region R_i, for example, but evaluation may be performed using local region f_i, similar to the evaluation in the first modified example shown in Fig. 4, or evaluation may be performed using local regions {f_i} and degrees of belonging {a_i(j)} obtained by dividing the learning image 91, similar to the evaluation in the second modified example shown in Fig. 6.

[0105] In the third modified example described above, when a user designates integration or division of the regions of interest R_i, the user can designate the regions of interest R_i on the design data. For example, the user can designate that the dotted lines 902 and 903 in FIG. 11 be integrated into the same region of interest R_i, or conversely, designate that the dotted lines 902 and 903 be divided into different regions of interest R_i. This allows the image segmentation engine Ec to learn so that the regions of interest R_i specified by the user are output.

[0106] 8, the appearance analysis system 1 determines that the region of interest R_4 does not need to be inspected, and excludes it from evaluation using the evaluation engine Ed_i. Thus, in this modification, as in the third modification, the image segmentation engine Ec can determine the region of interest R_i that does not need to be inspected.

[0107] <Processing method in the inspection phase in the fourth modified example> Fig. 12 is a diagram showing an example of a processing method in the inspection phase of the appearance analysis system 1 in the fourth modified example. The processing performed by the appearance analysis system 1 in step S101 is the same as the processing performed in step S31 in Fig. 3. Moreover, the processing performed by the appearance analysis system 1 in steps S102 to S103 is the same as the processing performed in steps S92 to S93 in Fig. 10.

[0108] This modification makes it possible to efficiently divide learning images and inspection images by utilizing design data of the object to be inspected.

[0109] 13 is a diagram showing an example of the hardware configuration of the appearance analysis system 1. The appearance analysis system 1 has the above-mentioned image capture device 106 and a computer 100. The image capture device 106 is as described above. The computer 100 is a component that processes the appearance analysis method of this embodiment, and includes a processor 101, a memory resource 102, a GUI device 103, an input device 104, and a communication interface 105.

[0110] The processor 101 is a processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), but is not limited to these, and may be any device that can execute the above-described appearance analysis method. At least one processor 101 may be single-core or multi-core. Alternatively, it may be a broad processor device such as a circuit that is a collection of gate arrays whose functions are realized by a hardware description language that performs part or all of the processing (for example, an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit).

[0111] The storage resource 102 is a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), or a non-volatile memory (such as a flash memory), and functions as a storage area from which programs and data are temporarily read. The storage resource 102 may store a program (referred to as an appearance analysis program) that causes the processor 101 to execute the appearance analysis method described in the above embodiment.

[0112] The GUI device 103 is a device that displays a GUI, and is, for example, a display such as an OLCD (Organic Liquid Crystal Display) or a projector. The input device 104 is a device that accepts input operations from a user, and is, for example, an input device such as a keyboard, a mouse, or a touch panel. The input device 104 is not particularly limited as long as it is a component that can accept operations from a user, and the input device 104 and the GUI device 103 may be an integrated device.

[0113] The communication interface 105 is an interface that mediates the input and output of information, such as USB, Ethernet, or Wi-Fi. Note that the communication interface 105 is not limited to the example shown here, as long as it is an interface that can directly receive images from the imaging device 106 or that allows a user to transmit the images to the computer 100. Note that a portable non-volatile storage medium (such as a flash memory, DVD, CD-ROM, or Blu-ray disc) that stores the images can be connected to the communication interface, and the images can be stored in the computer 100.

[0114] The appearance analysis system 1 may include a plurality of computers 100, and may include a plurality of imaging devices 106. The aforementioned appearance analysis program can be distributed to the computer 100 by connecting a portable non-volatile storage medium storing the appearance analysis program to a communication interface. Alternatively, the appearance analysis program can be distributed to the computer 100 by a program distribution server. In this case, the program distribution server includes a storage resource 102 storing the appearance analysis program, a processor that performs distribution processing to distribute the appearance analysis program, and a communication interface device that can communicate with the communication interface device of the computer 100. The appearance analysis program distributed or delivered to the computer 100 has various functions realized by the processor 101.

[0115] In addition, as described above, the appearance analysis system 1 executes a learning phase in which the inspection engine is trained, and an inspection phase in which the inspection image is evaluated using the inspection engine trained in the learning phase. The processor 101 that executes the learning phase and the processor 101 that executes the inspection phase may be the same or different. If the processor 101 that executes the learning phase and the processor 101 that executes the inspection phase are different, the processor 101 that executes the learning phase can pass the internal parameters of the inspection engine to the processor 101 that executes the inspection phase.

[0116] As mentioned above, the embodiments described so far do not limit the invention according to the claims, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0117] In addition, in this embodiment, one test image is used as the input information for the inspection phase, but the present invention can also be applied to cases where there are multiple test images and multiple types of estimated evaluation values. That is, the evaluation engine can accept the input of many images, and many images can be used as evaluation targets.

[0118] Although the above describes various embodiments and variations of the present invention, the present invention is not limited to the above-described exemplary embodiments and includes various variations. For example, the above-described exemplary embodiments have been described in detail to facilitate understanding of the present invention, and the present invention is not limited to those including all of the components described herein. Furthermore, it is possible to replace part of the components of one exemplary embodiment with the components of another exemplary embodiment. It is also possible to add the components of another exemplary embodiment to the components of one exemplary embodiment. Furthermore, it is also possible to add, delete, or replace part of the components of each exemplary embodiment with other components. Furthermore, some or all of the above-described components, functions, processing units, processing means, etc. may be implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the control lines and information lines in the figures are only those considered necessary for explanation, and are not necessarily all shown. It is also possible to consider that almost all components are interconnected.

[0119] Furthermore, the functional configuration of the above-described appearance analysis system 1 is classified according to the main processing content for ease of understanding. The classification method and names of the components do not limit the present invention. As described above, the appearance analysis system 1 can be classified into more components according to the processing content. Furthermore, it is also possible to classify components so that one component performs even more processing.

[0120] As described above, this embodiment makes it possible to obtain highly accurate inspection results even for industrial products having diverse structures while suppressing the load on each inspection engine. [Explanation of symbols]

[0121] 1: Appearance analysis system, 2·4·6·8·9: Training image set, 21·41·61·81·91: Training images, 3·5·7·10·905: Inspection images, 100: Computer, 101: Processor, 102: Memory resource, 103: GUI device, 104: Input device, 105: Communication interface, 106: Imaging device, 900: Design data, 901: Dash-dotted line, 902·903·904: Dotted line, 906·907·908·909: Images

Claims

1. 1. A visual appearance analysis system comprising at least one processor and a memory resource, The processor: Acquire a training dataset including a plurality of training images showing the appearance of an article and accuracy evaluation values for the training images. * Using a segmentation model, the training image is divided into multiple regions of interest. An evaluation model for evaluating the region of interest is generated for each region of interest, and the evaluation results of the region of interest using each evaluation model are output. The segmentation model is trained by optimizing the internal parameters of the inspection engine using an evaluation value for the learning image, which is determined using the evaluation result and the correct evaluation value. Appearance analysis system.

2. The appearance analysis system according to claim 1, the training data set includes accuracy evaluation values for the plurality of training images; The processor: * The test accuracy rate obtained using the evaluation result and the correct evaluation value is used as the evaluation value to learn the division model. Appearance analysis system.

3. The appearance analysis system according to claim 2, The processor: The accuracy rate of the test of the learning image is calculated using the estimated evaluation value obtained by integrating the evaluation results for each of the attention areas constituting the learning image and the correct evaluation value. Appearance analysis system.

4. The appearance analysis system according to claim 1, The processor: * Outputting the evaluation result for the attention area obtained by dividing one unit of the learning image among the plurality of learning images. The segmentation model is trained using the evaluation value determined for each unit of training image. * Using the learned segmentation model, segment the other unit of the learning image. Appearance analysis system.

5. The appearance analysis system according to claim 1, The processor: * Obtain an inspection image showing the appearance of the item Segment the test image using the learned segmentation model * Output the evaluation results by the evaluation model for each of the divided attention areas. * Using the evaluation results, output the inspection results of the inspection image Appearance analysis system.

6. The appearance analysis system according to claim 1, The processor: * Divide the learning image to obtain multiple local regions * Outputting an evaluation result of the region of interest using information indicating the relationship between the local region and the region of interest and an evaluation of the local region. Appearance analysis system.

7. The appearance analysis system according to claim 6, The processor: *Estimate the degree of belonging to each region of interest for each local region * The evaluation result of the region of interest is output by weighting the evaluation of the local region based on the degree of belonging. Appearance analysis system.

8. The appearance analysis system according to claim 1, The processor: * Accepts designation of division conditions for a predetermined number of the regions of interest. * Divide the learning image so that the specified division conditions are met Appearance analysis system.

9. The appearance analysis system according to claim 1, The processor: * Acquire design information for the item * Dividing the learning images using the correspondence between the design information and the learning images Appearance analysis system.

10. An appearance analysis method using an appearance analysis system, a learning dataset acquisition step of acquiring a learning dataset including a plurality of learning images showing the appearance of an article and accuracy evaluation values for the learning images; a learning image segmentation step of segmenting the learning image into a plurality of interest regions using a segmentation model; a learning image evaluation step of generating an evaluation model for evaluating the region of interest for each of the regions of interest and outputting an evaluation result of the region of interest using each of the evaluation models; a learning step of learning the segmentation model by optimizing internal parameters of an inspection engine using an evaluation value for the learning image, which is determined using the evaluation result and the correct evaluation value.

11. The appearance analysis method according to claim 10, the training data set includes a correct evaluation value for each of the plurality of training images, and in the training step, the segmentation model is trained using an inspection accuracy rate obtained using the evaluation result and the correct evaluation value as the evaluation value.

12. The appearance analysis method according to claim 11, In the learning step, the inspection accuracy rate of the learning image is calculated using an estimated evaluation value that integrates the evaluation results for each of the attention areas that constitute the learning image and the correct evaluation value.

13. The appearance analysis method according to claim 10, In the learning image evaluation step, the evaluation result for the attention area obtained by dividing one unit of the learning image among the plurality of learning images is output; In the learning step, the segmentation model is learned using the evaluation value determined for each unit of learning image; In the learning image segmentation step, the learned segmentation model is used to segment another unit of the learning image.

14. The appearance analysis method according to claim 10, an inspection image acquisition step of acquiring an inspection image showing the appearance of the article; a test image segmentation step of segmenting the test image using the segmentation model learned in the learning step; an inspection image evaluation step of outputting an evaluation result based on the evaluation model for each of the regions of interest obtained in the inspection image division step; an inspection result output step of outputting an inspection result of the inspection image using the evaluation result; An appearance analysis method comprising:

15. The appearance analysis method according to claim 10, a local region acquisition step of dividing the learning image to obtain a plurality of local regions, In the learning image evaluation step, an evaluation result of the region of interest is output using information indicating the relationship between the local region and the region of interest and an evaluation of the local region.

16. The appearance analysis method according to claim 15, a degree of belonging estimation step of estimating a degree of belonging to each of the regions of interest for each of the local regions; In the learning image evaluation step, an evaluation result of the region of interest is output by weighting the evaluation of the local region based on the degree of belonging.

17. The appearance analysis method according to claim 10, a division specifying step of receiving a specification of a division condition for a predetermined plurality of the regions of interest, In the learning image dividing step, the learning image is divided so as to satisfy the division condition specified in the division specifying step.

18. The appearance analysis method according to claim 10, a design information acquisition step of acquiring design information of the article; In the learning image dividing step, the learning image is divided using a correspondence between the design information and the learning image.

19. 19. A method for visual analysis according to any one of claims 10 to 18, program.

Images