Image generation method and appearance inspection device

Abstract

The present invention suppresses inaccuracy of determination of the presence or absence of an abnormality by machine learning. This appearance inspection device is provided with a processor that acquires a plurality of appearance images in which the appearance of an object to be inspected is captured, and generates a statistical distribution indicating the variation in the characteristics of the respective appearance images when the plurality of appearance images are taken as a whole. The processor generates an additional image in which the appearance is captured on the basis of the deviation indicated by the statistical distribution, and generates a learned model by machine learning using learning data including the plurality of appearance images and the additional image.

Description

Technical Field

[0001] The present invention relates to an image generation method and an appearance inspection device. The present invention claims priority to Japanese Patent Application No. 2022-097546 filed on June 16, 2022, and the contents described in the Japanese Patent Application No. 2022-097546 are incorporated into the present application by reference for designated countries that allow incorporation by reference. Background Art

[0002] In production lines where products are produced, image recognition technology is used to identify whether a product or its components are defective during production, so as to detect defects at an early stage. In particular, with the development of machine learning in recent years, it is also popular to use machine learning to detect defects. In this case, after a machine learning model such as CNN (Convolutional Neural Network) learns various learning images, the machine learning model that has learned the image determines whether the product has defects based on the image of the actual product.

[0003] However, if the learning image has little variation, the learning of the machine learning model is insufficient and the determination of whether there is a defect is inaccurate. Therefore, Patent Document 1 proposes the following technology: extracting scars from the learning image, generating local images obtained by subjecting the scars to various deformations, and synthesizing them into a synthetic target image as a new learning image, thereby increasing the variation of the learning image.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-109563 Summary of the invention

[0007] However, since the frequency of product defects varies depending on the defect, if the changes in the learning image are increased without considering this frequency, there is a concern that the machine learning model may mistakenly detect defects or miss defects, resulting in inaccurate judgment results of the machine learning model.

[0008] The present invention has been made in view of such a situation, and an object of the present invention is to prevent the determination of the presence or absence of an abnormality based on machine learning from becoming inaccurate.

[0009] The present application includes a plurality of means for solving at least a part of the above-mentioned problems, and examples thereof are as follows.

[0010] In order to solve the above-mentioned problems, an appearance inspection device of one scheme of the present invention is an appearance inspection device equipped with a processor, wherein the processor obtains multiple appearance images that capture the appearance of an inspection object, generates a statistical distribution, and the statistical distribution represents the deviation of the characteristics of each of the above-mentioned appearance images when the above-mentioned multiple appearance images are taken as a whole. The processor generates an additional image that captures the above-mentioned appearance based on the above-mentioned deviation shown in the above-mentioned statistical distribution, and generates a learned model through machine learning using learning data including the above-mentioned multiple appearance images and the above-mentioned additional image.

[0011] Effects of the Invention

[0012] According to the present invention, it is possible to suppress inaccuracy in the determination of the presence or absence of an abnormality based on machine learning.

[0013] Other problems, structures, and effects than those described above will become clear from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram showing an example of the functional configuration of the appearance inspection device.

[0015] Figure 2 This is a schematic diagram showing an example of a flowchart of the appearance inspection method according to the first embodiment.

[0016] Figure 3 This is a flowchart showing an example of the additional image generation process according to the first embodiment.

[0017] Figure 4 This is a schematic diagram for explaining an example of the additional image generation process according to the first embodiment.

[0018] Figure 5 This is a schematic diagram for explaining an example of the additional image generation process according to the first embodiment when a position where a component is deformed is adopted as a feature of an appearance image.

[0019] Figure 6 This is a schematic diagram for explaining an example of the additional image generation process according to the first embodiment when the brightness of the entire image of the external image is adopted as the feature of the external image.

[0020] Figure 7 This is a schematic diagram for explaining an example of the additional image generation process according to the first embodiment when the contrast of the appearance image is adopted as the feature of the appearance image.

[0021] Figure 8 This is a schematic diagram for explaining an example of the additional image generation process according to the first embodiment when the noise intensity of the external image is adopted as the feature of the external image.

[0022] Fig. 9 This is a schematic diagram showing an example of a method for generating a learned model according to the first embodiment.

[0023] Fig.10 This is a schematic diagram showing an example of the inspection method according to the first embodiment.

[0024] Fig.11 It is a schematic diagram showing a display example of the display unit according to the first embodiment.

[0025] Fig.12 This is a schematic diagram for explaining an example of the additional image generation process in the second embodiment.

[0026] Fig.13 This is a schematic diagram showing an example of a method for generating a learned model in the third embodiment.

[0027] Fig.14 This is a schematic diagram showing an example of the inspection method in the third embodiment.

[0028] Fig.15 It is a diagram showing an example of the hardware configuration of the appearance inspection device according to the first to third embodiments. DETAILED DESCRIPTION

[0029] Hereinafter, an embodiment of the present invention will be described based on the accompanying drawings. It should be noted that, in all the accompanying drawings used to illustrate the embodiments, the same figure mark is marked on the same component in principle and its repeated description is appropriately omitted. In addition, in the following embodiments, there is no doubt that, except for the cases that are particularly clearly stated and the cases that are considered to be obviously necessary in principle, its constituent elements (including element steps, etc.) are not necessary. In addition, when it is said that "composed of A", "formed by A", "having A", "including A", except for the cases that are particularly clearly stated that it is only the main purpose of this element, etc., there is no doubt that elements other than this are not excluded. Similarly, in the following embodiments, when it comes to the shape, positional relationship, etc. of the constituent elements, etc., except for the cases that are particularly clearly stated and the cases that are considered to be obviously not the case in principle, etc., shapes that are substantially similar or similar to the shape, etc. are included.

[0030] <First embodiment>

[0031] Figure 1 1 is a schematic diagram showing an example of the functional configuration of the appearance inspection device 100 according to the present embodiment.

[0032] The appearance inspection device 100 is a device that inspects whether or not there is any abnormality in the appearance of a component as an inspection target object, for example, and includes a processing unit 110 , a storage unit 120 , an input unit 130 , a display unit 140 , and an imaging unit 150 .

[0033] The input unit 130 is an input device such as a keyboard or a mouse for receiving various inputs from a user. The display unit 140 is a display device such as a liquid crystal display or an organic EL (ElectroLuminescence) display for displaying the inspection results of the appearance of a component.

[0034] The imaging unit 150 is an imaging device such as a camera that captures the appearance of a component that is an inspection target object and stores an appearance image 121 a capturing the appearance in the storage unit 120 .

[0035] The storage unit 120 is a functional unit that stores an appearance image DB (Database) 121 , an extended learning image DB 122 , a learned parameter DB 123 , and a program 124 .

[0036] The appearance image DB 121 is a database storing appearance images 121a and attribute information 121b of components captured by the imaging unit 150. The attribute information 121b includes information indicating whether the component captured in the appearance image 121a is normal or abnormal, the type of abnormality, and the position where the abnormality occurs in the component.

[0037] The extended learning image DB 122 is a database storing extended learning images 122a and attribute information 122b. The extended learning image 122a is learning data when learning a machine learning model that detects whether a component has an abnormality. As an example, the extended learning image 122a is an image including the aforementioned appearance image 121a and an additional image generated by the image generation unit 113 described later. By using not only the appearance image 121a but also the additional image as learning data as described above, the variation of the learning data can be increased.

[0038] The attribute information 122 b is information including information indicating whether the component captured in the extended learning image 122 a is normal or abnormal, the type of abnormality, and the position where the abnormality occurs in the component.

[0039] The learned parameter DB 123 is a database that stores internal parameters of the machine learning model that has been learned using the expanded learning image 122 a as learning data.

[0040] The program 124 is a visual inspection program of the present embodiment. When the visual inspection apparatus 100 executes the program 124, each function of the processing unit 110 is realized.

[0041] The processing unit 110 is a functional unit that controls each unit of the appearance inspection device 100. As an example, the processing unit 110 includes an image acquisition unit 111, a statistical distribution generation unit 112, an image generation unit 113, a learning unit 114, and an inspection unit 115.

[0042] The image acquisition unit 111 is a functional unit that acquires the appearance image 121 a from the appearance image DB 121 .

[0043] The statistical distribution generating unit 112 is a processing unit that generates a statistical distribution that indicates the deviation of the features of each appearance image 121a when the plurality of appearance images 121a are taken as a whole. As an example of such a feature, there is the amount of deformation of the component to be inspected or the position where the component is deformed, as described later. In addition, the brightness, contrast, and noise intensity of each appearance image 121a are also examples of features.

[0044] The image generation unit 113 is a functional unit that generates an additional image showing the appearance of a component based on the deviation of the feature indicated by the statistical distribution generated by the statistical distribution generation unit 112 , and stores the additional image as an extended learning image 122 a in the extended learning image DB.

[0045] The learning unit 114 is a functional unit that generates a learned model by machine learning using the expanded learning image 122 a as learning data.

[0046] The inspection unit 115 is a functional unit that uses the learned model to inspect whether the appearance of the inspection target component has any abnormality. In addition, the inspection unit 115 can also instruct the display unit 140 to display the inspection result, etc., so that the user who views the display unit 140 can understand the inspection result.

[0047] Figure 2 It is a schematic diagram showing an example of a flowchart of the appearance inspection method according to the present embodiment.

[0048] First, the image acquisition unit 111 acquires one or more appearance images 121a from the appearance image DB 121 (step S21). In this example, the image acquisition unit 111 randomly acquires one or more normal appearance images 121a from all appearance images 121a stored in the appearance image DB 121. Whether the appearance image 121a is normal can be determined by the image acquisition unit 111 based on the attribute information 121b corresponding to the appearance image 121a. In addition, the image acquisition unit 111 also acquires the attribute information 121b corresponding to each of the acquired appearance images 121a from the appearance image DB 121.

[0049] Next, the statistical distribution generating unit 112 and the image generating unit 113 perform a process of generating an additional image (step S22 ). The details of this generation process will be described later.

[0050] Next, the learning unit 114 generates a learned model by machine learning using the expanded learning image 122 a as learning data (step S23 ). The details of this step will be described later.

[0051] Next, the inspection unit 115 uses the learned model to inspect whether the component has an abnormality (step S24). Here, the inspection unit 115 obtains the inspection image of the component captured by the camera unit 150, and detects whether the component captured in the inspection image has an abnormality. The details will be described later.

[0052] Through the above steps, the basic processing of the appearance inspection method of this embodiment is completed.

[0053] Next, the additional image generation process in step S22 will be described. Figure 3 : is a flowchart showing an example of the generation process of the additional image. Figure 4 2 is a schematic diagram for explaining an example of the generation process of the additional image. The generation process of the additional image is an example of the image generation method.

[0054] First, if Figure 3 As shown, the statistical distribution generating unit 112 generates a statistical distribution indicating the variation of the features of each appearance image 121 a when each appearance image 121 a acquired in step S21 is regarded as a population (step S31 ).

[0055] exist Figure 4 In the example of , each appearance image 121a that becomes the population of the statistical distribution is represented by the image set 401. The statistical distribution generating unit 112 generates the reference value image 402 based on each appearance image 121a included in the image set 401. In this example, the statistical distribution generating unit 112 calculates the median of the pixel values ​​of each appearance image 121a for each position, and generates a median image whose pixel values ​​at each position become the median as the reference value image. In addition, the average value of the pixel values ​​may be used instead of the median. In addition, the median may be calculated at the position of each pixel, or may be calculated at an area including a plurality of pixels. Next, the statistical distribution generating unit 112 obtains the difference in pixel values ​​between each appearance image 121a and the reference value image 402 for each position. The difference calculated at a certain position is the deformation amount of the component captured in the appearance image 121a at the position when the reference value image 402 is used as the reference. Then, the statistical distribution generating unit 112 obtains the cumulative value obtained by summing the differences of each appearance image 121a for each position. The position where the cumulative value is large is the position where the deviation of the deformation amount is large when the image set 401 is taken as a whole. Conversely, the position where the cumulative value is small is the position where the deviation of the deformation amount is small when the image set 401 is taken as a whole. The statistical distribution generation unit 112 generates a statistical distribution 403 by mapping such cumulative values. In the statistical distribution 403, the position with a dark color is the position where the deviation of the deformation amount is large, and the position with a light color is the area where the deviation of the deformation amount is small.

[0056] Here, the statistical distribution generating unit 112 generates the statistical distribution 403 indicating the deviation of the deformation amount at each position of the component, but the statistical distribution generating unit 112 may generate a plurality of statistical distributions 403. For example, the statistical distribution generating unit 112 may generate a statistical distribution indicating the deviation of the shape of the component as described later, or may generate a statistical distribution indicating the deviation of each of the brightness, contrast, and noise intensity of each appearance image 121a.

[0057] Refer again Figure 3 Next, the image generation unit 113 selects one or more of the plurality of statistical distributions generated by the statistical distribution generation unit 112 (step S32). Here, for simplicity of description, the case where the image generation unit 113 selects the statistical distribution 403 indicating the deviation of the deformation amount is described as an example.

[0058] Next, the image generation unit 113 generates an additional image based on the selected statistical distribution 403 (step S33). Figure 4 Provide explanation.

[0059] As shown in the statistical distribution 403, the deviation of the deformation amount varies depending on the position of the component. It is considered that the appearance image 112a of the component deformed at the position with a large deviation is statistically smaller in number and has less variation than the appearance image 112a of the component deformed at the position with a small deviation.

[0060] Therefore, the image generation unit 113 generates an additional image 404 obtained by deforming the appearance image 121a included in the appearance image DB 121 so as to increase the variation of the deformation. The amount of deformation and the number of additional images 404 are determined by the image generation unit 113 based on the statistical distribution 403. For example, the image generation unit 113 increases the amount of deformation and increases the number of additional images 404 for a distribution area with a larger deviation in the statistical distribution 403. In addition, the appearance image 121a subjected to the deformation process may be one appearance image 121a arbitrarily selected from the appearance image DB 121, or may be a plurality of appearance images 121a. Thus, the variation of the additional image 404 with a large deviation can be increased.

[0061] In addition, the image generation unit 113 makes the deformation amount in the additional image 404 fall within the distribution range of the statistical distribution 403 of the normal appearance image 121a. As a result, the parts captured in the additional image 404 can be regarded as normal. In addition, the appearance image 121 that is transformed into the additional image 404 by the deformation process can be normal or abnormal. In addition, the image generation unit 113 can also generate an abnormal additional image 404 by performing deformation processing on the normal or abnormal appearance image 121 in this way. This will be described later. Figures 5 to 8The same is true in the example.

[0062] Refer again Figure 3 . Next, the image generation unit 113 saves all the appearance images 121a included in the appearance image DB and all the additional images 404 generated in step S33 in the extended learning image DB 122 (step S34). In addition, the image generation unit 113 saves the attribute information 121b of the appearance image 121a in the extended learning image DB 122, and also saves the attribute information 122b of the additional image 404 in the extended learning image DB 122. In addition, since the normal appearance image 121a is set as the image set 401 here, the parts captured in the additional image 404 can also be regarded as normal as described above. Therefore, the attribute information 122b of the additional image 404 becomes information indicating that the parts are normal.

[0063] Through the above steps, the basic processing in the additional image generation processing is completed.

[0064] The appearance image 121a and the additional image 404 in the extended learning image DB 122 generated as described above become the learning data when the learning unit 114 generates the learning completed model. In this embodiment, as described above, the number of additional images 404 in the distribution area where the deviation of the deformation amount is large in the statistical distribution 403 is increased, so the change of the learning data in the distribution area increases. As a result, the learning unit 114 can accurately learn the recognition boundary that distinguishes between normal and abnormal based on the learning data. As a result, the possibility of the inspection unit 115 mistakenly judging a component with a large deformation within the normal range as abnormal can be reduced, and the judgment of the presence or absence of abnormality based on machine learning can be suppressed from becoming inaccurate.

[0065] Figure 5 This is a schematic diagram for explaining an example of a process of generating the additional image 404 when a position where a component has been deformed is adopted as a feature of the appearance image 121 a .

[0066] exist Figure 5 In the example of , the image set 501 is used to represent a plurality of appearance images 121a that constitute a population of statistical distribution. Furthermore, in step S31, the statistical distribution generating unit 112 extracts the shape of each appearance image 121a included in the image set 501 by image processing such as contour extraction. Next, the statistical distribution generating unit 112 generates a median image representing the median of the shapes of the components in the image set 501 as a reference value image 502. In addition, the statistical distribution generating unit 112 may generate an average value image representing the average value of the shapes of the components involved in the image set 501 instead of the median image.

[0067] The statistical distribution generating unit 112 calculates the difference in pixel value between each appearance image 121a included in the image set 501 and the reference value image 502 for each position, thereby calculating the position where the component has been deformed for each appearance image 121a with reference to the reference value image 502. Then, the statistical distribution generating unit 112 generates a statistical distribution 503 indicating the deviation of the position where the deformation has occurred with the image set 501 as the matrix.

[0068] In this example, an abnormal template 505 storing various defect images such as scratches and stains is pre-stored in the storage unit 120. Then, in step S33, the image generation unit 113 processes the appearance image 121a included in the appearance image DB 121 by image processing, etc., thereby generating images in which the shape of the component is deformed in various ways within the normal range. For example, the image generation unit 113 generates a large number of images in which the more the distribution area with a larger deviation in the statistical distribution 503, the larger the deformation.

[0069] The image generation unit 113 may obtain any one appearance image 121a from the appearance image DB 121 and perform the above-mentioned image processing on the appearance image 121a. Alternatively, the image generation unit 113 may obtain a plurality of appearance images 121a from the appearance image DB 121 and perform image processing on each appearance image 121a. Then, the image generation unit 113 generates an additional image 504 by superimposing (combining) the defect image of the abnormal template 505 on the image subjected to the image processing.

[0070] The position where the defect image is superimposed on the appearance image 121a is set to the position where the deformation occurs in the appearance image 121a. For example, when the edge of the component is deformed in a certain appearance image 121a, the image generation unit 113 superimposes the defect image on the edge.

[0071] The image generating unit 113 may perform any one of rotation, enlargement, and reduction or a combination of these processes on the defect image of the abnormal template 505 , and superimpose the processed image on the appearance image 121 a .

[0072] The additional image 504 includes a defective image, but the image used as the basis of the additional image 504 is an image in which the shape of the component is deformed within the normal range. Therefore, the attribute information 122b of the additional image 504 stored in the extended learning image DB 122 in step S34 is information indicating normality.

[0073] The appearance image DB 121 includes appearance images 121a in which the positions where the parts are deformed have various deviations, but Figure 4Similarly to the example of , it is considered that the number of appearance images 121 a in the distribution area with large deviation in the statistical distribution 503 is smaller and the variation is less than that of the appearance images 121 a in the distribution area with small deviation.

[0074] Therefore, by generating the additional image 504 as in this example, the variation of the image can be increased, and the variation of the learning data becomes richer. Moreover, by superimposing the defect image on the appearance image 121a, the combination of deformation and defect becomes richer, and the variation of the learning data further increases.

[0075] Thus, the learning unit 114 can accurately learn the identification boundary for distinguishing normal from abnormal based on the learning data that there is a defect at the position where the deformation occurs. As a result, the possibility that the inspection unit 115 mistakenly judges the shape deviation within the normal range as abnormal can be reduced.

[0076] Figure 6 This is a schematic diagram for explaining an example of the generation process of the additional image when the brightness of the entire image of the external image 121 a is adopted as the feature of the external image 121 a .

[0077] exist Figure 6 In the example of FIG. 6 , the image set 601 is used to represent the plurality of appearance images 121a that constitute the population of the statistical distribution. In step S31, the statistical distribution generating unit 112 calculates the average brightness obtained by averaging the brightness of the entire image of the appearance image 121a in the image set 601 as the reference brightness. In addition, the statistical distribution generating unit 112 may calculate the median of the brightness in the image set 601 as the reference brightness instead of the average brightness.

[0078] In addition, the statistical distribution generating unit 112 calculates the difference between the brightness of the entire image and the reference brightness for each appearance image 121a included in the image set 601, and generates a statistical distribution 602 indicating the variation of the difference. The horizontal axis of the statistical distribution 602 is the difference between the reference brightness and the brightness, and the vertical axis is the number of appearance images 121a.

[0079] Then, in step S33, the image generation unit 113 appropriately selects an appearance image 121a from the appearance image DB 121. The number of appearance images 121a selected may be one or more. Then, the image generation unit 113 performs brightness correction processing on the selected appearance images 121a, thereby generating various additional images 604 in a number corresponding to the statistical distribution 602 such that the difference between the reference brightness and the brightness falls within the distribution range of the statistical distribution 602.

[0080] For example, the image generating unit 113 increases the number of additional images 604 for a distribution region where the number of appearance images 121a is smaller in the statistical distribution 602. This can increase the variation of images with a smaller number in the statistical distribution 602.

[0081] Furthermore, since the difference between the brightness of the additional image 604 and the reference brightness is within the distribution range of the statistical distribution 602 of the normal observation image 121a, the component captured in the additional image 604 can be regarded as normal. Therefore, the attribute information 122b of the additional image 604 stored in the extended learning image DB 122 in step S34 becomes information indicating normality.

[0082] According to this example, the variation of images with a small number of images in the statistical distribution 602 increases, so the variation of brightness in the learning data becomes richer and the bias of brightness in the learning data decreases. Therefore, the learning unit 114 can learn the appearance of normal parts in consideration of the color of the parts to be inspected. As a result, the possibility that the inspection unit 115 mistakenly judges a normal part as abnormal due to the difference in color can be reduced.

[0083] Figure 7 This is a schematic diagram for explaining an example of the generation process of the additional image when the contrast of the external image 121 a is adopted as the feature of the external image 121 a .

[0084] exist Figure 7 In the example of , the image set 701 is used to represent a plurality of appearance images 121a that form a population of statistical distribution. In addition, in step S31, the statistical distribution generating unit 112 calculates a brightness histogram 702 that associates brightness values ​​with the number of pixels for each appearance image 121a included in the image set 701. Next, the statistical distribution generating unit 112 calculates an average brightness histogram obtained by averaging the brightness histograms 702 in the image set 701 as a reference histogram 703.

[0085] The statistical distribution generating unit 112 calculates the reference contrast, for example, based on the difference between the maximum brightness and the minimum brightness in the reference histogram 703. Similarly, the statistical distribution generating unit 112 calculates the contrast of each appearance image 121a included in the image set 701 based on the difference between the maximum brightness and the minimum brightness in each brightness histogram 702. Then, the statistical distribution generating unit 112 calculates the difference between the contrast of each appearance image 121a and the reference contrast, and generates a statistical distribution 704 indicating the deviation of the difference. The horizontal axis of the statistical distribution 704 is the difference between the reference contrast and the contrast, and the vertical axis is the number of appearance images 121a.

[0086] Then, in step S33, the image generation unit 113 appropriately selects an appearance image 121a from the appearance image DB 121. The number of selected appearance images 121a may be one or more. Then, the image generation unit 113 performs contrast correction processing on the selected appearance images 121a, thereby generating various additional images 705 in a number corresponding to the statistical distribution 704 such that the difference between the reference contrast and the contrast falls within the distribution range of the statistical distribution 704.

[0087] As an example, the image generating unit 113 increases the number of additional images 705 having the deviation in a distribution region where the number of appearance images 121a is small in the statistical distribution 704. This can increase the variation of images with a small number in the statistical distribution 704.

[0088] Furthermore, since the difference between the contrast of the additional image 705 and the reference contrast is within the distribution range of the statistical distribution 704 of the normal appearance image 121a, the component captured in the additional image 705 can be regarded as normal. Therefore, the attribute information 122b of the additional image 705 stored in the extended learning image DB 122 in step S34 becomes information indicating normality.

[0089] According to this example, the variation of images with a small number of images in the statistical distribution 704 increases, so the variation of contrast in the learning data becomes richer, and the bias of contrast in the learning data decreases. Therefore, the learning unit 114 can learn the appearance of normal parts in consideration of the contrast of the image. As a result, the possibility that the inspection unit 115 erroneously judges a normal part as abnormal due to the difference in the contrast of the image can be reduced.

[0090] Figure 8 This is a schematic diagram for explaining an example of the processing for generating an additional image when the noise intensity of the external image 121 a is adopted as the feature of the external image 121 a .

[0091] exist Figure 8 In the example of , a plurality of appearance images 121 a constituting a population of statistical distribution are represented by the image set 801 . Then, in step S31 , the statistical distribution generating unit 112 generates a denoised image 802 obtained by removing noise from each appearance image 121 a included in the image set 801 .

[0092] Next, the statistical distribution generating unit 112 generates a difference image between each appearance image 121a of the image set 801 and the corresponding denoised image 802, and calculates the average noise intensity in the entire image of the difference image. Then, the statistical distribution generating unit 112 calculates the average of the average noise intensity in the image set 801 as the reference noise intensity. In addition, the median of the average noise intensity in the image set 801 may be set as the reference noise intensity. Furthermore, the statistical distribution generating unit 112 calculates the difference between the average noise intensity of each appearance image 121a and the reference noise intensity, and generates a statistical distribution 803 indicating the deviation of the difference. The horizontal axis of the statistical distribution 803 is the difference between the reference noise intensity and the average noise intensity, and the vertical axis is the number of appearance images 121a.

[0093] Then, in step S33, the image generation unit 113 appropriately selects an appearance image 121a from the appearance image DB 121. The number of selected appearance images 121a may be one or more. Then, the image generation unit 113 generates various additional images 804 such that the difference between the reference noise intensity and the average noise intensity falls within the distribution range of the statistical distribution 803 by performing noise addition processing on the selected appearance images 121a, with the number of images corresponding to the statistical distribution 803.

[0094] As an example, the image generating unit 113 increases the number of additional images 804 for a distribution region where the number of appearance images 121a is smaller in the statistical distribution 803. This can increase the variation of images with a smaller number in the statistical distribution 803.

[0095] Furthermore, since the difference between the average noise intensity of the additional image 804 and the reference noise intensity is within the distribution range of the statistical distribution 803 of the normal appearance image 121a, the component captured in the additional image 804 can be regarded as normal. Therefore, the attribute information 122b of the additional image 804 stored in the extended learning image DB 122 in step S34 becomes information indicating normality.

[0096] According to this example, since the variation of images with a small number of images increases in the statistical distribution 803, the variation of the average noise intensity in the learning data becomes richer, and the bias of the average noise intensity in the learning data decreases. Therefore, the learning unit 114 can learn the appearance of normal parts in consideration of the average noise intensity of the image. As a result, the possibility that the inspection unit 115 erroneously judges a normal part as abnormal due to the variation of the noise intensity caused by the shooting environment can be reduced.

[0097] Next, explain Figure 2 The method for generating the learning completed model in step S23.

[0098] Fig. 9This is a schematic diagram showing an example of a method for generating a learned model.

[0099] First, the learning unit 114 acquires one or more expanded learning images 122a from the expanded learning image DB 122. Here, the set of expanded learning images 122a acquired in this way is referred to as a learning image set 901.

[0100] Next, the learning unit 114 inputs each extended learning image 122a of the learning image set 901 as learning data to a machine learning model 902 such as a CNN. As a result, the machine learning model 902 determines whether the component captured in the extended learning image 122a is normal or abnormal based on its internal parameters, and outputs an estimated evaluation value 903 including the determination result. In addition to the determination result of whether it is normal or abnormal, the estimated evaluation value 903 also includes the type of abnormality and the location where the abnormality occurred.

[0101] Next, the learning unit 114 calculates the error between the estimated evaluation value 903 and the attribute information 122b, and updates the internal parameters of the machine learning model 902 so as to minimize the error. Then, the learning unit 114 stores the updated internal parameters in the learned parameter DB 123.

[0102] Then, the machine learning model 902 uses the internal parameters stored in the learned parameter DB to output the estimated evaluation value 903. The machine learning model 902 that outputs the estimated evaluation value 903 using the internal parameters stored in the learned parameter DB is a learned model.

[0103] Through the above steps, the basic processing for generating a learned model is completed. According to this example, the extended learning image 122a of the learning image set 901 is selected from the extended learning image DB 122 that has been changed and increased by adding the number of images corresponding to any one of the above-mentioned statistical distributions 403, 503, 602, 704, and 803, and is used as the learning data of the machine learning model 902. As a result, the machine learning model 902 learns the learning data rich in changes, so the possibility of the learned machine learning model 902 making an erroneous judgment can be reduced.

[0104] Next, explain Figure 2 The method of checking in step S24.

[0105] Fig.10 1 is a schematic diagram showing an example of an inspection method. First, the inspection unit 115 acquires an inspection image 1001 of a component captured by the imaging unit 150 .

[0106] Next, after the inspection unit 115 reads the internal parameters from the learned parameter DB 123 into the machine learning model 902, it inputs the inspection image 1001 into the machine learning model 902. Thus, the machine learning model 902, which is a learned model, determines whether the component captured in the inspection image 1001 is normal or abnormal based on its internal parameters, and outputs an estimated evaluation value 903 including its determination result.

[0107] Then, if the estimated evaluation value 903 indicates normality, the inspection unit 115 determines that the component has no abnormality (OK). On the other hand, if the estimated evaluation value 903 indicates abnormality, the inspection unit 115 determines that the component has an abnormality (NG).

[0108] The above steps complete the basic processing when inspecting parts.

[0109] Fig.11 1 is a schematic diagram showing a display example of the display unit 140. In this example, the display unit 140 displays the appearance image 121a in the appearance image DB 121. In addition, the display unit 140 may also display the distinction between normal and abnormal shown in the attribute information 121b and the type of abnormality such as "scratches" in the case of an abnormality together with the appearance image 121a.

[0110] The display unit 140 also displays the expanded learning image 122a in the expanded learning image DB 122. At this time, the display unit 140 may also display the distinction between normal and abnormal indicated by the attribute information 122b and the type of abnormality such as "stain" in the case of an abnormality, together with the expanded learning image 122a.

[0111] Furthermore, the display unit 140 also displays the statistical distribution in each of the appearance image DB 121 and the extended learning image DB 122. Here, as the statistical distribution, the display unit 140 displays the statistical distributions 704 and 803 selected to generate the additional image in the extended learning image DB 122. In this case, the additional image 705 generated using the statistical distribution 704 and the additional image 804 generated using the statistical distribution 803 are included in the extended learning image 122a of the extended learning image DB 122.

[0112] In the appearance image DB 121, as shown in the statistical distribution 704, the greater the difference between the reference contrast and the contrast, the fewer the number of images, and the less the change in the appearance image 121a. On the other hand, in the extended learning image DB 122, as shown by the upward arrow, the deviation of the statistical distribution 704 is eliminated, and the number of images becomes the same regardless of the size of the contrast. The same is true for the statistical distribution 803. As a result, it is possible to obtain an extended learning image 122a that is rich in changes regardless of the size of the contrast and the average noise intensity. As a result, by making the machine learning model 902 learn the extended learning image 122a as learning data, a learning model with fewer misjudgments can be obtained.

[0113] Furthermore, the display unit 140 also displays the inspection result obtained by the inspection unit 115. In this example, the display unit 140 displays the inspection image 1001 and the estimated evaluation value 903. The estimated evaluation value 903 includes the probability of being normal and the probability of including abnormalities such as "stains" and "scratches". In addition, when there is an abnormality, the display unit 140 also displays the defect position.

[0114] This allows the user to understand the location of the defect and the type of abnormality.

[0115] <Second embodiment>

[0116] In the first embodiment, if Figures 4 to 8 As shown in FIG. 1 , the image acquisition unit 111 acquires a normal appearance image 121a from the appearance image DB 121. In contrast, in the present embodiment, the image acquisition unit 111 acquires an abnormal appearance image 121a from the appearance image DB 121 as follows.

[0117] Fig.12 This is a schematic diagram for explaining an example of the generation process of the additional image in this embodiment.

[0118] First, in Figure 1 In step S21, the image acquisition unit 111 randomly acquires one or more abnormal appearance images 121a and their attribute information 121b from all appearance images 121a stored in the appearance image DB 121. The acquired appearance images 121a are images of a statistical distribution population, which are represented by an image set 1201 below.

[0119] Next, in step S31, the statistical distribution generating unit 112 identifies pixels at the positions of the abnormalities indicated by the attribute information 121b for the acquired appearance image 121a. Next, the statistical distribution generating unit 112 generates a statistical distribution 1202 indicating the distribution of the pixels identified in the image set 1201. The statistical distribution 1202 is a distribution that indicates the frequency of abnormalities caused by deformation using the depth of color, and the darker the color, the more frequent the abnormality, the more likely it is to deform, and the greater the deviation in the amount of deformation.

[0120] The appearance image DB 121 includes abnormal appearance images 121a having various deformation amounts. However, most of the abnormal appearance images 121a have deformation amounts near the median value in the image set 1201 , and the number of abnormal appearance images 121a having large deformation amounts is statistically small.

[0121] Therefore, in step S31, the image generation unit 113 appropriately selects a normal appearance image 121a from the appearance image DB 121. The number of the selected appearance images 121a may be one or more. Then, the image generation unit 113 generates various additional images 1203 in the number corresponding to the statistical distribution 1202 by performing image processing or the like on the selected appearance images 121a.

[0122] At this time, the image generation unit 113 increases the number of additional images 1203 for distribution areas with larger deviations in the amount of deformation in the statistical distribution 1202. This can increase the variation of normal additional images 1203 in distribution areas where abnormalities are likely to occur. The amount of deformation of the additional images 1203 is determined based on the statistical distribution 1202 of the abnormal image set 1201.

[0123] Then, in step S34, the image generating unit 113 stores all the appearance images 121a and all the additional images 1203 included in the appearance image DB 121 in the expanded learning image DB 122. At this time, the image generating unit 113 also stores the attribute information of each of the appearance image 121a and the additional image 1203 in the expanded learning image DB 122.

[0124] Thus, as described above, changes in the normal extended learning image 122a in the region where abnormalities are likely to occur increase in the extended learning image DB 122. As a result, the learning unit 114 can accurately learn the recognition boundary for distinguishing abnormalities from normals, and can reduce the possibility of erroneous determination by the inspection unit 115.

[0125] <Third embodiment>

[0126] In this embodiment, an example in which an autoencoder is used to generate a learned model in step S23 will be described.

[0127] Fig.13 It is a schematic diagram showing an example of a method for generating a learned model in this embodiment.

[0128] In the present embodiment, first, the learning unit 114 acquires one or more normal extended learning images 122a from the extended learning image DB 122. The set of extended learning images 122a acquired in this way is hereinafter referred to as a learning image set 1302.

[0129] Next, the learning unit 114 inputs each extended learning image 122a of the learning image set 1302 as correct answer data to the autoencoder 1303. As a result, the autoencoder 1303 performs processing based on the internal parameters and outputs the reconstructed image 1304. The autoencoder 1303 is a model that performs learning in a manner that makes the input image and the reconstructed image 1304 the same image, so the internal parameters are updated in a manner that minimizes the error between the input image and the reconstructed image 1304. Then, the learning unit 114 stores the updated internal parameters in the learned parameter DB 123.

[0130] Then, the autoencoder 1303 uses the internal parameters stored in the learned parameter DB to output a reconstructed image 1304. The autoencoder 1303 that outputs the reconstructed image 1304 using the internal parameters stored in the learned parameter DB is a learned model in this embodiment.

[0131] The above steps complete the basic processing when using an autoencoder to generate a learned model.

[0132] Next, explain Figure 2 The method of checking in step S24.

[0133] Fig.14 14 is a schematic diagram showing an example of an inspection method in the present embodiment. First, the inspection unit 115 acquires an inspection image 1401 of a component captured by the imaging unit 150 .

[0134] Next, the inspection unit 115 reads the internal parameters from the learned parameter DB 123 into the autoencoder 1303, and then inputs the inspection image 1401 into the autoencoder 1303. As a result, the autoencoder 1303 outputs a reconstructed image 1304 based on the internal parameters.

[0135] like Fig.13As shown in FIG. 1 , the autoencoder 1303 learns the normal extended learning image 122a as the correct answer data, and thus outputs a normal reconstructed image 1304 without abnormality. Therefore, even if the inspection image 1401 contains foreign matter, the normal reconstructed image 1304 with the foreign matter removed is output. Therefore, if the inspection image 1401 contains foreign matter, the difference image 1305 obtained by taking the difference between the inspection image 1401 and the reconstructed image 1304 contains the foreign matter.

[0136] Then, the inspection unit 115 determines that the inspection target component is abnormal when the difference image 1305 includes foreign matter, and determines that the component is normal when the difference image 1305 does not include foreign matter.

[0137] The above steps complete the basic processing for inspecting components in this embodiment. Thus, by taking the difference between the reconstructed image 1304 output from the encoder 1303 and the inspection image 1401 , the inspection unit 115 can inspect whether a component has an abnormality.

[0138] <Hardware Structure>

[0139] Fig.15 It is a diagram showing an example of the hardware configuration of the appearance inspection apparatus 100 according to the first to third embodiments.

[0140] like Fig.15 As shown, the appearance inspection device 100 includes an imaging device 100a, a memory 100b, a processor 100c, a storage device 100d, a display device 100e, an input device 100f, and a reading device 100g. These devices are connected to each other via a bus 100i.

[0141] The camera device 100a is used to implement Figure 1 The imaging unit 150 is hardware. For example, the imaging device 100a is a camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor for capturing the appearance of a component.

[0142] The memory 100 b is hardware for temporarily storing data, such as DRAM (Dynamic Random Access Memory), and the program 124 is developed therein.

[0143] The processor 100c is a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit) that controls each part of the visual inspection device 100. The processor 100c cooperates with the memory 100b to execute the program 124 to realize Figure 1 The processing unit 110.

[0144] The storage device 100 d is a nonvolatile storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores the program 124 .

[0145] Alternatively, the program 124 may be recorded in advance on a computer-readable recording medium 100h, and the processor 100c may read the program 124 from the recording medium 100h.

[0146] The recording medium 100h includes, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), a USB (Universal Serial Bus) memory, and other physically removable recording media. A semiconductor memory such as a flash memory or a hard disk drive may also be used as the recording medium 100h.

[0147] The program 124 may be stored in a device connected to a public line, a network, a LAN (Local Area Network), etc. In this case, the processor 100 c only needs to read and execute the program 124 .

[0148] Figure 1 The storage unit 120 is implemented by the memory 100b and the storage device 100d.

[0149] The display device 100e is used to implement Figure 1 The input device 100f is used to implement the hardware such as a liquid crystal display or an organic EL display of the display unit 140. Figure 1 The input unit 130 may include hardware such as a keyboard or a mouse.

[0150] The reading device 100g is hardware such as a CD drive for reading data recorded in the recording medium 100h.

[0151] The effects described in this specification are merely illustrative and are not limiting, and other effects may also be present.

[0152] The present invention is not limited to the above-mentioned embodiments, and includes various modifications. Figure 1In the example of , the appearance inspection device 100 is provided with the camera unit 150, but the camera unit 150 may be provided outside the appearance inspection device 100. In this case, the camera unit 150 and the appearance inspection device 100 are connected by a network (not shown) such as a LAN or the Internet, and the appearance inspection device 100 stores the appearance image 121a photographed by the camera unit 150 in the appearance image DB 121. By adopting such a configuration, the appearance inspection device 100 can be used to realize a cloud server in which the learning unit 114 generates a learned model using the extended learning image 122a including the appearance image 121a as learning data and outputs the internal parameters of the learned model.

[0153] In addition, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not limited to having all the described constituent elements. In addition, a part of the composition of a certain embodiment can be replaced with the composition of another embodiment, and the composition of another embodiment can be added to the composition of a certain embodiment. In addition, with respect to a part of the composition of each embodiment, other compositions can be added, deleted, or replaced.

[0154] In addition, with respect to each of the above-mentioned structures, functions, processing units, processing means, etc., part or all of them can also be implemented by hardware by designing using, for example, integrated circuits. In addition, each of the above-mentioned structures, functions, etc. can also be implemented by software by interpreting and executing programs that implement their respective functions by a processor. Information such as programs, decision tables, files, etc. that implement each function can be placed in a storage device such as a memory, HDD, SSD, or a recording medium such as an IC (Integrated Circuit) card, SD (Secure Digital) card, DVD (Digital Versatile Disc). In addition, the control lines and information lines show the parts that are considered necessary for the description, and not all control lines and information lines may be shown on the product. It can also be considered that almost all structures are actually connected to each other.

[0155] Description of Reference Numerals

[0156] 100: appearance inspection device, 110: processing unit, 111: image acquisition unit, 112: statistical distribution generation unit, 113: image generation unit, 114: learning unit, 115: inspection unit, 120: storage unit, 121a: appearance image, 121b: attribute information, 122a: extended learning image, 122b: attribute information, 124: program, 130: input unit, 140: display unit, 150: camera unit, 401, 501, 601, 701, 801, 1201: image set, 402: reference value image, 403, 504 3, 602, 704, 803, 1202: statistical distribution, 404, 504, 604, 705, 804, 1203: additional image, 502: reference value image, 505: abnormal template, 702: brightness histogram, 703: reference histogram, 802: denoised image, 901: learning image set, 902: machine learning model, 903: estimated evaluation value, 1001: inspection image, 1302: learning image set, 1303: autoencoder, 1304: reconstructed image, 1305: difference image, 1401: inspection image.

Claims

1. A visual inspection device comprising a processor, wherein: The processor is configured as follows: acquiring a plurality of appearance images showing the appearance of the inspection object, generating a statistical distribution representing a deviation of features of each of the appearance images when the plurality of appearance images are taken as a population, generating an additional image capturing the appearance based on the deviation indicated by the statistical distribution, A learned model is generated by machine learning using learning data including the plurality of appearance images and the additional image.

2. The visual inspection device according to claim 1, wherein: The processor increases the number of the additional images for a distribution region having a larger deviation in the statistical distribution.

3. The visual inspection device according to claim 2, wherein: The feature is an amount of deformation of the inspection object captured in the appearance image.

4. The visual inspection device according to claim 2, wherein: The feature is a position where the inspection object is deformed as captured in the appearance image. The processor generates the additional image by superimposing a defect at the position of the appearance image.

5. The visual inspection device according to claim 1, wherein: The processor increases the number of the additional images for a distribution region in which the number of the external images is smaller in the statistical distribution.

6. The visual inspection device according to claim 5, wherein: The feature is any one of brightness, contrast, and noise intensity of the appearance image.

7. The visual inspection device according to claim 1, wherein: The processor also uses the learned model to check whether there is any abnormality in the inspection object captured in the inspection image.

8. The visual inspection device according to claim 7, wherein: The learned model is an autoencoder that has been learned using the learning data as correct answer data. The processor inputs the inspection image to the autoencoder, and checks whether the inspection object has an abnormality by determining whether a differential image obtained by taking the difference between a reconstructed image output from the autoencoder and the inspection image contains foreign matter.

9. The visual inspection device according to claim 1, wherein: The plurality of appearance images are images of normal appearance of the inspection object.

10. The visual inspection device according to claim 1, wherein: The plurality of appearance images are images of the appearance of the inspection object having an abnormality.

11. The visual inspection device according to claim 9 or 10, wherein: The processor processes an image showing a normal appearance of the inspection object to generate the additional image showing normality.

12. The visual inspection device according to claim 9, wherein: The processor processes an image showing an abnormal or normal appearance of the inspection object to generate the additional image showing the abnormality.

13. A method for generating an image, wherein a computer performs the following steps: A step of acquiring a plurality of appearance images showing the appearance of the inspection object; a step of generating a statistical distribution, wherein the statistical distribution represents a deviation of features of each of the appearance images when the plurality of appearance images are taken as a whole; A step of generating an additional image having the appearance based on the deviation indicated by the statistical distribution; and A step of generating a learned model by machine learning using learning data including the plurality of appearance images and the additional image.

Images