System, method and program for generating training image data for supervised machine learning

Abstract

To provide a system and a method for generating training image data for supervised machine learning, and a program that are able to easily generate training image data for performing defect classification using supervised machine learning.SOLUTION: A generation system 1 for generating training image data for supervised machine learning for training a defect classifier to be applied to visual inspection includes data generation means 8, the data generation means being configured to: arrange a defect model arbitrarily selected from a storage 7 to store defect models obtained by modeling shapes and optical characteristics of defects and an inspection target surface model arbitrarily selected from the storage 7 to store inspection target surface models obtained by modeling shapes and optical characteristics of inspection target surfaces in an arbitrary space in which ray tracing is performed by ray tracing simulation software in which a lighting optical system, an imaging optical system 60, and an imaging sensor 63 including a plurality of pixels are modeled; trace a plurality of light rays; calculate illuminance in each of the pixels based on intensity and the number of light rays entering each of the pixels; and generate, on the basis of the illuminance, pseudo captured image data as training image data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a training image data generation system, a generation method, and a program for supervised machine learning that generate training image data for training a defect classifier applied to classify the type of surface defect in the visual inspection of a workpiece. [Background technology]

[0002] Visual inspection devices that use a digital camera to capture the surface of a workpiece illuminated by a controlled lighting means and then process the captured image to detect surface defects are used in various industrial fields. In recent years, there has been a strong demand for classifying defect types from captured images for the purpose of analyzing the causes of defects and providing information for repair processes, and the development of defect type classification functions using supervised machine learning is underway.

[0003] Sufficient training data (labeled images) is collected for each defect type to be classified, and a supervised machine learning algorithm is trained to generate a classifier. In the field of image classification, a deep learning method called CNN (Convolutional Neural Network) is widely used due to its performance advantages.

[0004] To obtain the desired defect classification performance using supervised machine learning, sufficient training data is required, but this has the following challenges: 1. Humans with knowledge of defects need to assign correct labels to a large number of images, which requires a lot of effort, and different people may assign different labels to the same image. 2. It takes time to collect image data for defects that occur infrequently. 3. Images change depending on the characteristics of the test object (e.g., shape or color), so even if the defect is the same, images must be collected and trained for each test object characteristic. 4. Images change depending on shooting conditions such as lighting conditions and image resolution, so even for the same defect, images must be collected for each shooting condition and trained.

[0005] Patent Document 1 discloses an optical condition determination system having an objective of deriving optimal optical conditions to be applied to visual inspection, the system comprising: a simulation execution unit that generates a surface property image and a defect image by performing optical simulation under a plurality of optical conditions for a surface property model that models the surface property of an object to be inspected for appearance, and a defect model that models defects of the object, respectively; an image synthesis unit that generates a synthetic image by synthesizing the surface property image and the defect image generated by the optical simulation under the same optical conditions; an evaluation value calculation unit that calculates an evaluation value that indicates the ease of detection of the defect in the synthetic image; a correlation analysis unit that analyzes the correlation between the optical conditions corresponding to the synthetic image and the evaluation value; and an optimal condition search unit that searches for the optical conditions suitable for the visual inspection based on the results of the correlation analysis.

[0006] Patent Document 2 also discloses a method for determining the layout of a surface defect inspection device, in which the respective layout positions and optical axis directions of the illumination unit and imaging unit are set as first and second layout information, and an image obtained by illuminating the object to be inspected with the illumination unit according to the first layout information and imaging the inspection area with the imaging unit according to the second layout information is calculated based on each piece of information that quantifies the illumination unit, imaging unit, and appearance of the object to be inspected including defects, and defects are detected based on this image. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2021-43010 [Patent Document 2] International Publication No. WO2018 / 225406 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the technologies described in Patent Documents 1 and 2 are both aimed at deriving optimal optical conditions for visual inspection, and are not technologies for generating training image data for training a supervised machine learning algorithm. Therefore, the above-mentioned problem of acquiring training data could not be solved.

[0009] The present invention has been made in consideration of the above technical background, and aims to provide a training image data generation system, a generation method, and a program for supervised machine learning, which can easily generate appropriate training image data for defect classification using supervised machine learning. [Means for solving the problem]

[0010] The above object can be achieved by the following means: (1) A generation system for generating training image data for supervised machine learning to train a defect classifier applied to visual inspection, comprising: a data generating means for arranging a defect model, which models the shape and optical characteristics of a defect, and a test surface model, which models the shape and optical characteristics of a test surface, in an arbitrary space where light rays are traced by ray tracing simulation software, which models an illumination means and an imaging means, and for tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; Equipped with 、 The test surface is a painted surface having a clear layer formed on a base coat layer, and the optical characteristics of the test surface model include the scattering characteristics of the base coat layer and the reflection characteristics of the clear layer. A training image data generation system for supervised machine learning, comprising: (2) 2. The system for generating training image data for supervised machine learning described in claim 1, wherein the defect model is a paint defect in which a foreign object has been mixed into the clear layer. ( 3 a defect model storage unit for storing the defect model; a test surface model storage unit that stores the test surface model; The preceding paragraph 1 OR 2 A training image data generation system for supervised machine learning as described above. ( 4) A label representing a defect type is assigned to all defect models stored in the defect model storage unit, and the pseudo captured image data generated by the data generating means is assigned the same label as that assigned to the defect model used at the time of generation. 3 A training image data generation system for supervised machine learning as described above. ( 5 The data generating means generates the pseudo image data by arranging the defect model and the test surface model at a plurality of different positions on the simulation software. Or 2 A training image data generation system for supervised machine learning described herein. ( 6 The data generating means has a function of changing brightness and / or adding noise by performing image processing on the pseudo captured image data generated by the optical simulation. Or 2 A training image data generation system for supervised machine learning described herein. ( 7 ) The data generating means synthesizes the shapes of the defect model and the test surface model, and places them in an arbitrary space where rays are traced by ray tracing simulation software. Or 2 A training image data generation system for supervised machine learning described herein. ( 8 2.) A method for generating training image data for supervised machine learning to train a defect classifier applied to visual inspection, comprising: a data generation step of arranging a defect model in which the shape and optical characteristics of a defect are modeled, and a test surface model in which the shape and optical characteristics of a test surface are modeled, in an arbitrary space in which light rays are traced by ray tracing simulation software in which an illumination means and an imaging means are modeled, tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; Including fruit, The test surface is a painted surface having a clear layer formed on a base coat layer, and the optical characteristics of the test surface model include the scattering characteristics of the base coat layer and the reflection characteristics of the clear layer. A method for generating training image data for supervised machine learning. ( 9 ) 9. The method for generating training image data for supervised machine learning according to claim 8, wherein the defect model is a paint defect in which a foreign object has been mixed into the clear layer. ( 10 ) the defect model is stored in a defect model storage unit, A label representing a defect type is assigned to all defect models stored in the defect model storage unit, and the pseudo captured image data generated in the data generating step is assigned the same label as that assigned to the defect model used at the time of generation. 8 or 9 A method for generating training image data for supervised machine learning according to the present invention. ( 11 In the data generating step, the defect model and the test surface model are arranged at a plurality of different positions on the simulation software to generate the pseudo-captured image data. 8 or 9 A method for generating training image data for supervised machine learning according to the present invention. ( 12 In the data generating step, the pseudo captured image data generated by the optical simulation is subjected to image processing to change brightness and / or add noise. 8 or 9 A method for generating training image data for supervised machine learning according to the present invention. ( 13 In the data generation step, the shapes of the defect model and the test surface model are synthesized and placed in an arbitrary space where rays are traced by ray tracing simulation software. 8 or 9 A method for generating training image data for supervised machine learning according to the present invention. ( 14 2.) A program for causing a computer to execute a process of generating training image data for supervised machine learning for training a defect classifier applied to visual inspection, The program is a data generation step of arranging a defect model in which the shape and optical characteristics of a defect are modeled, and a test surface model in which the shape and optical characteristics of a test surface are modeled, in an arbitrary space in which light rays are traced by ray tracing simulation software in which an illumination means and an imaging means are modeled, tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; causing the computer to execute 、 The test surface is a painted surface having a clear layer formed on a base coat layer, and the optical characteristics of the test surface model include the scattering characteristics of the base coat layer and the reflection characteristics of the clear layer. program. ( 15 ) 15. The program according to claim 14, wherein the defect model is a paint defect caused by foreign matter being mixed into the clear layer. ( 16 ) all defect models are given a label representing the defect type, and the pseudo captured image data generated in the data generating step is given the same label as that given to the defect model used at the time of generation. 14 or 15 The program described in. ( 17 In the data generating step, the defect model and the test surface model are arranged at a plurality of different positions on the simulation software, and the computer is caused to execute a process of generating the pseudo-captured image data. 14 or 15 The program described in. ( 18 In the data generating step, the computer is caused to execute a process of changing brightness and / or adding noise by performing image processing on the pseudo captured image data generated by the optical simulation. 14 or 15 The program described in. ( 19 In the data generation step, the computer is further caused to execute a process of synthesizing the shapes of the defect model and the test surface model and arranging them in an arbitrary space where light rays are traced by a ray tracing simulation software. 14 or 15 The program described in. Effect of the Invention

[0011] According to the system and method for generating training image data for machine learning of the present invention, a defect model that models the shape and optical characteristics of a defect is generated. and A test surface model that models the shape and optical characteristics of the test surface Ruto ,illumination means , Imaging The means A model is placed in an arbitrary space where rays are traced by a ray tracing simulation software, and multiple rays are traced to A plurality of modeled Pixels each Intensity and number of incident light rays Based on Then, pseudo captured image data, which is the training image data, is generated. In other words, since the training image data is generated by modeling the defects and tracing the rays, all the generated training image data is correct image data corresponding to the defect model, and there is no need to assign a correct label. Therefore, correct training image data can be easily generated, and the cost of generating training data can be reduced.

[0012] In addition, since it is possible to freely model combinations of defect shapes and other characteristics (e.g., test surface shape, test surface optical characteristics, lighting conditions, shooting conditions, etc.), it is easy to obtain a sufficient amount of training data even for defects with a low occurrence rate.

[0013] According to the program of the present invention, a defect model in which the shape and optical characteristics of a defect are modeled is generated. And, Test surface model that models the shape and optical characteristics of the test surface And ,illumination means , Imaging hand Steps A model is placed in an arbitrary space where rays are traced using ray tracing simulation software, and multiple rays are traced to obtain the above. A plurality of modeled Pixels each The intensity of the light incident on the Number The computer can be caused to execute a process of generating pseudo captured image data, which is training image data, based on the acquired pseudo captured image data. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of a training image data generation system according to an embodiment of the present invention. [Diagram 2] 13A and 13B are diagrams for explaining an example of optical characteristics in a defect model. [Diagram 3] 1A to 1C are diagrams showing an example of the shape of a defect in a defect model. [Figure 4] 1A to 1C are diagrams showing an example of the shape of a test surface in a test surface model. [Diagram 5] FIG. 1 is an explanatory diagram of modeling of an illumination optical system. [Figure 6] FIG. 2 is an explanatory diagram of modeling of an imaging optical system. [Figure 7] 1 is a diagram showing an example of the arrangement of each element of a defect model, a test surface model, an illumination optical system including illumination, an imaging optical system, and an imaging sensor in a ray tracing simulation; [Figure 8] (A) shows pseudo-image data of a defect model of a surface defect caused by dust near the door of a vehicle body and a test surface model, (B) shows a defect model of a surface defect caused by a pinhole near the wheelhouse of a vehicle body and a test surface model, and (C) shows a defect model of a surface defect caused by paint dripping near the accent line of a vehicle body and a test surface model, each of which is moved a small amount. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] 1 is a configuration diagram of a training image data generation system 1 according to an embodiment of the present invention. In this embodiment, a case will be described in which training image data is generated for training a defect classifier that is applied to classify types of surface defects in the appearance inspection of a vehicle body.

[0017] This training image data generation system 1 includes a personal computer (hereinafter also referred to as a PC), and in terms of hardware, the PC includes a CPU (Central Processing Unit) 2, a RAM (Random Access Memory) 3, an input unit 4 such as a mouse or keyboard, a display unit 5 such as a liquid crystal display, a network interface (network I / F) 6 for communicating with external devices via a network, and a storage unit 7 including a hard disk or SSD (Solid State Drive). A defect model, a test surface model, ray tracing simulation software, other programs, and the like are stored in the storage unit 7. These will be described later.

[0018] Meanwhile, the PC functionally comprises a data generation unit 8. The data generation unit 8 realizes its functions by the CPU 2 operating in accordance with a program stored in the storage unit 7 and loaded into the RAM 3.

[0019] The defect models stored in the storage unit 7 are models of the shape and optical characteristics of the defects. It is preferable that each defect model stored in the storage unit 7 is assigned a label representing the defect type.

[0020] An example of the optical characteristics in the defect model is shown in Figure 2. The characteristics shown in Figure 2 are an example of the optical characteristics for a paint defect that occurs when a primer layer 13, a base coat layer 14, and a clear layer 15 are painted in this order on the surface of a steel plate (painted surface) 11 in a vehicle body 10 via an electrodeposition layer 12, and a foreign matter 20 is mixed into the clear layer 15, as shown in Figure 2(A), and the light absorption rate of the foreign matter 20 is set to 100%, the reflectance of the surface of the clear layer 15 is set to 4%, and the scattering characteristics of the base coat layer 14 are set to the characteristics shown in Figure 2(B). However, it is desirable to impart the reflectance, absorbance, and scattering characteristics with characteristics that match the surface to be actually inspected.

[0021] Examples of defect shapes in the defect model are shown in Figure 3 (A) to (C). Differences in defect types are mainly due to differences in defect shapes.

[0022] The model in Figure 3(A) shows dust getting mixed into the paint, resulting in a cone-shaped surface convex defect 31. The model in Figure 3(B) shows a pinhole appearing on the paint surface, resulting in a cone-shaped surface concave defect 32. The model in Figure 3(C) shows a paint droplet causing a surface convex defect 33 with a semi-elliptical cross section. The shape of each defect is stored as a 3D coordinate array.

[0023] The test surface model stored in the storage unit 7 is a model of the shape and optical characteristics of the test surface, which is the surface to be inspected.

[0024] The shape of the test surface in the test surface model is, for example, a model as shown in Figures 4(A) to 4(C). The model in Figure 4(A) shows the shape of test surface 41 of the door part of vehicle body 10 shown in Figure 4(D), the model in Figure 4(B) shows the shape of test surface 42 of the accent line of the door of vehicle body 10 shown in Figure 4(D), and the model in Figure 4(C) shows the shape of test surface 43 of the wheel house which is the tire housing part of vehicle body 10 shown in Figure 4(D). The shapes of each of test surfaces 41 to 43 are held as an array of three-dimensional coordinates.

[0025] 2A, the optical characteristics of the test surfaces 41 to 43 in the test surface model are the reflection characteristics of the clear layer 15 and the scattering characteristics of the base coat layer 14 in the test surfaces 41 to 43. It is desirable to impart the reflectance and scattering characteristics with characteristics that match the test surfaces that are actually inspected.

[0026] In this embodiment, the defect model and the test surface model are stored in one storage unit 7 built into the PC constituting the training image data generation system 1, but they may be stored in different storage units. At least one of the defect model and the test surface model may be stored in a storage unit located outside the PC. When stored in a storage unit located outside the PC, the PC acquires any defect model or test surface model from the external storage unit.

[0027] The data generation unit 8 generates training image data, and includes a shape synthesis unit 81 and an image processing unit 82. The shape synthesis unit 81 synthesizes the shape of the defect model and the shape of the test surface model. The image processing unit 82 performs image processing on the generated training image data.

[0028] A specific method for generating the training image data by the data generating unit 8 is as follows.

[0029] That is, a defect model stored in the storage unit 7 is arbitrarily selected. Similarly, a test surface model stored in the storage unit 7 is arbitrarily selected. Then, the selected defect model and test surface model are placed in an arbitrary space where rays are traced by the ray tracing simulation software. In the ray tracing simulation software, an illumination optical system 50, an imaging optical system 60, and an imaging sensor 63 composed of a plurality of pixels are modeled.

[0030] Modeling of the illumination optical system 50 is realized, for example, by a model as shown in Fig. 5. In the example of Fig. 5, an illumination optical system 50 consisting of a strip-shaped illumination is arranged surrounding the vehicle body 10 in the width direction, and the angular distribution of the illumination optical system 50 is set to Lambert, and the spatial distribution is set to that shown in Fig. 5(B). The horizontal axis in Fig. 5(B) is the width of the illumination optical system 50. However, it is desirable to generate a model that matches the spatial luminance distribution and orientation distribution of the light source mounted on the inspection device in which the classifier is to be installed.

[0031] Modeling of the imaging optical system 60 is realized, for example, by a model as shown in Fig. 6. In the example of Fig. 6, reflected light 70 from test surfaces 41 to 43 is focused on an imaging sensor 63 via a lens 61 and an aperture 62. In order to reproduce the optical aberration and peripheral light falloff that actually occur on an image, it is desirable to model it according to the lenses and the like mounted on the inspection device in which the classifier is to be installed.

[0032] When the defect model and the test surface model are arranged in an arbitrary space, the shape of the defect and the shape of the test surface are synthesized by a shape synthesis unit 81. The shape synthesis may be performed by, for example, any one of the following methods (1) to (3). (1) The shapes of the defect model and the test surface model are stored as a 3D coordinate array, and the shapes are synthesized by adding up the z values ​​of points with the same (x, y) coordinates. (2) The shapes of the defect model and the test surface model are stored in a two-dimensional matrix whose value is height, and the shapes are synthesized by adding together the same matrix elements. (3) The shapes of the defect model and the test surface model are held in an arbitrary form (for example, an array of the above three-dimensional coordinates, the above two-dimensional matrix, or a polygon), and the shapes are synthesized by Boolean operations.

[0033] Ray tracing simulation software is software that can geometrically simulate how light rays transmit, refract, reflect, and propagate. For example, well-known software such as "LightTools", a product name of Synopsys, Inc., can be used.

[0034] FIG. 7 shows an example of the arrangement of the composite 90 of the defect model and the test surface model, the illumination optical system 50, the imaging optical system 60, and the imaging sensor 63 in the ray tracing simulation.

[0035] With each element positioned in this manner, the illuminance at each pixel is calculated based on the intensity and number of light rays reflected from the test surface and incident on the pixel of the image sensor 63, and pseudo-image data is generated based on the calculated illuminance, and this image data is used as the training image data.

[0036] The generated pseudo image data is outputted and stored in a storage unit 7 in the training image data generation system or in a storage unit provided outside the system in a general image format (e.g., bitmap, Tiff, etc.). When a label indicating a defect type is added to the generated pseudo image data, it is preferable to add the same label as the defect type of the defect model used and store the data in order to make the generated pseudo image data easier to manage.

[0037] An example of the generated pseudo image data is shown in Fig. 8. In the appearance inspection of the vehicle body, images are continuously captured by the imaging device while at least one of the vehicle body and the lighting device is moved. Therefore, the pseudo image data shown in Fig. 8 is also generated by moving the defect model and the test surface model 90 by a small amount and arranging them in multiple different positions.

[0038] Specifically, Fig. 8(A) shows pseudo-image data obtained when a composite 90 of a defect model representing a surface defect caused by dust near the door of the vehicle body 10 and a test surface model is moved a very small amount. Fig. 8(B) shows pseudo-image data obtained when a composite 90 of a defect model representing a surface defect caused by a pinhole near the wheel house of the vehicle body 10 and a test surface model is moved a very small amount. Fig. 8(C) shows pseudo-image data obtained when a composite 90 of a defect model representing a surface defect caused by paint dripping near the accent line of the vehicle body 10 and a test surface model is moved a very small amount. In all cases, black dots in the light bands indicate defects.

[0039] The brightness of the actual captured image data of the surface defect may vary slightly or noise may occur in the image depending on the type of the surface defect, the imaging conditions, etc. Therefore, in order to match the generated pseudo captured image data with the actual captured image data, the image processing unit 82 may perform image processing to change the brightness and / or add noise.

[0040] An example of a workflow for setting a classifier using the training image data generation system 1 according to this embodiment will be described below. [1] Information on defects that occur at the mass production site is obtained in advance using a three-dimensional shape measuring device or the like, and multiple defect models are prepared. An example of a three-dimensional shape measuring device that can be used at the site is GelSight Mobile from GelSight, Inc. In addition, the distribution of variation in shape and characteristics may be predicted from multiple defect models, and the number of defect models may be increased using random numbers. [2] The inspection device in which the classifier is installed acquires information about the workpiece to be inspected and prepares multiple models of the test surface. For example, in the case of a paint defect inspection device for car bodies, the user of the inspection device can provide CAD (computer-aided design) data in advance, and the surface of the CAD data can be cut out for each measurement range to generate a distribution of the test surface shape. [3] The illumination optical system 50, the imaging optical system 60, and the imaging sensor 63 of the inspection device are placed on the ray tracing simulation software. [4] The defect model and the test surface model are arbitrarily combined, and the composite 90 is placed on the ray tracing simulation software. The defect model and the test surface model may be combined according to a predetermined probability distribution based on the actual occurrence frequency. For example, in the case of a car body paint defect, it is possible to control the probability of combination with the test surface model for a defect type that is likely to occur on the side of the car body and a defect type that is likely to occur on the top surface of the car body. [5] Ray tracing is performed using ray tracing simulation software, and the brightness value of each pixel is calculated using the intensity and number of rays that reach each pixel of the image sensor 93. The brightness value is then converted into an image format and output to and stored in the PC's memory unit 7 or an external memory unit. [6] The composite 90 of the defect model and the test surface shape model is moved by a small amount, ray tracing is performed again, and an image file is output. The small amount is determined to match the shooting frame rate of the actual inspection device and the moving speed of the test object and the imaging means. By repeating the ray tracing of this movement a predetermined number of times, an image set consisting of a set of multiple images is generated. [7] Through the above process, multiple image sets (usually several hundred or more) are generated for each defect type to be classified, and a learning model for supervised machine learning is trained. At this time, it is advisable to separate the image sets into training and evaluation sets, and check the classification performance using the evaluation image set before installing the learning model in the inspection system.

[0041] As described above, according to the training image data generation system 1 of this embodiment, a defect model arbitrarily selected from the storage unit 7 that holds a defect model in which the shape and optical characteristics of a defect are modeled, and a test surface model arbitrarily selected from the storage unit 7 that holds a test surface model in which the shape and optical characteristics of a test surface are modeled are placed in an arbitrary space where light rays are traced by a ray tracing simulation software in which an illumination optical system, an imaging optical system 60, and an imaging sensor 63 composed of a plurality of pixels are modeled, and a plurality of light rays are traced to calculate the illuminance at each pixel based on the intensity and number of light rays incident on the pixels of the imaging sensor 63, and pseudo captured image data, which is training image data, is generated based on the calculated illuminance. In other words, since the defect is modeled and the training image data is generated by ray tracing, all the generated training image data becomes appropriate image data corresponding to the defect model, and there is no need to assign a correct answer label. Therefore, appropriate training image data can be easily generated, and the cost of generating training data can be reduced.

[0042] In addition, since it is possible to freely model combinations of defect shapes and other characteristics (e.g., test surface shape, test surface optical characteristics, lighting conditions, shooting conditions, etc.), it is easy to obtain a sufficient amount of training data even for defects with a low occurrence rate. [Explanation of symbols]

[0043] 1. Training image data generation system 2 CPU 3. RAM 4 Input section 5 Display section 6 Network Interface 7 Memory section 8 Data Generation Section 10. Body 13 Primer layer 14 Basecoat layer 15 Clear layer 20 Foreign object 31 Convex surface defects caused by dust 32 Surface concave defects caused by pinholes 33 Convex surface defects caused by paint dripping 41~43 Test surface 50 Illumination optical system 60 Imaging Optical System 61 Lens 62 Aperture 63 Image Sensor 70 Reflected light 81 Shape synthesis section 82 Image Processing Section 90 Defect model and test surface model

Claims

1. A generation system for generating training image data for supervised machine learning to train a defect classifier for application to visual inspection, comprising: a data generating means for arranging a defect model, which models the shape and optical characteristics of a defect, and a test surface model, which models the shape and optical characteristics of a test surface, in an arbitrary space where light rays are traced by ray tracing simulation software, which models an illumination means and an imaging means, and for tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; Equipped with A training image data generation system for supervised machine learning, characterized in that the test surface is a painted surface having a clear layer formed on a base coat layer, and the optical properties of the test surface model include the scattering properties of the base coat layer and the reflection properties of the clear layer.

2. A training image data generation system for supervised machine learning as described in Claim 1, wherein the defect model is a paint defect in which foreign matter has been mixed into the clear layer.

3. a defect model storage unit that stores the defect model; a test surface model storage unit that stores the test surface model; The system for generating training image data for supervised machine learning according to claim 1 or 2, comprising:

4. 4. The training image data generation system for supervised machine learning according to claim 3, wherein all defect models stored in the defect model storage unit are assigned a label representing the defect type, and the pseudo-image data generated by the data generation means is assigned the same label as that assigned to the defect model used at the time of generation.

5. The training image data generation system for supervised machine learning according to claim 1 or 2, wherein the data generation means generates the pseudo-captured image data by placing the defect model and the test surface model at a plurality of different positions on the simulation software.

6. The training image data generation system for supervised machine learning as described in claim 1 or 2, wherein the data generation means has a function of changing brightness and / or adding noise by performing image processing on the generated pseudo captured image data.

7. The training image data generation system for supervised machine learning according to claim 1 or 2, wherein the data generation means synthesizes the shapes of the defect model and the test surface model and places them in an arbitrary space where ray tracing is performed by ray tracing simulation software.

8. 1. A method for generating training image data for supervised machine learning to train a defect classifier for application to visual inspection, comprising: a data generation step of arranging a defect model in which the shape and optical characteristics of a defect are modeled, and a test surface model in which the shape and optical characteristics of a test surface are modeled, in an arbitrary space in which light rays are traced by ray tracing simulation software in which an illumination means and an imaging means are modeled, tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; Including, A method for generating training image data for supervised machine learning, wherein the test surface is a painted surface having a clear layer formed on a base coat layer, and the optical properties of the test surface model include the scattering properties of the base coat layer and the reflection properties of the clear layer.

9. A method for generating training image data for supervised machine learning as described in claim 8, wherein the defect model is a paint defect in which foreign matter has been mixed into the clear layer.

10. The defect model is stored in a defect model storage unit, 10. The method for generating training image data for supervised machine learning described in claim 8 or 9, wherein all defect models stored in the defect model storage unit are assigned a label representing the defect type, and the pseudo-image data generated in the data generation step is assigned the same label as that assigned to the defect model used at the time of generation.

11. The training image data generating method for supervised machine learning according to claim 8 or 9, wherein the data generating step generates the pseudo-image data by placing the defect model and the test surface model at a plurality of different positions on the simulation software.

12. The method for generating training image data for supervised machine learning according to claim 8 or 9, wherein in the data generation step, image processing is performed on the generated pseudo captured image data to change brightness and / or add noise.

13. The method for generating training image data for supervised machine learning according to claim 8 or 9, wherein in the data generation step, the shapes of the defect model and the test surface model are synthesized and placed in an arbitrary space where ray tracing is performed by ray tracing simulation software.

14. A program for causing a computer to execute a process of generating training image data for supervised machine learning for training a defect classifier applied to visual inspection, comprising: The program is a data generation step of arranging a defect model in which the shape and optical characteristics of a defect are modeled, and a test surface model in which the shape and optical characteristics of a test surface are modeled, in an arbitrary space in which light rays are traced by ray tracing simulation software in which an illumination means and an imaging means are modeled, tracing a plurality of light rays, and generating pseudo-captured image data, which is the training image data, based on the intensities and numbers of light rays incident on each of a plurality of pixels modeled in the imaging means; causing the computer to execute The test surface is a painted surface having a clear layer formed on a base coat layer, and the optical characteristics of the test surface model include the scattering characteristics of the base coat layer and the reflection characteristics of the clear layer.

15. The program described in claim 14, wherein the defect model is a paint defect in which foreign matter has been mixed into the clear layer.

16. The program according to claim 14 or 15, wherein all defect models are assigned a label representing the defect type, and the program causes the computer to execute a process of assigning to the pseudo-image data generated in the data generation step the same label as that assigned to the defect model used at the time of generation.

17. 16. The program according to claim 14, wherein the data generating step causes the computer to execute a process of generating the pseudo image data by arranging the defect model and the test surface model at a plurality of different positions on the simulation software.

18. 16. The program according to claim 14, wherein in the data generating step, the computer is made to execute a process of changing brightness and / or adding noise by performing image processing on the generated pseudo captured image data.

19. The program according to claim 14 or 15, wherein the data generation step further causes the computer to execute a process of synthesizing the shapes of the defect model and the test surface model and arranging them in an arbitrary space where rays are traced by ray tracing simulation software.

Images