Inspection device and inspection method

Abstract

To provide an inspection device and inspection method that can perform inspections easily and with high accuracy. [Solution] The inspection device 1 includes an imaging means 10 for photographing the object to be inspected M, an inspection area cropping means 20 for cropping an area including the inspection area from the image obtained by the imaging means 10 using fixed coordinates set in advance as an inspection area image, a reference area detection means 30 for detecting a reference area that will serve as a reference position for detecting the inspection area from the inspection area image, a discrimination image cropping means 40 for detecting the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area and cropping it as a discrimination image, and a discrimination means 50 for inputting the discrimination image into a learning model 51 that has been trained by machine learning to identify good products of the inspection area, and for determining whether the inspection area is good or bad based on the obtained discrimination value.

Description

Technical Field

[0001] The present invention relates to an inspection apparatus and an inspection method for inspecting the quality (good or bad) of an inspection target.

Background Art

[0002] In the assembly of a vehicle body or the like, for example, before assembly, it is inspected whether the press-worked product has cracks. Conventionally, for example, an inspector visually inspects for cracks. However, since the size of the crack is very small, only a few millimeters, there is a problem in the visual inspection by the inspector that there are variations in sensitivity among inspectors and there is concern about the reliability of inspection accuracy.

[0003] In Patent Document 1, a learning model that has learned only good products is prepared, one-dimensional vector data obtained by extracting an inspection location from a photographed image is input into the learning model, and the likelihood of defectiveness output as a result is determined based on a set threshold value to determine the presence or absence of a defect. However, in the method of Patent Document 1, since pattern matching is used in the extraction of the inspection location, stable detection may be difficult when the situation of the inspection surface is different, and there is also a problem that the matching process takes time.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present invention has been made based on such problems, and an object thereof is to provide an inspection apparatus and an inspection method that can be easily inspected with high accuracy.

Means for Solving the Problems

[0006] The inspection apparatus of the present invention is for inspecting whether an object to be inspected is good or bad, and comprises: an imaging means for photographing the object to be inspected; an inspection area cropping means for cropping an area including the inspection area from the image obtained by the imaging means using fixed coordinates set in advance as an inspection area image; a reference area detection means for detecting a reference area that serves as a reference position for detecting the inspection area from the inspection area image; a discrimination image cropping means for detecting the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area and cropping it as a discrimination image; and a discrimination means for inputting the discrimination image into a learning model that has been trained by machine learning to identify good products of the inspection area, and for determining whether the inspection area is good or bad based on the obtained discrimination value.

[0007] The present invention provides an inspection method for inspecting whether an object to be inspected is good or bad, and includes: an imaging procedure for photographing the object to be inspected; an inspection area cropping procedure for cropping an area containing the inspection area from the image obtained by the imaging procedure using fixed coordinates set in advance as an inspection area image; a reference area detection procedure for detecting a reference area from the inspection area image that will serve as a reference position for detecting the inspection area; a discrimination image cropping procedure for detecting the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area and cropping it as a discrimination image; and a discrimination procedure for inputting the discrimination image into a learning model that has been trained by machine learning to identify good products of the inspection area, and determining whether the inspection area is good or bad based on the obtained discrimination value. [Effects of the Invention]

[0008] According to the present invention, a reference area is detected from the inspection area image, which serves as a reference position for detecting the inspection area. The inspection area is then detected using the relative coordinates of the inspection area with respect to the reference area. By using a reference area that can be detected with high accuracy, the inspection area can be detected with high accuracy, stably, easily, and quickly. Therefore, the object to be inspected can be easily and accurately inspected.

[0009] Furthermore, if the determination of whether an item is good or bad is based on a threshold set between the average value + 4σ of each good item discrimination value obtained by inputting multiple good item discrimination images into the learning model, and the lower limit of each defective item discrimination value obtained by inputting multiple defective item discrimination images into the learning model, then since the threshold is set using both good item discrimination values ​​and defective item discrimination values, the threshold accuracy can be increased, and the judgment accuracy can be improved. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows the configuration of an inspection device according to one embodiment of the present invention. [Figure 2] This figure shows an example of an image captured using a photographic device. [Figure 3] This figure shows an example of an inspection area image cropped from a captured image. [Figure 4] This is a conceptual diagram showing the location of the basic and examination areas in the examination area image. [Figure 5] This is an example of a histogram showing the frequency of the good product discrimination value. [Figure 6] This is another example of a histogram showing the good product discrimination value and frequency. [Figure 7] This figure shows an example of the hardware configuration of the inspection area cropping means, reference area detection means, discrimination image cropping means, and discrimination means shown in Figure 1. [Figure 8] This diagram shows the flow of an inspection method according to one embodiment of the present invention. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] Figure 1 shows the configuration of an inspection device 1 according to one embodiment of the present invention. This inspection device 1 inspects whether an object to be inspected M is good or bad. Specifically, it can be used to inspect, for example, cracks in press-formed products, or sink marks and scratches on injection-molded products. In this embodiment, the case of inspecting whether a press-formed product such as a car body has cracks or other defects will be explained as an example.

[0013] The inspection device 1 includes, for example, an imaging means 10 for photographing the object to be inspected M, an inspection area cropping means 20 for cropping an area containing the inspection area from the image obtained by the imaging means 10 using pre-set fixed coordinates as an inspection area image, a reference area detection means 30 for detecting a reference area that serves as a reference position for detecting the inspection area from the inspection area image, a discrimination image cropping means 40 for detecting the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area and cropping it as a discrimination image, and a discrimination means 50 for inputting the discrimination image to a learning model 51 that has been trained by machine learning to identify good products of the inspection area, and for determining whether the inspection area is good or bad based on the obtained discrimination value. Furthermore, the inspection device 1 preferably includes, for example, a display means 60 such as a display for displaying the discrimination result of the discrimination means 50, and more preferably an alarm means 70 for issuing an alarm if the discrimination result of the discrimination means 50 is bad.

[0014] The imaging means 10 is composed of a camera, such as a CCD camera. The imaging means 10 is arranged to photograph, for example, an area of ​​the inspection target M that includes the inspection area. Figure 2 shows an example of an image taken of the inspection target M by the imaging means 10. In Figure 2, the area enclosed by the white dashed line is the area that includes the inspection area. Examples of inspection areas include areas where defects such as cracks have occurred in the past in press-formed products of the same type, or areas where defects such as cracks have occurred in press-formed products of other vehicle types. The imaging means 10 may, for example, photograph the entire inspection target M, or it may photograph a part of the inspection target M that includes the inspection area. However, it is preferable to photograph a relatively wide area so that the inspection area is included in the shooting range even if the position of the inspection target M varies.

[0015] The inspection area cutting means 20, the reference part detection means 30, the discrimination image cutting means 40, and the discrimination means 50 can be configured by, for example, a computer, and are configured to function as the inspection area cutting means 20, the reference part detection means 30, the discrimination image cutting means 40, or the discrimination means 50 by executing a program.

[0016] The inspection area cutting means 20 is connected to, for example, the imaging means 10, and is configured to cut out an area including the inspection part from the captured image input from the imaging means 10 according to preset fixed coordinates. For example, it is configured to cut out with the x - coordinates and y - coordinates of four fixed points so as to include a range including the inspection part and a reference part to be described later. The range for cutting out the area including the inspection part from the captured image is preferably set to a relatively wide range so that the inspection part is included in the cutting range even if the position of the inspection part in the captured image varies. FIG. 3 shows an example of an inspection area image cut out from the captured image.

[0017] The reference part detection means 30 is preferably connected to, for example, the inspection area cutting means 20, and is configured to detect the reference part from the inspection area image cut by the inspection area cutting means 20 using an object detection method. The reference part is a reference position for detecting the inspection part and is a part that can be detected with higher accuracy than the inspection part. As the reference part, for example, it is preferable to select a part that can be detected with higher accuracy in the inspection area image. Specifically, for example, uneven shape parts such as character lines and design parts with simple shapes such as round shapes and clear light and dark are preferable.

[0018] As an object detection method for detecting a reference part, for example, there is a method based on bounding box object detection using a DNN (Deep Neural Network) such as YOLO (You Only Look Once). The reference part detection means 30 is configured to, for example, as shown in FIG. 4, surround the detected reference part with a rectangle and detect the coordinates of the center position thereof. In FIG. 4, the reference part is shown surrounded by a white dotted line.

[0019] The discrimination image cutting means 40 is connected to, for example, the inspection area cutting means 20 and the reference part detection means 30, and based on the position coordinates of the reference part detected by the reference part detection means 30 from the inspection area image cut by the inspection area cutting means 20, it is configured to detect the inspection part based on the positional relationship between the registered reference part and the inspection part. Specifically, for example, it is preferable to register the relative coordinates of the center coordinates of the inspection part with respect to the center coordinates of the reference part and the size of the inspection part, and configure it to detect the center coordinates and size of the inspection part from the center coordinates of the reference part. In FIG. 4, the detected inspection part is shown surrounded by a white broken line. Note that FIG. 4 shows the case where two detection parts are detected. Also, the discrimination image cutting means 40 is configured to, for example, individually cut out the part of the detected inspection part as a discrimination image.

[0020] The discrimination means 50 is connected to, for example, the discrimination image cutting means 40 and is configured to receive the discrimination image from the discrimination image cutting means 40. The discrimination means 50 preferably has, for example, a learning model 51 that learns the good products of the inspection part by machine learning, a good / bad determination unit 52 that determines the good / bad of the inspection part based on the discrimination value (score value) obtained by inputting the discrimination image into the learning model 51, and a learning model generation means 53 that generates the learning model 51.

[0021] The learning model 51 is preferably trained only on good products of the inspection area. This is because there are countless shapes of defects, making it difficult to model them. The learning model 51 is configured to use, for example, an image classification discriminator (e.g., Efficient-Net) to extract intermediate feature values ​​and to obtain a discriminant value using the Mahalanobis general distance.

[0022] The good / bad judgment unit 52 is preferably configured to determine a product as good if the discriminant value obtained by the learning model 51 is below a preset threshold, and as bad if it is greater than the threshold. The threshold is preferably set between the average value of each good product discriminant value obtained by inputting multiple good product discriminant images into the learning model 51 + 4σ (i.e., mean + 4 × standard deviation) and the lower limit of each defective product discriminant value obtained by inputting multiple defective product discriminant images into the learning model 51.

[0023] Defects such as cracks in pressed products are very small in size, and the difference between good and defective products is minute. Therefore, the values ​​for good product discrimination and defective product discrimination are close together, and if a threshold is set based only on the good product discrimination value, the broad base of the good product discrimination value may lead to a threshold that incorrectly classifies good products as defective. For this reason, the accuracy of the judgment can be improved by setting the threshold using the defective product discrimination value as a reference. It is preferable to set the threshold by, for example, displaying each good product discrimination value and each defective product discrimination value in a histogram and setting it between the mean value of each good product discrimination value + 4σ and the lower limit of each defective product discrimination value.

[0024] Figures 5 and 6 show examples of histograms representing good and bad product discrimination values. In Figures 5 and 6, the frequency of each good product discrimination value is shown as a bar, and the positions of each bad product discrimination value, the mean of each good / bad product discrimination value, the mean of each good product discrimination value + 4σ, the mean of each good product discrimination value + 4σ + 20, and the set threshold are shown as lines. Note that the mean of each good product discrimination value + 4σ + 20 is a value commonly used as a threshold. In Figures 5 and 6, each bad product discrimination value is shown as a thin solid line, the mean of each good product discrimination value is shown as a dashed line, the mean of each good product discrimination value + 4σ is shown as a dashed line, the mean of each good product discrimination value + 4σ + 20 is shown as a dotted line, and the set threshold is shown as a thick solid line. Furthermore, each bad product discrimination value was obtained by creating multiple defective products by attaching tape simulating cracks to good products and inputting the discrimination images of each defective product into the learning model 51. The tape is available in three lengths: 2mm, 5mm, and 10mm.

[0025] As shown in Figures 5 and 6, some good product discrimination values ​​are greater than the average value of each good product discrimination value + 4σ + 20. Therefore, if the threshold is set to the average value of each good product discrimination value + 4σ + 20, good products may be mistakenly identified as defective. Thus, by displaying each good product discrimination value and each defective product discrimination value in a histogram and setting a threshold between the average value of each good product discrimination value + 4σ and the lower limit of each defective product discrimination value, high-accuracy judgment can be achieved.

[0026] Preferably, the learning model generation means 53 is configured to prepare multiple learning images of good products in the inspection area as positive examples of discriminant images, extract intermediate features using a trained image classification network (e.g., Efficient-Net), and determine the discriminant value using the Mahalanobis general distance. Since only execution is performed and no deep learning is carried out, the reference threshold can be determined in a short time. The learning images can be obtained, for example, by photographing a good product to be inspected M with the photography means 10, cutting out the inspection area image from the obtained photographed image with the inspection area cutting means 20, detecting the reference area with the reference area detection means 30, and cutting out the inspection area with the discriminant image cutting means 40.

[0027] Figure 7 shows an example of the hardware configuration of the inspection area cropping means 20, the reference area detection means 30, the discrimination image cropping means 40, and the discrimination means 50. The inspection area cropping means 20, the reference area detection means 30, the discrimination image cropping means 40, and the discrimination means 50 each include, for example, a CPU (Center Processing Unit) 81, a ROM (Read Only Memory) 82, a RAM (Random Access Memory) 83, an HDD (Hard Disk Drive) 84, and an operation interface (operation I / F) 85. The CPU 81 executes various processes according to various programs recorded in the ROM 82 or various programs loaded from the HDD 84 into the RAM 83. The RAM 83 also appropriately stores data necessary for the CPU 81 to execute various processes. The HDD 84 stores various data.

[0028] The inspection device 1 is used, for example, as follows. Figure 8 shows the flow of the inspection method using the inspection device 1. First, as a preparation step, a learning model 51 is generated by the learning model generation means 53 (learning model generation procedure; step S110). In the learning model generation procedure (step S110), first, for example, multiple good product inspection targets M are photographed by the photography means 10, an inspection area image is cut out from the obtained photographed image, a reference part is detected from the inspection area image, and the inspection part is cut out using the relative coordinates with respect to the reference part to prepare a learning image. Next, for example, the features of the inspection part are learned by deep learning using the learning image to generate the learning model 51.

[0029] Furthermore, as a preparation step, a threshold is set to determine whether the discriminant value obtained by inputting the discriminant image into the learning model 51 is good or bad (threshold setting procedure; step S120). In the threshold setting procedure (step S120), first, for example, multiple discriminant images of good products are input into the learning model 51 to obtain good product discriminant values ​​for each, and multiple discriminant images of defective products are input into the learning model 51 to obtain defective product discriminant values ​​for each.

[0030] The images for identifying good products and defective products are obtained, for example, by photographing each good product or each defective product with the photographing means 10, cropping an inspection area image from the obtained photographed image, detecting a reference part from the inspection area image, and cropping the inspection part using the relative coordinates with respect to the reference part. Defective products can also be created, for example, by attaching tape that mimics a crack to a good product. Next, for example, the obtained good product discrimination value, each defective product discrimination value, and the average value of each good product discrimination value + 4σ are displayed on the hiss and blam, and it is preferable to set a threshold between the average value of each good product discrimination value + 4σ and the lower limit of each defective product discrimination value (see Figures 5 and 6).

[0031] After the threshold setting procedure (step S120), the inspection target M is inspected for good or bad. First, the inspection target M is photographed using the imaging means 10 (imaging procedure; step S130) (see Figure 2). At this time, it is preferable to photograph a relatively wide area so that the inspection area is included in the imaging range even if the position of the inspection target M varies.

[0032] Next, for example, the inspection area cropping means 20 crops the area containing the inspection site from the captured image obtained by the imaging means 10 using pre-set fixed coordinates to obtain an inspection area image (inspection area cropping procedure; step S140) (see Figure 3). It is preferable that the cropping range of the captured image be a reasonably wide range so that the inspection site is included in the cropping range even if the position of the inspection site in the captured image varies.

[0033] Next, for example, the reference area detection means 30 detects a reference area from the obtained inspection area image, which will serve as a reference position for detecting the inspection area (reference area detection procedure; step S150) (see Figure 4). Specifically, for example, an object detection method is used to detect the reference area from the inspection area image. The object detection method is as described above.

[0034] Next, for example, the discriminant image cropping means 40 detects the inspection area from the obtained inspection area image based on the relative coordinates of the inspection area with respect to the reference area, and crops it as a discriminant image (discriminant image cropping procedure; step S150) (see Figure 4). That is, for example, from the inspection area image, the inspection area is detected and cropped based on the positional relationship between the reference area and the inspection area, which is registered in advance, based on the positional coordinates of the reference area, and this is used to obtain a discriminant image. In this way, when detecting the inspection area, if a reference area that can be detected with high accuracy is used, and the detection is performed based on the relative coordinates with respect to the reference area, the inspection area can be detected easily, stably and with high accuracy.

[0035] After the discriminant image cropping procedure (step S150), for example, the discriminant means 50 inputs the discriminant image into a learning model 51 that has been trained by machine learning to identify good products of the inspection area, and determines whether the inspection area is good or bad based on the obtained discriminant value (discrimination procedure; step S170). For example, it is preferable that the discriminant value obtained by inputting the discriminant image into the learning model 51 is considered good if it is below a preset threshold, and bad if it is above the threshold. In this case, as described above, the threshold is set using a defective product discrimination value in addition to a good product discrimination value, so the threshold accuracy is high and the judgment accuracy can be improved.

[0036] Subsequently, for example, the display means 60 displays the discrimination result obtained by the discrimination means 50 (display procedure; step S180). Also, for example, if the discrimination result obtained by the discrimination means 50 is poor, the alarm means 70 issues an alarm (alarm procedure; step S190). Examples of alarms include audible alarms and warning text displays on a display or the like.

[0037] As described above, according to this embodiment, a reference area is detected from the inspection area image to serve as a reference position for detecting the inspection area, and the inspection area is detected using the relative coordinates of the inspection area with respect to the reference area. Therefore, by using a reference area that can be detected with high accuracy, the inspection area can be detected with high accuracy, stably, easily, and in a short time. Thus, the inspection target M can be inspected easily and with high accuracy.

[0038] Furthermore, if the determination of whether an item is good or bad is made based on a threshold set between the average value + 4σ of each good item discrimination value obtained by inputting multiple good item discrimination images into the learning model 51, and the lower limit of each defective item discrimination value obtained by inputting multiple defective item discrimination images into the learning model 51, then since the threshold is set using the defective item discrimination value in addition to the good item discrimination value, the threshold accuracy can be increased, and the judgment accuracy can be improved.

[0039] The present invention has been described above with reference to embodiments, but the present invention is not limited to the above embodiments and can be modified in various ways. For example, although each component was described in detail in the above embodiments, the specific structure and shape of each component may differ, and the present invention does not have to include all of the above-mentioned components, but may include other components as well. [Explanation of Symbols]

[0040] 1...Inspection device, 10...Photography means, 20...Inspection area cutting means, 30...Reference part detection means, 40...Discrimination image cutting means, 50...Discrimination means, 51...Learning model, 52...Good / bad judgment unit, 53...Learning model generation means, 60...Display means, 70...Alarm means, 81...CPU, 82...ROM, 83...RAM, 84...HDD, 85...Operation interface, M...Inspection target

Claims

1. An inspection device for checking whether an object to be inspected is good or bad, A means of photographing the object to be inspected, An inspection area cropping means crops the area including the inspection site from the image captured by the aforementioned imaging means using fixed coordinates set in advance as an inspection area image, A reference area detection means for detecting a reference area that serves as a reference position for detecting the inspection area from the inspection area image, A discriminant image cropping means detects the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area, and crops it out as a discriminant image. A learning model that has been trained by machine learning to identify good products of the inspection area is input with the discriminant image, and a discriminant means that determines whether the inspection area is good or bad based on the obtained discriminant value. An inspection device characterized by being equipped with

2. The discrimination means determines that the obtained discrimination value is good if it is less than or equal to a preset threshold, and bad if it is greater than the threshold. The threshold is set between the average value of each good product discrimination value obtained by inputting multiple good product discrimination images into the learning model + 4σ and the lower limit of each defective product discrimination value obtained by inputting multiple defective product discrimination images into the learning model. The inspection apparatus according to claim 1, characterized in that it is a feature of the present invention.

3. An inspection method for checking whether an object to be inspected is good or bad, The procedure for photographing the object to be examined, A procedure for cropping an inspection area from the image obtained by the above-mentioned imaging procedure, in which the region including the area to be examined is cropped as an inspection area image using pre-set fixed coordinates, A reference area detection procedure for detecting a reference area that serves as a reference position for detecting the inspection area from the inspection area image, A discriminant image cropping procedure, which involves detecting the inspection area from the inspection area image using the relative coordinates of the inspection area with respect to the reference area, and cropping it as a discriminant image, A discrimination procedure is performed by inputting the discrimination image into a learning model that has been trained by machine learning to identify good products of the inspection part, and determining whether the inspection part is good or bad based on the obtained discrimination value. A testing method characterized by including [a certain component].

4. In the aforementioned determination procedure, if the obtained determination value is less than or equal to a predetermined threshold, it is determined to be good; if it is greater than the threshold, it is determined to be bad. The threshold is determined by displaying a histogram of the good product discrimination values ​​obtained by inputting multiple good product discrimination images into the learning model, and the defective product discrimination values ​​obtained by inputting multiple defective product discrimination images into the learning model, and setting the threshold between the mean value of each good product discrimination value + 4σ and the lower limit of each defective product discrimination value. The inspection method according to claim 3, characterized by the features described above.

Images