Inspection device and inspection method

Abstract

To provide an inspection device and inspection method that can perform inspections easily and with high accuracy. [Solution] The inspection device 1 inspects whether the coating is good or bad for a coated area on an object M, where a coating agent is applied in a strip shape with at least a portion of it curved. The inspection device 1 includes, for example, an image acquisition means 10 that photographs the coated area while moving the photographing means 11 along the coated area relative to the object M, and acquires multiple photographic images that divide the coated area into multiple parts in the direction of extension, and a discrimination means 20 that determines whether the coating is good or bad based on the photographic images obtained by the image acquisition means 10 and a learning model 21 that has learned the characteristics of the coated area by machine learning.

Description

Technical Field

[0001] The present invention relates to an inspection apparatus and an inspection method for inspecting the quality of a coating state of a coating portion obtained by applying a coating agent to an object in a band shape with at least a part thereof curved.

Background Art

[0002] In the manufacturing process of automobiles, for example, when assembling a vehicle body, there is a process of applying a coating agent such as a sealing agent to a predetermined position. In this coating process, inspections are carried out on the quality of the coating state of the coating portion where the coating agent is applied, such as whether there is displacement, breakage, or holes. Conventionally, for example, an inspector visually inspected the quality of the coating state. However, there was a problem that variations occurred in the determination in the visual inspection by the inspector.

[0003] In Patent Document 1, regarding a coating agent inspection apparatus for determining the quality of the coating state of a coating agent applied to an object, after photographing the adhesive coating part with a camera while partially overlapping, the captured image is divided to detect the edge position, the center position between the edges is calculated, and the quality of the coating state is determined by comparing the calculated data with the master data. However, in the apparatus of Patent Document 1, although displacement and breakage of the coating portion can be discriminated, a discrimination method for holes is not disclosed, and there was a problem that holes cannot be discriminated from the deviation of coordinate values.

[0004] In addition, Patent Document 2 describes an abnormality detection apparatus that extracts an inspection target part from a captured image of an inspection target by a feature part, then divides the image, and determines the quality of the inspection target by a model that has learned a plurality of pre-divided learning divided images. However, in the apparatus of Patent Document 2, for example, when inspecting a coating portion obtained by applying a coating agent in a band shape with at least a part thereof curved, a large number of learning divided images have to be prepared, which is time-consuming.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Patent No. 6550240 [Patent Document 2] Patent No. 7535484 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] This invention was made based on these problems and aims to provide an inspection device and inspection method that can perform inspections easily and with high accuracy. [Means for solving the problem]

[0007] The inspection apparatus of the present invention inspects whether the coating is good or bad for a coated portion of an object in which a coating agent has been applied in a strip shape with at least a portion curved. The apparatus comprises an image acquisition means that photographs the coated portion while moving the photographing means along the coated portion relative to the object, and acquires multiple photographic images that divide the coated portion into multiple parts in the direction of extension, and a discrimination means that determines whether the coating is good or bad based on the photographic images obtained by the image acquisition means and a learning model that has learned the characteristics of the coated portion by machine learning.

[0008] The present invention provides an inspection method for inspecting the quality of a coating applied to an object in a strip-like manner, with at least a portion of the coating curved. The method includes an image acquisition procedure in which a photographing means is moved relative to the object along the coating, and multiple images are obtained by dividing the coating into multiple sections in the direction of extension. The method also includes a determination procedure in which the captured images obtained in the image acquisition procedure are used to determine whether the coating is good or bad based on a learning model that has learned the characteristics of the coating through machine learning. [Effects of the Invention]

[0009] According to the present invention, the photographing means is moved relative to the object along the coated area to photograph the coated area, and multiple images are acquired by dividing the coated area into multiple sections in the extending direction. As a result, the coated area is displayed linearly in each image. Therefore, the number of learning models can be reduced, and even if the object is different, the same learning model can be used as long as the width of the coated area is the same. Furthermore, for each image in which the coated area is divided into multiple sections in the extending direction, the quality of the coating is determined based on the learning model that has learned the characteristics of the coated area. As a result, not only misalignment and breaks in the coated area but also holes can be detected. Consequently, the coating condition can be easily inspected with high accuracy.

[0010] Furthermore, by extracting the coated portion from the captured image based on the characteristic features of the object, the coated portion can be easily extracted. Therefore, the positional shift of the coated portion due to the shift in the shooting position can be easily corrected, improving the accuracy of cropping the coated portion and improving the accuracy of determining the coating state.

[0011] In addition, if the system includes a position correction means for correcting the position of the imaging means relative to the object, then, for example, when the structure is complex and it is difficult to clearly photograph the coated area if the imaging position is not appropriate, the accuracy of cropping the coated area can be improved by correcting the imaging position, thereby further improving the accuracy of determining the coating state. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows the configuration of an inspection device according to one embodiment of the present invention. [Figure 2] This figure schematically represents an example of a coated area to be inspected using the inspection device shown in Figure 1. [Figure 3] This is a diagram showing an example of a captured image. [Figure 4] This diagram illustrates the difference between moving the imaging device along the curve of the coating area and moving it in a straight line without following the curve. [Figure 5] It is a figure schematically showing an example of a captured image of a portion continuing from the roof of the coating part to the back side. [Figure 6] It is a figure showing an example of extracting the coating part from the captured image. [Figure 7] It is a figure showing an example of cutting out the coating part from the captured image. [Figure 8] It is a figure schematically showing an example of a discrimination image. [Figure 9] It is a figure showing a configuration example of the hardware of the position correction means and the discrimination means shown in FIG. 1. [Figure 10] It is a figure showing the flow of the inspection method according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] FIG. 1 shows the configuration of an inspection apparatus 1 according to an embodiment of the present invention. FIG. 2 schematically shows an example of a coating part M1 inspected by the inspection apparatus 1. The inspection apparatus 1 inspects the quality of the coating state of a coating part M1 in which a coating agent is applied to an object M in a curved带状 shape at least in part. Specifically, the inspection apparatus 1 can be used, for example, in the manufacturing process of an automobile, to inspect the quality of the coating state of a coating part M1 in which a coating agent such as a sealing agent is applied to a vehicle body as the object M in a curved带状 shape at least in part.

[0015] In this embodiment, the case of inspecting a coating part M1 obtained by applying a coating agent to the roof M2 of a vehicle body as the object M will be described as an example. This coating part M1 extends in the longitudinal direction of the vehicle body along the side member M3, for example, at the upper part of the roof M2, is gently curved toward the center in the width direction toward the central part in the longitudinal direction, and is provided in a带状 shape, and is continuously provided in a带状 shape from the roof M2 to the back side at the rear side of the vehicle body.

[0016] The inspection device 1, for example, photographs the application part M1 while moving the photographing means 11 relatively to the object M along the application part M1, and acquires a plurality of photographed images 11A obtained by dividing the application part M1 into a plurality in the extending direction. The inspection device 1 includes an image acquisition means 10, and a discrimination means 20 that discriminates whether the application state is good or bad based on a learning model 21 obtained by learning the characteristics of the application part by machine learning for the photographed images 11A obtained by the image acquisition means 10. The inspection device 1 preferably further includes display means 30 such as a display for displaying the discrimination result of the discrimination means 20.

[0017] The image acquisition means 10 preferably includes, for example, a photographing means 11, a moving means 12 for relatively moving the photographing means 11 to the object M, and a lighting means 13 for irradiating light on the application part M1. The photographing means 11 is constituted by a camera such as a CCD camera, for example. The moving means 12 can be constituted by an industrial robot that self-runs on a slide rail (not shown) extending in the front-rear direction of the vehicle body which is the object M, and the photographing means 11 and the lighting means 13 are disposed on the industrial robot. In FIG. 1, the moving means 12 is conceptually shown.

[0018] The photographing means 11 is preferably configured to move from the front side to the rear side of the vehicle body along the application part M1 provided on the upper part of the roof M2 of the vehicle body which is the object M, and to move from the roof M2 to the back side along the application part M1 at the rear side of the vehicle body. Further, the photographing means 11 is preferably configured to obtain a plurality of photographed images 11A obtained by photographing the application part M1 by dividing it into a plurality in the extending direction while moving along the application part M1. FIG. 3 shows an example of the photographed image 11A on the roof M2. In FIG. 3, the white part extending in the horizontal direction is the part of the application part M1.

[0019] The reason for moving the photographing means 11 along the coated area M1 and photographing the coated area M1 in multiple sections is that even if the coated area M1 is curved, the position and shape of the coated area M1 in each photographed image 11A of the roof M2 can be made the same. Figure 4 shows a comparison of the case where the photographing means 11 is moved linearly in the front-rear direction of the vehicle body without following the curve of the coated area M1 and the photograph is taken, and the case where the photographing means 11 is moved along the curve of the coated area M1 and the photograph is taken. In Figure 4, for clarity, the coated area M1 and the portion of the coated area M1 in the photographed image 11A are shown with a thick gray line.

[0020] As shown in Figure 4(A), when the shooting means 11 is moved linearly in the longitudinal direction of the vehicle body, the position and shape of the coated area M1 differ in each captured image 11A of the roof M2. However, as shown in Figure 4(B), when the shooting means 11 is moved along the curvature of the coated area M1, the coated area M1 is displayed linearly in a predetermined position in each captured image 11A of the roof M2. This makes it possible to reduce the number of learning models 21.

[0021] The imaging means 11 may, for example, capture each image 11A as a still image, or it may capture a video and extract each image 11A. It is preferable that each image 11A is partially overlapped with adjacent images 11A so that there are no parts of the coated area M1 that are not captured. For example, if the coated area M1 is divided into multiple sections in the extending direction and captured, it is preferable to capture them with some overlap.

[0022] Preferably, the image acquisition means 10 also includes a position correction means 14 for correcting the position of the shooting means 11 relative to the object M. This is because, depending on the shooting position, the portion of the coating area M1 that extends from the roof M2 to the rear side of the vehicle body may be obscured by the back door M4. Figure 5 schematically shows an example of a captured image 11A of the portion of the coating area M1 that extends from the roof M2 to the rear side. In Figure 5, the portion of the coating area M1 is shown in gray for clarity.

[0023] The position correction means 14 is preferably configured to correct the position of the imaging means 11 relative to the object M when, for example, the structure of the object M is complex and it is difficult to clearly photograph the coated portion M1 if the shooting position is not appropriate. This is to improve inspection accuracy while shortening the time required for inspection by performing position correction only when necessary. In this embodiment, for example, when photographing the portion of the coated portion M1 that extends from the roof M2 to the rear side, it is preferable to correct the position of the imaging means 11 to a position where it is easy to photograph the coated portion M1 between the back door M4 and the side member M3.

[0024] The position correction means 14 can be configured, for example, by a computer, and is configured to function as a position correction means 14 by executing a program. Preferably, the position correction means 14 is configured to obtain the coordinates of the feature portion of the object M from the captured image 11A obtained by the shooting means 11 by template matching or the like, and to adjust the position of the shooting means 11 so that the coordinates of the feature portion of the object M are at a predetermined coordinate position. An example of a feature portion of the object M is the part of the coated portion M1 that curves from the roof M2 toward the back side. In Figure 5, the area enclosed by the dashed line is the part detected by template matching, and its center coordinates are indicated by the dotted circle.

[0025] Preferably, the discrimination means 20 includes, for example, a learning model 21 that has learned the features of the coated portion M1 by machine learning; an extraction means 22 that extracts the portion of the coated portion M1 from the captured image 11A obtained by the image acquisition means 10 based on the features of the object M; an extraction means 23 that cuts out the portion of the coated portion M1 extracted by the extraction means 22; a division means 24 that divides the portion of the coated portion M1 cut out by the extraction means 23 into multiple parts in the length direction to form a discrimination image 24A; a good / bad determination means 25 that determines whether the coating state is good or bad based on the discrimination image 24A and the learning model 21; and a learning model generation means 26 that generates the learning model 21.

[0026] The discrimination means 20, namely the learning model 21, extraction means 22, cutting means 23, division means 24, good / bad judgment means 25, and learning model generation means 26, can be configured, for example, by a computer, and are configured to function as the discrimination means 20, namely the learning model 21, extraction means 22, cutting means 23, division means 24, good / bad judgment means 25, and learning model generation means 26 by executing a program.

[0027] The learning model 21 is preferably trained only on good products of the coated area M1. This is because there are countless shapes of defects, making it difficult to model them. Specifically, the learning model 21 is preferably trained on images 24A of good products obtained by extracting and cutting out the coated area M1 portion from captured images 11A of good products of the coated area M1, and dividing it into multiple parts in the length direction.

[0028] Furthermore, in this inspection device 1, the imaging means 11 is moved relative to the coated area M1 to acquire multiple images 11A that divide the coated area M1 into multiple sections. As a result, the portion of the coated area M1 in each image 11A becomes linear, and the good product identification image 24A can be standardized. Therefore, with this inspection device 1, the number of learning models 21 can be reduced, and even if the type of vehicle body, which is the target object M, is different, the same learning model 21 can be used as long as the width of the coated area M1 is the same.

[0029] The extraction means 22 corrects the positional shift of the coated portion M1 in the captured image 11A by extracting the coated portion M1 from each captured image 11A, thereby improving the accuracy of cropping the coated portion M1. Preferably, the extraction means 22 is connected to the image acquisition means 10 and is configured to extract the coated portion M1 from each captured image 11A based on the characteristic parts of the target object M using template matching or the like. An example of a characteristic part of the target object M is the uneven surface of the roof of a car body. In Figure 6, the coated portion M1 extracted from the captured image 11A of the roof M2 is enclosed in a thick dashed line. Extracting the coated portion M1 based on the characteristic parts of the target object M in this way is preferable because it allows for easy extraction.

[0030] Preferably, the cutting means 23 is connected to the extraction means 22 and is configured to cut out the coated portion M1 by masking the portion of the coated portion M1 other than the portion of the coated portion M1 extracted by the extraction means 22. Figure 7 shows an example in which the portion of the coated portion M1 is cut out from the captured image 11A by masking the portion of the coated portion M1 other than the portion of the coated portion M1.

[0031] The dividing means 24 is connected, for example, to the cutting means 23. The dividing means 24 is intended to improve the accuracy of the judgment by dividing the portion of the coated area M1 cut out by the cutting means 23 into multiple parts. Figure 8 shows an example of a judgment image 24A. In Figure 8, the portion of the coated area M1 is shown in gray for clarity. Figure 8(A) schematically shows a judgment image 24A of a good product, Figure 8(B) schematically shows a judgment image 24A of a defective product where the coated area M1 is cut, Figure 8(C) schematically shows a judgment image 24A of a defective product where the coated area M1 has a hole, and Figure 8(D) schematically shows a judgment image 24A of a defective product where the coated area M1 is only half coated. It is preferable that each judgment image 24A is divided so that a portion overlaps.

[0032] The good / bad judgment means 25 is preferably configured to determine whether the coating state is good or bad based on the judgment value (score value) obtained by inputting each judgment image 24A into a learning model 21, for example. Specifically, it is preferable that the learning model 21 is configured to determine whether the coating state is good if the judgment value obtained is below a preset threshold, and whether it is bad if it is greater than the threshold. The threshold can be set, for example, based on the average value of each good judgment value obtained by inputting a plurality of good judgment images 24A into the learning model 21 + 4σ (σ is the standard deviation).

[0033] The learning model generation means 26 is preferably configured to prepare multiple discrimination images 24A of good coated parts M1 as positive examples of discrimination images 24A, and to learn the features of the inspection area by deep learning. For deep learning, it is preferable to use a convolutional neural network (CNN), such as EfficientNet. The learning image can be obtained, for example, by extracting the portion of the coated part M1 from an image 11A taken by the imaging means 11 of a good coated part M1 using the extraction means 22, cutting it out with the cutting means 23, and dividing it into multiple parts in the length direction.

[0034] Figure 9 shows an example of the hardware configuration of the position correction means 14 and the discrimination means 20. The position correction means 14 and the discrimination means 20 include, for example, a CPU (Center Processing Unit) 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, an HDD (Hard Disk Drive) 44, and an operation interface (operation I / F) 45. The CPU 41 executes various processes according to various programs recorded in the ROM 42 or various programs loaded from the HDD 44 into the RAM 43. The RAM 43 also appropriately stores data necessary for the CPU 41 to execute various processes. The HDD 44 stores various data.

[0035] The inspection device 1 is used, for example, as follows. Figure 10 shows the flow of the inspection method using the inspection device 1. First, as a preparation procedure, a learning model 21 is generated by the learning model generation means 26 (preparation procedure; step S110). In the learning model generation procedure (step S110), first, for example, a good product of the coated part M1 is photographed by the photography means 11, the portion of the coated part M1 is extracted from the obtained photographed image 11A by the extraction means 22, cut out by the cutting means 23, and then divided into multiple parts in the length direction to prepare a learning image. Next, for example, the features of the coated part M1 are learned by deep learning using the learning image to generate the learning model 21.

[0036] Next, for example, in the automobile production process, the learning model 21 is used to inspect the quality of the coating of the coated area M1 on the car body M. First, for example, the imaging means 11 is moved along the coated area M1 and multiple images 11A are obtained, dividing the coated area M1 into multiple sections in the extending direction (image acquisition procedure; step S120). When photographing the portion of the coated area M1 that extends from the roof M2 to the rear, it is preferable to correct the position of the imaging means 11 using the position correction means 14.

[0037] Next, the discrimination means 20 determines whether the coating condition is good or bad based on the image acquisition procedure (step S120) obtained by the captured image 11A (discrimination procedure; step S130).

[0038] In the discrimination procedure (step S130), first, for example, the extraction means 22 extracts the coated portion M1 from each captured image 11A based on the characteristic parts of the object M (extraction procedure; step S131). Next, for example, the cutting means 23 cuts out the coated portion M1 (cutting procedure; step S132), and then the dividing means 24 divides the coated portion M1 into multiple parts in the length direction to obtain a discrimination image 24A (dividing procedure; step S133). Next, for example, the discrimination image 24A is input to the learning model 21, and the good / bad state of the coating is determined from the obtained discrimination value (good / bad determination procedure; step S134). Specifically, for example, if the discrimination value is below a preset threshold, it is determined to be good, and if it is greater than the threshold, it is determined to be bad.

[0039] Subsequently, the display means 30 displays the result determined by the determination means 20 (display procedure; step S140).

[0040] As described above, according to this embodiment, the photographing means 11 is moved relative to the object M along the coated area M1 to photograph the coated area M1, and multiple photographed images 11A are obtained by dividing the coated area M1 into multiple parts in the extending direction. As a result, in each photographed image 11A, the portion of the coated area M1 can be displayed linearly. Therefore, the number of learning models 21 can be reduced, and even if the object M is different, the same learning model 21 can be used as long as the width of the coated area M1 is the same. Furthermore, for each photographed image 11A that divides the coated area M1 into multiple parts in the extending direction, the good or bad state of the coating is determined based on the learning model 21 that has learned the characteristics of the coated area M1. As a result, not only misalignment and cuts in the coated area M1 but also holes can be determined. Consequently, the coating state can be easily inspected with high accuracy.

[0041] Furthermore, by extracting the coated portion M1 from the captured image 11A based on the characteristic features of the object M, the coated portion M1 can be easily extracted. Therefore, the positional shift of the coated portion M1 due to the shift in the shooting position can be easily corrected, improving the accuracy of cutting out the coated portion M1 and improving the accuracy of determining the coating state.

[0042] In addition, if the system includes a position correction means 14 for correcting the position of the shooting means 11 relative to the object M, for example, if the structure is complex and it is difficult to clearly photograph the coated portion M1 if the shooting position is not appropriate, the accuracy of cropping the coated portion M1 can be improved by correcting the shooting position, thereby further improving the accuracy of determining the coating state.

[0043] The present invention has been described above with reference to embodiments, but the present invention is not limited to the above embodiments and can be modified in various ways. For example, although each component was described in detail in the above embodiments, the specific structure and shape of each component may differ, and the present invention does not have to include all of the above-mentioned components, but may include other components as well. [Explanation of Symbols]

[0044] 1...Inspection device, 10...Image acquisition means, 11...Shooting means, 11A...Shooting image, 12...Movement means, 13...Illumination means, 14...Position correction means, 20...Discrimination means, 21...Learning model, 22...Extraction means, 23...Cutting means, 24...Dividing means, 24A...Discrimination image, 25...Good / bad judgment means, 26...Learning model generation means, 30...Display means, 41...CPU, 42...ROM, 43...RAM, 44...HDD, 45...Operation interface, M...Object, M1...Coated part, M2...Roof, M3...Side member, M4...Back door

Claims

1. An inspection device for inspecting the quality of a coating applied to an object, where the coating agent is applied in a strip-like manner, with at least a portion of it curved, to determine whether the coating is good or bad. An image acquisition means that photographs the coated portion while moving the photographing means along the coated portion relative to the object, and acquires multiple photographic images that divide the coated portion into multiple sections in the extending direction, A discrimination means determines whether the coating state is good or bad based on the captured image obtained by the image acquisition means, using a learning model that has learned the characteristics of the coated area by machine learning. An inspection device characterized by being equipped with

2. The aforementioned determination means is The aforementioned learning model, An extraction means for extracting the portion of the coated area from the captured image obtained by the image acquisition means, based on the characteristic features of the object, A cutting means for cutting out the portion of the coated area extracted by the extraction means, A division means for dividing the portion of the coated area cut out by the cutting means into multiple parts in the length direction to form a discriminant image, A good / bad determination means for determining whether the coating state is good or bad based on the discriminant image and the learning model. The inspection apparatus according to claim 1, characterized by having the following features.

3. The aforementioned image acquisition means is The aforementioned photographic means, A moving means for moving the aforementioned photographic means relative to the aforementioned object, Position correction means for correcting the position of the shooting means relative to the object, The inspection apparatus according to claim 1, characterized by having the following features.

4. An inspection method for checking the quality of a coating applied to an object, where the coating agent is applied in a strip-like manner, with at least a portion of it curved, and for inspecting the quality of the coating. An image acquisition procedure comprising: moving a shooting means relative to the object along the coating area while photographing the coating area, and acquiring multiple images that divide the coating area into multiple sections in the direction of extension; A discrimination procedure is performed to determine whether the coating is good or bad based on the captured image obtained by the above image acquisition procedure, using a learning model that has learned the characteristics of the coated area by machine learning. A testing method characterized by including [a certain component].

Images