Workpiece surface defect detection device and detection method, workpiece surface inspection system and program

Abstract

Provided are surface defect detection devices for workpieces that are capable of stably detecting small surface defects with high precision. In a state where a light and dark pattern caused by an illumination device is relatively moved relative to a workpiece as a surface defect detection target, multiple images of a measured portion of the workpiece are acquired to extract tentative defect candidates. If the number of images including the tentative defect candidate among the multiple images from which the tentative defect candidate is extracted is greater than a preset threshold, the tentative defect candidate is determined as a defect candidate. A composite image is produced by synthesizing multiple images including the determined defect candidate, and defect detection is performed based on the produced composite image.

Description

Technical Field

[0001] The present invention relates to a workpiece surface defect detection device and detection method, a workpiece surface inspection system and a program for detecting surface defects based on a plurality of images obtained by continuously photographing a measured portion such as a painted surface of the workpiece while a workpiece such as a vehicle body, which is a surface defect detection target, is moved relative to a lighting device and illumination light of a light and dark pattern is irradiated onto the measured portion such as a painted surface of the workpiece by a camera unit. Background Art

[0002] As described above, as a technique for detecting surface defects of a workpiece by continuously photographing a measured portion of the workpiece with an imaging unit while moving the workpiece such as a vehicle body relative to a lighting device, Patent Document 1 is known. However, the technique described in Patent Document 1 does not describe a specific defect detection algorithm using images obtained by the imaging unit.

[0003] In addition, Patent Document 2 describes the detection of surface defects by fusing a plurality of images into one image.

[0004] In addition, Patent Document 3 proposes a defect inspection method for an inspected surface that detects defects based on the moving state of defect candidate points extracted from each image by processing a plurality of images captured in a time series, so that defects can be detected with high accuracy. Specifically, the inspected surface 1 is moved in a manner that its imaging portion A moves with the passage of time, an image is input from an imaging unit 2 at any time, a defect candidate point B is extracted from each of two images whose imaging times are different from each other, an error range is set around the defect candidate point B in the image captured later, the defect candidate point B in the image captured earlier is moved in accordance with the moving distance of the inspected surface or the imaging unit, and the defect candidate point closest to the defect candidate point B after the movement among the defect candidate points existing in the error range is used as a matching candidate, the above processing is performed on a plurality of images captured in a time series, and the defect candidate point B that has been cited as a matching candidate more than a predetermined number of times is determined to be a defect existing on the inspected surface 1.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: U.S. Patent No. 7,639,349

[0008] Patent Document 2: U.S. Patent Application Publication No. 2017 / 0277979

[0009] Patent Document 3: Japanese Patent No. 3156602 Summary of the invention

[0010] Problems to be solved by the invention

[0011] However, the inspection method described in Patent Document 2 has a problem in that it is necessary to perform first to third transformation processes that are different from each other on the fused image, and thus the defect detection process is complicated.

[0012] Furthermore, the inspection method described in Patent Document 3 has a problem that since processing is basically performed based on a single image, the amount of information is insufficient and small surface defects cannot be identified with sufficient accuracy.

[0013] The present invention is made to solve such problems, and its purpose is to provide a workpiece surface defect detection device and detection method, a workpiece surface inspection system and program that can stably detect small surface defects with high precision without multiple complex transformation processes.

[0014] Means of solving problems

[0015] The above object is achieved by the following means.

[0016] (1) A surface defect detection device for a workpiece, comprising: an image acquisition unit, which acquires a plurality of images of a measured portion of the workpiece while causing a light and dark pattern caused by an illumination device to move relative to the workpiece as a surface defect detection object; a provisional defect candidate extraction unit, which extracts a provisional defect candidate for each image acquired by the image acquisition unit; a defect candidate determination unit, which determines the provisional defect candidate as a defect candidate if the number of images including the provisional defect candidate among the plurality of images from which the provisional defect candidate is extracted by the provisional defect candidate extraction unit is greater than a predetermined threshold value; an image synthesis unit, which synthesizes the plurality of images including the defect candidate determined by the defect candidate determination unit to produce a synthesized image; and a detection unit, which performs defect detection based on the synthesized image produced by the image synthesis unit.

[0017] (2) A surface defect detection device for a workpiece according to the preceding item 1, wherein, when the threshold is set to a first threshold, the defect candidate determination unit terminates the process of determining a provisional defect candidate as a defect candidate when the number of images including the provisional defect candidate is less than the first threshold and has not reached a predetermined second threshold in the middle of the process of determining the provisional defect candidate as a defect candidate.

[0018] (3) A surface defect detection device for a workpiece according to item 1 or 2 above, wherein if the image corresponding to the position of the tentative defect candidate in the multiple images is above a predetermined threshold, the defect candidate determination unit determines the tentative defect candidate as a defect candidate.

[0019] (4) A surface defect detection device for a workpiece according to item 3 above, wherein the position of the tentative defect candidate is represented by the coordinates of the tentative defect candidate, and the surface defect detection device for the workpiece includes a coordinate estimation unit, which obtains the coordinates of the tentative defect candidate extracted by the tentative defect candidate extraction unit, and calculates the coordinates to which the coordinates of the tentative defect candidate are to be moved for each of a plurality of subsequent images of the image from which the tentative defect candidate is extracted to obtain the estimated coordinates, and the defect candidate determination unit determines whether the estimated coordinates calculated by the coordinate estimation unit in each of the subsequent images correspond to the tentative defect candidate in the image, and if there is an image in which the estimated coordinates and the tentative defect candidate correspond in the subsequent image that is greater than a predetermined threshold, the tentative defect candidate is determined as a defect candidate.

[0020] (5) A surface defect detection device for a workpiece according to any one of items 1 to 4 above, wherein the surface defect detection device for the workpiece comprises an image group production unit, which, for each defect candidate determined by the defect candidate determination unit, cuts out a predetermined area around the defect candidate from a plurality of images including the defect candidate as an inference area to produce a plurality of inference area image groups, and the image synthesis unit synthesizes the plurality of inference area images produced by the image group production unit to produce a synthesized image for each defect candidate.

[0021] (6) A workpiece surface defect detection device according to any one of the preceding items 1 to 5, wherein the composite image is at least any one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

[0022] (7) The workpiece surface defect detection device according to the above Item 5, wherein the synthesis unit performs alignment of the plurality of estimated region images when creating the synthesized image by overlapping the center coordinates of the images.

[0023] (8) A surface defect detection device for a workpiece according to item 7 above, wherein the synthesis unit corrects the center coordinates of each image based on the position of each center coordinate from the boundary of the light and dark pattern of the image corresponding to the light and dark pattern of the lighting device, so that the corrected center coordinates overlap.

[0024] (9) A surface defect detection device for a workpiece according to any one of items 5 to 8 above, wherein the synthesis unit aligns multiple estimated area images when creating a synthetic image in a manner that maximizes the evaluation value among multiple combinations obtained by shifting the center coordinates of each image in the X-coordinate and Y-coordinate directions.

[0025] (10) A surface defect detection device for a workpiece according to any one of the preceding items 1 to 9, wherein the provisional defect candidate extraction unit extracts provisional defect candidates using at least two of a routine for extracting small defects and a processing routine for extracting gradual convex defects, and performs defect size correction on the provisional defect candidates detected in the routine for gradual convex defects.

[0026] (11) A surface defect detection device for a workpiece according to any one of items 1 to 10 above, wherein the detection unit determines a linear protrusion based on an amount of deviation of a two-dimensional defect signal in a phase image of a defect candidate from a perfect roundness of a two-dimensional shape.

[0027] (12) A surface inspection system for a workpiece, comprising: an illumination device for irradiating a workpiece as a surface defect detection object with illumination light having a light and dark pattern; a camera unit for capturing reflected light of the illumination light of the illumination device from a measured portion of the workpiece; a moving unit for causing the light and dark pattern caused by the illumination device to move relative to the workpiece; a control unit for controlling the camera unit so that the measured portion of the workpiece is captured while the light and dark pattern of the illumination device is moved relative to the workpiece by the moving unit; and a surface defect detection device for a workpiece as described in any one of the preceding items 1 to 11, wherein the image acquisition unit acquires a plurality of images of the measured portion of the workpiece from the camera unit.

[0028] (13) A method for detecting surface defects of a workpiece, wherein a surface defect detection device for the workpiece executes: an image acquisition step of acquiring a plurality of images of a measured portion of the workpiece while a light and dark pattern caused by an illumination device is relatively moved relative to the workpiece as a surface defect detection object; a provisional defect candidate extraction step of extracting a provisional defect candidate for each image acquired by the image acquisition step; a defect candidate determination step of determining the provisional defect candidate as a defect candidate if the number of images including the provisional defect candidate among the plurality of images from which the provisional defect candidate is extracted by the provisional defect candidate extraction step is greater than a predetermined threshold; an image synthesis step of synthesizing a plurality of images including the defect candidate determined by the defect candidate determination step to produce a synthesized image; and a detection step of performing defect detection based on the synthesized image produced by the image synthesis step.

[0029] (14) A surface defect detection method for a workpiece according to item 13 above, wherein, when the threshold is set to a first threshold, the defect candidate determination unit terminates the process of determining a provisional defect candidate as a defect candidate when the number of images including the provisional defect candidate is less than the first threshold and has not reached a predetermined second threshold in the middle of the process of determining the provisional defect candidate as a defect candidate.

[0030] (15) A method for detecting surface defects of a workpiece according to item 13 or 14 above, wherein, in the defect candidate determination step, if the image corresponding to the position of the temporary defect candidate in the multiple images is above a predetermined threshold, the temporary defect candidate is determined as a defect candidate.

[0031] (16) A method for inspecting a surface defect of a workpiece according to item 15 above, wherein the position of the tentative defect candidate is represented by the coordinates of the tentative defect candidate, and the method for inspecting the surface defect of the workpiece comprises a coordinate estimation step, in which the coordinates of the tentative defect candidate extracted by the tentative defect candidate extraction step are obtained, and for each of a plurality of subsequent images of the image from which the tentative defect candidate is extracted, the coordinates to which the coordinates of the tentative defect candidate are to be moved are calculated to obtain the estimated coordinates, and in the defect candidate determination step, it is determined whether the estimated coordinates calculated by the coordinate estimation step in each of the subsequent images correspond to the tentative defect candidate in the image, and if there is an image in which the estimated coordinates and the tentative defect candidate correspond to each other in the subsequent image and the value is greater than a predetermined threshold, the tentative defect candidate is determined as a defect candidate.

[0032] (17) A method for inspecting workpiece surface defects according to any one of items 13 to 16, wherein the method for inspecting workpiece surface defects comprises an image group producing step, in which, for each defect candidate determined by the defect candidate determining step, a predetermined area around the defect candidate is cut out from a plurality of images including the defect candidate as an inference area, and a plurality of inference area image groups are produced, and in the image synthesis step, for each defect candidate, a plurality of inference area images produced by the image group producing step are synthesized to produce a synthesized image.

[0033] (18) A method for detecting surface defects of a workpiece according to any one of items 13 to 17, wherein the composite image is at least one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

[0034] (19) A method for detecting surface defects of a workpiece according to item 17 above, wherein in the synthesis step, the plurality of estimated region images are aligned when the synthesized image is produced by overlapping the center coordinates of the images.

[0035] (20) A method for detecting surface defects of a workpiece according to item 19 above, wherein, in the synthesis step, the center coordinates of each image are corrected based on the position of each center coordinate from the boundary of the light and dark pattern of the image corresponding to the light and dark pattern of the lighting device so that the corrected center coordinates overlap.

[0036] (21) A method for detecting surface defects of a workpiece according to any one of items 17 to 20 above, wherein, in the synthesis step, the plurality of estimated area images are aligned when producing the synthesized image in such a manner that the evaluation value is maximized among a plurality of combinations obtained by mutually offsetting the center coordinates of each image in the X-coordinate and Y-coordinate directions.

[0037] (22) A method for detecting surface defects of a workpiece according to any one of items 13 to 21 above, wherein, in the step of extracting the provisional defect candidates, provisional defect candidates are extracted using at least two of a routine for extracting small defects and a processing routine for extracting gradual convex defects, and defect size correction is performed on the provisional defect candidates detected in the routine for gradual convex defects.

[0038] (23) A method for detecting surface defects of a workpiece according to any one of items 13 to 22 above, wherein, in the detection step, a linear protrusion is identified based on the amount of deviation of a two-dimensional defect signal in a phase image of a defect candidate from the roundness of a two-dimensional shape.

[0039] (24) A program for causing a computer to execute the workpiece surface defect detection method described in any one of the preceding items 13 to 23.

[0040] Effects of the Invention

[0041] According to the invention described in the preceding items (1), (12) and (13), a plurality of images of a measured portion of the workpiece are acquired while a light and dark pattern caused by an illumination device is relatively moved relative to a workpiece as a detection target of a surface defect, and a tentative defect candidate is extracted for each acquired image. If the number of images including the tentative defect candidate among the plurality of images from which the tentative defect candidate is extracted is greater than a predetermined threshold, the tentative defect candidate is determined to be a defect candidate, and a composite image is produced by synthesizing the plurality of images including the determined defect candidate. Then, defect detection is performed based on the produced composite image.

[0042] That is, multiple images including defect candidates are synthesized into one image, and defect detection is performed based on the synthesized image, so the synthesized image includes information from multiple images. Therefore, defect detection can be performed using a large amount of information for one defect candidate, so even small surface defects can be detected stably with high accuracy while suppressing over-detection and false detection.

[0043] In addition, when the number of images including provisional defect candidates is above a predetermined threshold, a composite image is created and defect detection is performed. Therefore, defect detection can be performed when the possibility of defects is high, the processing load is reduced, the detection efficiency is improved, and the detection accuracy is also improved.

[0044] Furthermore, there is no need to perform a plurality of different transformation processes on the fused image as described in Patent Document 2.

[0045] According to the invention described in the preceding items (2) and (14), the defect candidate determination unit terminates the process of determining the provisional defect candidate as a defect candidate when the number of images including the provisional defect candidate is less than the first threshold value and has not reached the predetermined second threshold value in the middle of the process of determining the provisional defect candidate as a defect candidate. This prevents useless processing from being continued even when the possibility that the provisional defect candidate is a defect candidate is not high, thereby further reducing the processing load and further improving the detection accuracy.

[0046] According to the invention described in the above items (3) and (15), if there are images corresponding to the position of the tentative defect candidate in a plurality of images that are equal to or greater than a preset threshold, the tentative defect candidate is determined to be a defect candidate.

[0047] According to the invention described in the preceding items (4) and (16), the coordinates of the extracted provisional defect candidate are obtained, and for each of the subsequent multiple images of the image from which the provisional defect candidate is extracted, the coordinates to which the coordinates of the provisional defect candidate are to be moved are calculated to obtain the estimated coordinates, and it is determined whether the estimated coordinates correspond to the provisional defect candidate in the image. If, in the subsequent images, the image corresponding to the estimated coordinates and the provisional defect candidate exists above a predetermined threshold, the provisional defect candidate is determined to be a defect candidate, so that it is possible to reliably determine whether the image corresponding to the position of the provisional defect candidate in the multiple images exists above a predetermined threshold.

[0048] According to the invention described in the preceding items (5) and (17), a plurality of estimated area images cut out from a plurality of images including defect candidates are synthesized into one image, and defect detection is performed based on the synthesized image, so that only the area required for defect detection can be synthesized, enabling high-precision defect detection.

[0049] According to the invention described in the preceding items (6) and (18), surface defects can be detected based on at least any composite image among the standard deviation image, phase image, phase difference image, maximum value image, minimum value image, and average value image.

[0050] According to the invention described in the preceding items (7) and (19), by overlapping the center coordinates of each image, alignment of multiple estimated area images is performed when producing a composite image, so that a highly accurate composite image can be produced and high-precision surface defect detection can be performed.

[0051] According to the invention described in the preceding items (8) and (20), the center coordinates of each image are corrected according to the position of each center coordinate from the boundary of the light and dark pattern of the image corresponding to the light and dark pattern of the lighting device, so that the corrected center coordinates overlap, thereby further improving the accuracy of the composite image.

[0052] According to the invention described in the preceding items (9) and (21), a plurality of estimated region images are aligned when producing a composite image in such a manner that the evaluation value becomes the maximum among a plurality of combinations obtained by mutually offsetting the center coordinates of each image in at least one of the X-coordinate and Y-coordinate directions. This enables production of a composite image with higher accuracy and enables high-precision surface defect detection.

[0053] According to the invention described in the preceding items (10) and (22), provisional defect candidates are extracted using at least two processing routines, namely, a routine for extracting small defects and a processing routine for extracting gradual convex defects, and defect size correction is performed on provisional defect candidates detected in the routine for gradual convex defects, so provisional defect candidates can be reliably extracted.

[0054] According to the invention described in the above items (11) and (23), it is possible to identify a linear protrusion based on the amount of deviation of a two-dimensional defect signal in a phase image of a defect candidate from the roundness of a two-dimensional shape.

[0055] According to the invention described in the preceding item (24), it is possible to cause a computer to execute a process of synthesizing a plurality of images including defect candidates into one image and detecting defects based on the synthesized image. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a plan view showing a configuration example of a workpiece surface inspection system according to one embodiment of the present invention.

[0057] Figure 2 This is a vertical cross-section of the lighting frame when viewed from the front in the direction of travel of the workpiece.

[0058] Figure 3 This is a vertical cross-sectional view of the camera frame when viewed from the front in the direction of travel of the workpiece.

[0059] Figure 4 It is shown Figure 1 A top view of the electrical structure of a workpiece surface inspection system is shown.

[0060] Figure 5 1 is a flowchart showing the overall processing of the workpiece surface defect inspection system.

[0061] Figure 6 (A) is a diagram showing images continuously acquired from one camera in a time series, (B) is a diagram showing a state in which coordinates of a tentative defect candidate are estimated in a subsequent image of the initial image of (A), (C) is a diagram showing a process of overlapping each image of an estimated area image group to produce a composite image, and (D) is a diagram showing other processes of overlapping each image of an estimated area image group to produce a composite image.

[0062] Figure 7 This is a diagram for explaining a process of correcting the center coordinates of an estimated region image based on the position of a defect candidate from the boundary between a light band portion and a dark band portion in an image.

[0063] Figure 8 (A) to (D) are diagrams showing a process of creating a composite image by superimposing each image of the estimation region image group in different schemes.

[0064] Fig. 9 This is a diagram for explaining an example of a process of extracting provisional defect candidates.

[0065] Fig.10 It is a flowchart showing the content of the first surface defect detection process executed in the defect detection PC.

[0066] Fig.11 is used to explain in more detail Fig.10 Flowchart of the matching process of step S17.

[0067] Fig.12 It is used to illustrate Fig.10 Flowchart of a modified example of the matching process of step S17.

[0068] Fig.13 It is shown Fig.10 This is a flowchart showing details of steps S12 to S18 of the flowchart.

[0069] Fig.14 It is a flowchart showing the second surface defect detection process executed in the defect detection PC.

[0070] Fig.15 It is shown Fig.14 This is a flowchart of details of steps S12 to S18 of the flowchart.

[0071] Fig.16 It is a diagram for explaining the third surface defect detection process, and is a diagram showing a plurality of (two in this example) images acquired continuously in time series.

[0072] Fig.17 This is a graph showing an example of the relationship between the position of the workpiece (vehicle body) and the image plane displacement amount.

[0073] Fig.18 It is a flowchart showing the contents of the third surface defect detection process executed in the defect detection PC.

[0074] Fig.19 It is shown Fig.18 This is a flowchart of details of steps S32 to S40 of the flowchart.

[0075] Fig. 20 : is a flowchart showing the process of creating a standard deviation image.

[0076] Fig.21 This is a diagram showing a pattern of illuminance on a workpiece by a lighting device that performs light-dark pattern lighting.

[0077] Fig. 22 This is a flowchart showing another example of the process of creating a standard deviation image.

[0078] Fig.23 This is a flowchart showing another example of the process of creating a standard deviation image. DETAILED DESCRIPTION

[0079] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0080] Figure 1 It is a top view showing a structural example of a surface inspection system for a workpiece involved in one embodiment of the present invention. In this embodiment, the workpiece 1 is a vehicle body, the measured part of the workpiece 1 is the painted surface of the vehicle body, and the surface defects of the painted surface are detected. Generally, the surface of the vehicle body is subjected to base treatment, metal coating, transparent (clear) coating, etc. to form a coating layer with a multi-layer structure, but due to the influence of foreign matter during painting, etc., concave-convex defects are generated in the top transparent layer. In this embodiment, it is applied to the detection of such defects, but the workpiece 1 is not limited to the vehicle body, and can also be a workpiece other than the vehicle body. In addition, the measured part can also be a surface other than the painted surface.

[0081] The inspection system includes a workpiece moving mechanism 2 that continuously moves a workpiece 1 in the direction of arrow F at a predetermined speed. Two lighting frames 3, 3 are installed in the middle of the longitudinal direction of the workpiece moving mechanism 2, in front and behind the moving direction of the workpiece, with both lower ends in the direction orthogonal to the moving direction of the workpiece fixed to support tables 4, 4. In addition, each lighting frame 3, 3 is connected to each other by two connecting members 5, 5. The number of lighting frames is not limited to two.

[0082] Each lighting frame 3 is as follows when viewed from the front in the direction of travel of the vehicle body: Figure 2 As shown in the vertical cross-sectional view of , the lighting assembly 6 for illuminating the workpiece 1 is formed into a gate shape, and a lighting assembly 6 for illuminating the workpiece 1 is installed in each lighting frame 3. In this embodiment, the lighting assembly 6 has a linear lighting portion installed along the inner shape of the lighting frame 3 so as to surround the circumference of the workpiece 1 except the lower surface, and a plurality of linear lighting portions in the linear lighting portion are installed on the lighting frame 3 at equal intervals in the moving direction of the workpiece 1. Therefore, the lighting assembly 6 diffusely illuminates the workpiece with lighting light of a light and dark stripe pattern composed of lighting portions and non-lighting portions that alternately exist in the moving direction of the workpiece 1. The lighting assembly may also be a curved surface.

[0083] In the middle of the two front and rear lighting frames 3, 3, the camera frame 7 is installed in a state where the two lower ends in the direction perpendicular to the moving direction of the workpiece are fixed to the support tables 4, 4. In addition, the camera frame 7 is installed as shown when viewed from the front in the moving direction of the workpiece 1. Figure 3 As shown in the vertical cross-sectional view of , the camera frame 7 is formed into a gate shape, and a plurality of cameras 8 as imaging units are installed in the camera frame 7 along the inner shape thereof so as to surround the peripheral surface of the workpiece 1 except for the lower surface.

[0084] With such a structure, while the workpiece 1 is moved at a predetermined speed by the workpiece moving mechanism 2 and diffusely illuminated by the illumination light of the light and dark stripe pattern caused by the illumination unit 6, the various parts of the circumferential direction of the workpiece 1 are continuously photographed as the measured parts by the multiple cameras 8 mounted on the camera frame 7. The photographing is performed in a manner that most of the photographing ranges overlap in the previous and next photographing. As a result, multiple images in which the positions of the measured parts of the workpiece 1 are continuously shifted in the moving direction of the workpiece 1 are output from each camera 8.

[0085] Figure 4 It is shown Figure 1 A top view of the electrical structure of a workpiece surface inspection system is shown.

[0086] In the moving area of ​​the workpiece 1 , a first position sensor 11 , a vehicle type information detection sensor 12 , a second position sensor 13 , a vehicle speed sensor 14 , and a third position sensor 15 are provided in order from the entry side along the moving direction of the workpiece 1 .

[0087] The first position sensor 11 is a sensor for detecting that the next workpiece 1 approaches the inspection area. The vehicle information detection sensor 12 is a sensor for detecting the ID, vehicle type, color, destination information, etc. of the vehicle body to be inspected. The second position sensor 13 is a sensor for detecting that the workpiece 1 enters the inspection area. The vehicle speed sensor 14 is a sensor for detecting the moving speed of the workpiece 1 and monitoring the position of the workpiece 1 by calculation, but the position sensor can also be used to directly monitor the position of the workpiece. The third position sensor 15 is a sensor for detecting that the workpiece 1 exits the inspection area.

[0088] The workpiece surface defect inspection system further includes a master PC 21 , a defect detection PC 22 , a HUB 23 , a NAS (Network Attached Storage) 24 , a display 25 , and the like.

[0089] The main PC 21 is a personal computer that controls the entire surface defect inspection system of the workpiece as a whole, and has a processor such as a CPU, a memory such as a RAM, a storage device such as a hard disk, other hardware and software. The main PC 21 has a movement control unit 211, a lighting unit control unit 212, a camera control unit 213, etc. as one of the functions of the CPU.

[0090] The movement control unit 211 controls the movement stop and movement speed of the movement mechanism 2, the lighting unit control unit 212 controls the lighting of the lighting unit 6, and the camera control unit 213 controls the camera 8. The camera 8 continuously performs the image capture in response to the trigger signal continuously sent from the host PC 21 to the camera 8.

[0091] The defect detection PC 22 is a surface defect detection device that performs detection processing of surface defects, and is composed of a personal computer having a processor such as a CPU, a memory such as a RAM, a storage device such as a hard disk, other hardware and software. The defect detection PC 22 has an image acquisition unit 221, a provisional defect candidate extraction unit 222, a coordinate estimation unit 223, a defect candidate determination unit 224, an image group creation unit 225, an image synthesis unit 226, a defect detection unit 227, etc. as one of the functions of the CPU.

[0092] The image acquisition unit 221 acquires a plurality of images that are continuously photographed by the camera 8 in a time series and transmitted from the camera 8 by GigE (Gigabit Ethernet (registered trademark)). The tentative defect candidate extraction unit 222 extracts tentative defect candidates based on the plurality of images from the camera 8 acquired by the image acquisition unit 221, and the coordinate estimation unit 223 estimates the coordinates of the extracted tentative defect candidates in subsequent images. The defect candidate determination unit 224 matches the coordinates of the estimated tentative defect candidates with the actual tentative defect candidates to determine the defect candidates, and the image group creation unit 225 cuts out the area around the determined defect candidates to create an image group consisting of a plurality of images for synthesizing images. The image synthesis unit 226 synthesizes each image of the created image group into one image, and the defect detection unit 227 detects and distinguishes defects based on the synthesized image. The specific surface defect detection processing performed by these units in the defect detection PC 22 will be described later.

[0093] The NAS 24 is a storage device on the network and stores various data. The display 25 is a display that displays the surface defects detected by the defect detection PC 22 in a state corresponding to the position information of the vehicle body as the workpiece 1. The HUB 23 has a function of sending and receiving data between the host PC 21, the defect detection PC 22, the NAS 24, the display 25, etc.

[0094] Next, the defect detection process performed by the defect detection PC 22 will be described.

[0095] While the moving mechanism 2 is moving the workpiece 1 at a predetermined speed, the lighting unit 6 is illuminating the workpiece 1 from the surroundings with the illumination light of the light and dark stripe pattern, and the main PC 21 continuously sends a trigger signal to each camera 8, and each camera 8 continuously photographs the measured part of the workpiece 1. The main PC 21 sets the photographing interval, in other words, the interval of the trigger signal, in such a way that most of the photographing range overlaps in the previous and next photographing. Through such photographing, a plurality of images are obtained from each camera 8, in which the position of the measured part of the workpiece 1 is continuously shifted in the moving direction according to the movement of the workpiece 1.

[0096] Such multiple images can be obtained from the camera 8 not only when only the workpiece 1 moves relative to the fixed lighting unit 6 and camera 8 as in the present embodiment, but also when the workpiece 1 is fixed and the lighting unit 6 and camera 8 are moved relative to the workpiece 1, or when the workpiece 1 and camera 8 are fixed and the lighting unit 6 is moved. That is, the light and dark pattern of the lighting unit 6 can be moved relative to the workpiece 1 by moving at least one of the workpiece 1 and the lighting unit 6.

[0097] The plurality of images obtained by the cameras 8 are sent to the defect detection PC 22, and the image acquisition unit 221 of the defect detection PC 22 acquires the plurality of images sent from the cameras 8. The defect detection PC 22 executes a surface defect detection process using these images.

[0098] Figure 5 The flow chart shows the overall processing of the workpiece surface inspection system.

[0099] In step S01, the host PC 21 determines whether the workpiece 1 is close to the inspection range based on the signal from the first position sensor 11. If not (No in step S01), the process remains in step S01. If close (Yes in step S01), in step S02, the host PC 21 obtains individual information such as the ID, vehicle type, color, and destination information of the vehicle to be inspected based on the signal from the vehicle information detection sensor 12. In step S03, the host PC 21 performs, as initial information setting, setting of parameters of the inspection system, setting of the inspection range on the vehicle, etc.

[0100] In step S04, the main PC determines whether the workpiece 1 has entered the inspection range based on the signal of the second position sensor 13. If it has not entered (No in step S04), the process remains in step S04. If it has entered (Yes in step S04), in step S05, the camera 8 takes images of the moving workpiece 1 in a time series in a state where most of the imaging range overlaps. Next, in step S06, the front-end processing of the surface defect detection processing performed by the defect detection PC 22 is implemented. The front-end processing will be described later.

[0101] In step S07, it is determined whether the workpiece 1 has exited the inspection range based on the signal of the third position sensor 15. If it has not exited (No in step S07), the process returns to step S05 to continue the imaging and the front-end processing. When the workpiece 1 exits the inspection range (Yes in step S07), in step S08, the back-end processing in the surface defect detection processing performed by the defect detection PC 22 is implemented. That is, in this embodiment, the back-end processing is implemented after all the imaging of the workpiece 1 is completed. The back-end processing will be described later.

[0102] After the post-stage processing, in step S09 , the result of the surface defect detection processing is displayed on the display 25 or the like.

[0103] Next, the surface defect detection process including the front-stage process of step S06 and the back-stage process of step S08 performed by the defect detection PC 22 will be specifically described.

[0104] [1] Surface defect detection process

[0105] As described above, the defect detection PC 22 acquires a plurality of images in which the positions of the measured portions of the workpiece 1 are continuously shifted in the moving direction from each camera 8 . Figure 6 A11 to A17 of (A) of the figure are images continuously acquired in time series from a single camera 8. The light and dark pattern of light bands (white parts) extending in the vertical direction and dark bands (black parts) alternating in the horizontal direction shown in the image corresponds to the light and dark stripe pattern of the illumination light caused by the illumination unit 6.

[0106] The tentative defect candidate extraction unit 222 of the defect detection PC 22 extracts tentative defect candidates from each image. The tentative defect candidates are extracted by, for example, background removal, binarization, etc. In this example, it is assumed that the tentative defect candidates 30 are extracted from all images A11 to A17.

[0107] Next, the coordinate estimation unit 223 calculates the representative coordinates of the position of the tentative defect candidate 30 for each extracted image, and sets a predetermined area around the representative coordinates as the tentative defect candidate area. Furthermore, based on the movement amount of the workpiece 1, the coordinates to which the calculated representative coordinates of the tentative defect candidate move relative to each of the subsequent images A12 to A17 are calculated, and the estimated coordinates in each image are obtained. For example, the coordinates to which the tentative defect candidate 30 extracted in the image A11 moves relative to each of the subsequent images A12 to A17 are calculated, and the estimated coordinates in each image are obtained.

[0108] exist Figure 6 In each of the images B12 to B17 of (B), the estimated coordinates 40 of the tentative defect candidate 30 are estimated in the subsequent images A12 to A17 of the image A11. In addition, the images B12 to B17 are the same as the images without the tentative defect candidate 30 in the images A12 to A17. Figure 6 In (B), several images in the middle are omitted. In addition, the light and dark patterns that appear in the image are also omitted.

[0109] Next, the defect candidate determination unit 224 performs Figure 6 The subsequent images A12 to A17 of the image A11 shown in (A) and the estimated coordinates 40 of the provisional defect candidate 30 are obtained. Figure 6The images corresponding to each other in each image B12 to B17 of (B), such as image A12 and image B12, image A13 and image B13, ... image A17 and image B17, are matched with each other. In the matching, it is determined whether the estimated coordinates 40 correspond to the actual provisional defect candidate 30 in the image. Specifically, it is performed by determining whether the estimated coordinates 40 are included in the predetermined provisional defect candidate area regarding the actual provisional defect candidate 30 in the image. In addition, it is also possible to determine whether there is a provisional defect candidate 30 within a predetermined range pre-set from the estimated coordinates 40, or whether there is an estimated coordinate 40 of the corresponding image within a predetermined range pre-set from the representative coordinates of the provisional defect candidate 30, thereby determining whether the estimated coordinates 40 correspond to the actual provisional defect candidate 30 in the image. In the case where the estimated coordinates 40 correspond to the provisional defect candidate 30 in the image, it can be considered that the provisional defect candidate 30 included in the original image A11 and the provisional defect candidate 30 included in the subsequent image are the same.

[0110] Next, as a result of the matching, the number of images corresponding to (matching) the estimated coordinates 40 and the actual tentative defect candidate 30 in the image is checked to determine whether the number is above a preset threshold. Then, if it is above the threshold, the probability that the tentative defect candidate 30 actually exists is high, so the tentative defect candidate 30 of each image is determined as a defect candidate. Figure 6 In the examples of (A) and (B), all subsequent images A12 to A17 of image A11 are matched. That is, the estimated coordinates 40 are included in the tentative defect candidate area of ​​the tentative defect candidate 30 in the image. When the number of images corresponding to the estimated coordinates 40 and the actual tentative defect candidate 30 is not greater than a preset threshold, it is considered that the possibility of the tentative defect candidate 30 being a defect candidate is not high, so the matching is terminated and the next tentative defect candidate 30 is extracted.

[0111] Next, the image group creation unit 225 generates all images including defect candidates, such as Figure 6 In each image A11 to A17 of (A), a predetermined area around the representative coordinates of the defect candidate is cut out as an inference area, as shown in (C) of the figure, and an inference area image group consisting of multiple inference area images C11 to C17 is produced. In addition, it is also possible to include not all images of the defect candidate but multiple images, but when the number of images is large, the amount of information increases, which is preferred in terms of being able to perform high-precision surface inspection. In addition, the inference area can also be performed by first finding the inference area of ​​the original image A11 and calculating the position of the inference area in each image according to the movement amount of the workpiece 1.

[0112] The image synthesis unit 226 synthesizes the estimated region images C11 to C17 of the estimated region image group thus prepared by superimposing them to prepare Figure 6 The composite image 51 shown in (C) is superimposed at the center coordinates of each estimated region image C11 to C17. As an example of the composite image 51, at least one of an image composited by calculating a statistical deviation value such as a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image can be cited. An image composited by calculating a statistical deviation value such as a standard deviation image will be described later.

[0113] Next, the defect detection unit 227 detects surface defects using the synthesized image 51. The detection criteria for surface defects can also be freely selected. For example, Figure 6 As shown in the signal graph 61 of (C), by determining that a defect exists when the signal is above the reference value, only the presence or absence of a defect is detected. Alternatively, the type of defect can be determined by comparison with a reference defect, etc. In addition, the criteria for determining the presence or absence of a defect and the type of defect can be changed by machine learning, etc., or a new criteria can be created.

[0114] The detection result of the surface defect is displayed on the display 25. It is preferable that a development view of the workpiece (vehicle body) 1 is displayed on the display 25, and the position and type of the surface defect are displayed in an understandable manner on the development view.

[0115] Thus, in this embodiment, a plurality of estimated region images C11 to C17 cut out from a plurality of images A11 to A17 including defect candidates are synthesized into one synthesized image 51, and defect detection is performed based on the synthesized image 51, so that information of a plurality of images is included in the synthesized image 51. Therefore, defect detection can be performed using a large amount of information for one defect candidate, so even small surface defects can be stably detected with high accuracy while suppressing over-detection and erroneous detection.

[0116] In addition, when the difference between the estimated coordinates 40 and the image corresponding to the actual provisional defect candidate 30 in the image is above a preset threshold, a composite image is created and defect detection is performed. Therefore, defect detection can be performed when the possibility of a defect is high, the processing load is small, the detection efficiency is improved, and the detection accuracy is also improved.

[0117] Furthermore, there is no need to perform a plurality of different transformation processes on the fused image.

[0118] [1-1] Modification Example 1 when Creating a Composite Image

[0119] However, accuracy may not be achieved simply by superimposing and synthesizing the plurality of estimated region images C11 to C17 at the center coordinates of the respective images.

[0120] Therefore, it is preferable to correct the center coordinates of the estimated region images C11 to C17 and then overlap them. One example of correcting the center coordinates is to perform the correction based on the relative position in the light-dark pattern of each image. Specifically, when there is a defect in the center of the light band portion or the dark band portion of the light-dark pattern, it is easy to form a bilaterally symmetrical shape. Figure 7 For example, as shown in the estimated image C14, when the defect candidate 30 approaches the boundary with the dark band 110 in the light band 120, the boundary side becomes darker. On the contrary, when the defect candidate 30 approaches the boundary in the dark band 110, the boundary side becomes brighter. Therefore, when performing the center position calculation, for example, the center position 30a of the defect candidate 30 is deviated. The deviated position is related to the position from the boundary, so the center coordinates of the image are corrected according to the position L from the boundary.

[0121] Figure 6 (D) is a diagram showing a scene in which each estimated region image C11 to C17 with the center position corrected is superimposed and synthesized at the center position to create a synthesized image 52. Compared with the synthesized image 51 and the signal graph 61 in (C) of the same figure, a clearer synthesized image 52 is obtained, and the signal height in the signal graph 62 is also increased. Therefore, a highly accurate synthesized image 52 can be created, and highly accurate surface defect detection can be performed.

[0122] [1-2] Modification Example 2 when Creating a Composite Image

[0123] according to Figure 8 Another method of creating a composite image will be described in the case where accuracy cannot be obtained simply by superimposing and composite a plurality of estimation region images C11 to C17 at the center coordinates of the respective images.

[0124] Until Figure 6 The preparation of the estimated region images C11 to C17 of (C) is the same. In this example, the alignment of the estimated region images C11 to C17 is attempted by obtaining multiple combinations by shifting the center coordinates of each image in at least one of the left-right direction (x direction) and the up-down direction (y direction) by various alignment amounts. Then, the combination with the largest evaluation value is adopted. Figure 8 In , the four types (A) to (D) are overlapped. The resulting composite images are represented by 53 to 56, and the signal graphs based on the composite images are represented by 63 to 66. Figure 8 In the example, (B) which obtains the highest signal is adopted.

[0125] In this way, the multiple estimated area images C11 to C17 are aligned when creating a synthetic image in a manner so that the evaluation value becomes the largest among the multiple combinations obtained by shifting the center coordinates of each image in at least one of the X-coordinate and Y-coordinate directions, so that a more accurate synthetic image can be created and high-precision surface defect detection can be performed.

[0126] [1-3] An example of tentative defect candidate extraction processing

[0127] An example of extraction processing of a tentative defect candidate having a large size and a slow change in curvature by the tentative defect candidate extraction unit 222 will be described.

[0128] First, we will return to the principle of the present method using illumination with a light and dark stripe pattern for explanation.

[0129] The illumination light is reflected from the surface of the workpiece 1 and enters each pixel of the camera 8. In other words, the light entering each pixel is the light from the area that is reflected from the surface of the workpiece 1 by the line of sight emitted from each pixel within the visible range of each pixel and reaches. If there is no illumination, a dark pixel signal is obtained, and if there is illumination, a bright pixel signal is obtained. If there is no defect and the workpiece 1 is a plane, the area on the illumination corresponding to each pixel is close to a point. In the case of a defect, there are two types of changes in the surface of the workpiece 1: (1) change in curvature and (2) tilt of the surface.

[0130] (1) Fig. 9 As shown in (A), when the curvature of the surface of the workpiece 1 changes due to the provisional defect candidate 30, the direction of the line of sight changes, but further, the area visible to each pixel expands. As a result, the area corresponding to each pixel is not a point but an expanded area, and the average brightness within the area corresponds to the pixel signal. That is, when the shape of the provisional defect candidate 30 changes dramatically, the curvature change becomes larger in the area visible to each pixel, and in addition to the tilt of the line of sight, the expansion of the area cannot be ignored. The expansion of the visible area becomes the averaging of the illumination distribution of the signal. In the case of light and dark stripe pattern illumination (in Fig. 9 In the case of a white area being bright and a black area being dark, when the area is expanded, the average value of the light and dark areas corresponding to the expansion method is obtained. When the light and dark stripe pattern of the part where this phenomenon occurs moves in sequence, its influence can be captured in the standard deviation image.

[0131] (2) Fig. 9As shown in (B), when the curvature radius of the surface of the workpiece 1 is large and remains roughly flat and tilted due to the temporary defect candidate 30, the corresponding area remains as a point, but faces a direction different from the non-tilted surface. When the temporary defect candidate 30 is large (the shape changes slowly), the visible area of ​​each pixel is the same, the change in the line of sight direction becomes significant, and the curvature change slows down. Its changes cannot be captured in the standard deviation image. When the defect is large, the difference in the inclination of the surface of the non-defective part and the defective part can be detected in the phase image. If it is a non-defective part, the phase in the direction parallel to the stripes in the phase image is the same, and a certain phase change occurs in the direction perpendicular to the stripes according to the period of the stripes. If it is a defective part, the regularity of the above phase is disturbed in the phase image. For example, by observing the phase images in the X direction and the Y direction, it is possible to detect a temporary defect candidate with a slower surface change.

[0132] It is also possible to extract arbitrary provisional defect candidates using two routines, one for smaller provisional defect candidates and one for larger provisional defect candidates. Candidates extracted using an arbitrary routine may be set as provisional defect candidates.

[0133] However, consider the case where it is desired to report the size of the detected tentative defect candidate 30 as a result. The correlation between the visual defect size and the defect size detected from the image is associated with the approximate circle of the portion where the surface of the defect surface is inclined at a predetermined angle. A certain linear relationship is observed when the defect size is small, but a defect with a large defect size and a gentle surface inclination is a nonlinear relationship. Therefore, the gentle tentative defect candidate 30 detected in the phase image needs to be corrected based on a calibration curve obtained separately because the defect signal and the defect size are not in a linear relationship.

[0134] [1-4] Detection of linear protrusion defects

[0135] As an example of defect detection performed by the defect detection unit 227 , a detection process of a linear protrusion will be described.

[0136] Linear protrusions are defects caused by linear foreign matter trapped under the coating. They are not round but elongated. There are examples where the line width is small (e.g. less than 0.2 mm) but the length is long (e.g. more than 5 mm). The width direction is very narrow, so it is small, and the length direction has a gentle curvature change. Sometimes it is missed by only using the same detection method for small defects and large defects (defects with a gentle inclination) as the extraction of temporary defect candidates. After the predetermined processing, binarization and granulation are performed, and whether it is a defect is determined based on the area of ​​each part.

[0137] The linear protrusions are narrow in width but long in length, so if properly detected, the predetermined area can be obtained. However, the linear protrusions are easy to detect when the length direction is parallel to the direction in which the light and dark pattern extends, but difficult to observe when it is perpendicular. If a defective portion is generated in the length direction and it is shorter than the actual length, that is, the granular area is likely to be smaller.

[0138] Therefore, when the defect has a certain extension in the longitudinal direction based on the shape information of the defect obtained from the phase image, that is, when the roundness is lower than a predetermined value, the threshold value for area determination is reduced to suppress failure to detect the linear protrusion.

[0139] [1-5] Flowchart

[0140] Fig.10 2 is a flowchart showing the contents of the surface defect detection process executed by the defect detection PC 22. The surface defect detection process is shown in more detail. Figure 5 The surface defect detection process is executed by the processor in the defect detection PC 22 operating in accordance with the operating program stored in the built-in storage device such as the hard disk device.

[0141] In step S11, the host PC 21 obtains the host PC 21 in Figure 5 The initial information includes the individual information obtained in step S02, the setting of the parameters set in step S03, the setting of the inspection range on the vehicle body, etc.

[0142] Next, in step S12 , an image captured by the camera 8 is acquired, and thereafter, in step S13 , pre-processing is performed, such as setting of position information for the image based on initial setting information.

[0143] Next, in step S14 , the tentative defect candidate 30 is extracted from each image, and then in step S15 , the movement amount of the workpiece 1 is calculated for one tentative defect candidate 30 , and in step S16 , the coordinates of the tentative defect candidate 30 in the subsequent image are estimated and set as the estimated coordinates 40 .

[0144] In step S17, matching is performed. That is, it is determined whether the estimated coordinates 40 exist in the predetermined tentative defect candidate area with respect to the actual tentative defect candidate 30 in the image, and when the number of images that have been matched is greater than a preset threshold, in step S18, the tentative defect candidate 30 of each image is determined as a defect candidate.

[0145] In step S19, for all images with defect candidates, a predetermined area around the representative coordinates of the defect candidate is cut out as an estimated area, and an estimated area image group consisting of a plurality of estimated area images C11 to C17 is created, and then the process proceeds to step S20. Steps S12 to S19 are pre-processing.

[0146] In step S20, based on the information from the main PC 21, it is determined whether the vehicle body as the workpiece 1 has exited the inspection range. If it has not exited the inspection range (No in step S20), it returns to step S12 and continues to obtain images from the camera 8. If the vehicle body has exited the inspection range (Yes in step S20), in step S21, the alignment amount of each estimated area image C11 to C17 is set. Then, in step S22, each estimated area image C11 to C17 is synthesized to produce a synthetic image, and then defect detection processing is performed in step S23. Steps S21 to S23 are post-processing. After defect detection, the detection results are output to the display 25, etc. in step S24.

[0147] Reference Fig.11 The flowchart of step S17 is described in detail.

[0148] In step S201, K, a variable representing the number of images matching the tentative defect candidate 30, is set to zero. In step S202, N, a variable representing the number of images to be judged as matching the tentative defect candidate 30, is set to zero.

[0149] After the tentative defect candidate 30 is extracted in step S203, N is set to N+1 in step S204. Next, in step S205, it is determined whether the tentative defect candidate 30 is consistent with the estimated coordinates 40. If they are consistent (yes in step S205), K is set to K+1 in step S206, and then the process proceeds to step S207. In step S205, if the tentative defect candidate 30 is inconsistent with the estimated coordinates 40 (no in step S205), the process proceeds to step S207.

[0150] In step S207, it is checked whether N has reached the predetermined number of images (in this case, 7 images). If not (in step S207, it is No), the process returns to step S203 to extract the tentative defect candidate 30 for the next image. In the case where N has reached the predetermined number of images (in step S207, it is Yes), in step S208, it is determined whether K is greater than a predetermined threshold value (in this case, 5 images) that is set in advance. If it is not greater than the threshold value (in step S208, it is No), the process returns to step S201. Therefore, in this case, the subsequent cutting out process of the estimated region image and the image synthesis process are not performed, N and K are reset, and the next tentative defect candidate 30 is extracted.

[0151] If K is above the threshold value (Yes in step S208), in step S209, the tentative defect candidate 30 is determined as a defect candidate and the information is saved, and then, in step S210, the estimated region image is cut out for the K images that have been matched. Then, in step S211, the cut K estimated region images are synthesized, and then, in step S212, it is determined whether a surface defect is detected. When a surface defect is detected (Yes in step S212), in step S213, it is determined to be a surface defect and the information is saved, and then the process proceeds to step S214. In the case where no surface defect is detected (No in step S212), the process proceeds to step S214 as it is.

[0152] In step S214, it is checked whether the detection process has been performed on all the inspection target parts of the workpiece. If it has not been performed (No in step S214), the process returns to step S201, N and K are reset, and the next tentative defect candidate 30 is extracted. If the detection process has been performed on all the inspection target parts (Yes in step S214), the process ends.

[0153] Thus, in this embodiment, when the number K of images corresponding (matching) to the tentative defect candidate 30 and the estimated coordinates 40 is not greater than the threshold, the number of matched images is small and the possibility that the tentative defect candidate 30 is a defect candidate is not high, so the subsequent processing is terminated. If the number of matched images is greater than K, the possibility that the tentative defect candidate 30 is a defect candidate is high, so the estimated region image is cut out, the image is synthesized, and the defect detection is performed. Therefore, compared with the case where the estimated region image is cut out, the image is synthesized, and the defect detection is performed regardless of the number of matched images, the processing load is small, the detection efficiency is improved, and the detection accuracy is also improved.

[0154] Fig.12 It is used to illustrate Fig.10Flowchart of a variation of the matching process of step S17. In this example, the following structure is adopted: when the number of images K matched does not reach a certain value before the number of images N reaches a predetermined number, it is determined that the possibility that the provisional defect candidate 30 is a defect candidate is not high, and the subsequent processing is terminated at this point in time.

[0155] In step S221, K, which is a variable representing the number of images matching the tentative defect candidate 30, is set to zero. In step S222, N, which is a variable representing the number of images to be judged as to whether or not the images match the tentative defect candidate 30, is set to zero.

[0156] After the tentative defect candidate 30 is extracted in step S223, N is set to N+1 in step S224. Next, in step S225, it is determined whether the tentative defect candidate 30 is consistent with the estimated coordinates 40. If they are consistent (yes in step S225), K is set to K+1 in step S226, and then the process proceeds to step S227. In step S225, if the tentative defect candidate 30 is inconsistent with the estimated coordinates 40 (no in step S225), the process proceeds to step S227.

[0157] In step S227, it is checked whether N has reached the second predetermined number of images (here, 8 images). If so (yes in step S227), in step S228, it is checked whether K has reached the second preset threshold value (here, 4 images). If not (no in step S228), the process returns to step S221. Therefore, in this case, the subsequent cutting out process of the estimated region image and the image synthesis process are not performed, N and K are reset, and the next extraction of the tentative defect candidate 30 is performed.

[0158] In step S228, if K reaches the second threshold value (Yes in step S228), the process proceeds to step S229. In step S227, if N does not reach the second predetermined number of images (8) (No in step S227), the process proceeds to step S229.

[0159] In step S229, it is checked whether N has reached the first predetermined number of sheets (here, 9 sheets). If not (No in step S229), the process returns to step S223 to extract the tentative defect candidate 30 for the next image. When N has reached the first predetermined number of sheets (Yes in step S229), it is determined in step S230 whether K is greater than the first threshold value (here, 5 sheets) set in advance. If it is not greater than the first threshold value (No in step S230), the process returns to step S201. Therefore, in this case, the subsequent cutting out process and image synthesis process of the estimated region image are not performed, N and K are reset, and the next tentative defect candidate 30 is extracted.

[0160] If K is greater than or equal to the first threshold value (yes in step S230), in step S231, the tentative defect candidate 30 is determined as a defect candidate and the information is saved, and then, in step S232, the estimated region image is cut out for the K images that have been matched. Then, in step S233, the cut K estimated region images are synthesized, and then, in step S234, it is determined whether a surface defect is detected. When a surface defect is detected (yes in step S234), in step S235, it is determined to be a surface defect and the information is saved, and then the process proceeds to step S236. When a surface defect is not detected (no in step S234), the process proceeds to step S236 as it is.

[0161] In step S236, it is checked whether the detection process has been performed on all the inspection target parts of the workpiece. If not (No in step S236), the process returns to step S201, N and K are reset, and the next tentative defect candidate 30 is extracted. If the detection process has been performed on all the inspection target parts (Yes in step S236), the process ends.

[0162] Thus, in this embodiment, in addition to playing the role of Fig.11 In addition to the same effects as those of the embodiment shown in the flowchart of , the following effects are also achieved. That is, in the stage where the number N of images from which the tentative defect candidates 30 are extracted is the first set value less than the second set value, that is, in the midway stage, if the number K of images corresponding (matching) between the tentative defect candidates 30 and the estimated coordinates 40 does not reach the first threshold value less than the second threshold value, it is determined that there are few matching images and the possibility that the tentative defect candidate 30 is a defect candidate is not high, and the matching process is not continued until the final image, and the subsequent processing is terminated. Therefore, useless processing is not continued, so the processing load can be further reduced and the detection accuracy can be further improved.

[0163] Fig.13 This is a diagram showing the front-end processing in the surface defect detection process. Fig.10 The flowchart of steps S12 to S18 of the flowchart of FIG. Fig.10 The same processing as in the flowchart is assigned the same step number.

[0164] The camera 8 continuously captures images of a workpiece 1 while the workpiece 1 is moving until it exits the inspection range, and the defect detection PC 22 acquires images from the first capture to the last capture in step S12. Here, it is assumed that the images capturing a tentative defect candidate 30 are from the image captured at the nth capture to the image captured at the (n+m-1)th capture.

[0165] After preprocessing each image in step S13, in step S14, for each image from the nth to the (n+m-1)th time of shooting, the tentative defect candidate 30 is extracted, and the representative coordinates and tentative defect candidate area of ​​the extracted tentative defect candidate 30 are obtained. Next, in step S16, the representative coordinates of the tentative defect candidate are calculated based on the calculation of the movement amount of the workpiece 1 in step S15, etc., to which coordinates the representative coordinates of the tentative defect candidate move relative to each image in the subsequent images, and the estimated coordinates 40 in each image are obtained.

[0166] In step S17, matching is performed for each subsequent image, and when the number of matched images obtained is greater than a threshold value (e.g., m), in step S18, the provisional defect candidate 30 is determined as a defect candidate. In step S19, an estimated region is calculated for each image, and an estimated region image group consisting of a plurality of estimated region images C11 to C17 is created.

[0167] [2] Second surface defect detection process

[0168] In the above-described first surface defect detection process, it is assumed that the defect detection PC 22 extracts the tentative defect candidates 30 from the images continuously acquired by the camera 8 in time series.

[0169] The method of extracting the provisional defect candidates 30 is not limited, but a configuration in which the provisional defect candidates 30 are extracted by performing the following processing is preferable in that defective parts can be emphasized and provisional defect candidates 30 can be extracted with higher accuracy.

[0170] That is, for each of the images A11 to A17 ( Figure 6 After binarization processing is performed (as shown), the feature points of the image are extracted by applying a threshold or by applying an angle detection function. In addition, the tentative defect candidate 30 can also be extracted by obtaining a multi-dimensional feature quantity for each extracted feature point.

[0171] More preferably, before extracting the feature points, after binarizing each image obtained from the camera 8 and extracting the contour, an image that has been expanded and contracted a predetermined number of times is subtracted to create an orange peel mask that removes the boundary between the light band and the dark band. It is preferable to extract the feature points of each image after masking the boundary between the light band and the dark band using the created mask, thereby enabling the tentative defect candidate to be extracted with higher accuracy.

[0172] Alternatively, the tentative defect candidates 30 may be extracted by extracting feature points of an image and then obtaining multi-dimensional feature quantities for all pixels in a specific range around each feature point based on brightness gradient information in all directions of the pixel.

[0173] After the tentative defect candidates 30 are extracted, similarly to the above-mentioned first surface defect detection process, an estimated region image group consisting of a plurality of estimated region images C11 to C17 is created, and then defect detection is performed for each tentative defect candidate using the estimated region image group.

[0174] In this way, in the second surface defect detection process, feature points of the images obtained from the camera 8 and showing the continuously offset positions of the measured parts of the workpiece 1 are extracted, and a multi-dimensional feature quantity is calculated for each extracted feature point to extract the provisional defect candidate 30. Therefore, the provisional defect candidate 30 can be extracted with high precision, and high-precision surface defect detection can be performed.

[0175] Furthermore, the coordinates of the extracted tentative defect candidate 30 are obtained, and for each of the subsequent multiple images of the image from which the tentative defect candidate 30 is extracted, the coordinates to which the coordinates of the tentative defect candidate 30 are moved are calculated to obtain the estimated coordinates 40, and it is determined whether the estimated coordinates 40 correspond to the tentative defect candidate 30 in the image, and if the estimated coordinates 40 in the subsequent image and the image corresponding to the tentative defect candidate are greater than a preset threshold, the tentative defect candidate 30 is determined as a defect candidate. Then, for each determined defect candidate, a predetermined area around the defect candidate is cut out from the multiple images including the defect candidate as an estimated area, and an estimated area image group consisting of multiple estimated area images C11 to C17 is created, and defect discrimination is performed based on the created estimated area image group.

[0176] That is, since multiple estimated region images C11 to C17 including defect candidates include multiple information about one defect candidate, defect detection can be performed using more information. Therefore, even small surface defects can be stably detected with high accuracy while suppressing over-detection and false detection.

[0177] [2-1] Flowchart

[0178] Fig.14 2 is a flowchart showing the second surface defect detection process executed in the defect detection PC. Fig.10 Steps S11 to S13 and steps S15 to S20 are the same, so the same step numbers are added and the descriptions are omitted.

[0179] After the pre-processing in step S13 , an orange peel mask is created in step S141 , and in step S142 , the created orange peel mask is used to extract feature points.

[0180] Next, in step S143 , a multi-dimensional feature quantity is calculated for each extracted feature point, and in step S144 , a provisional defect candidate 30 is extracted, and then the process proceeds to step S16 .

[0181] When the vehicle body as the workpiece 1 exits the inspection range in step S20 (YES in step S20 ), defect identification processing is executed in step S23 using the generated estimated region image group, and the identification result is displayed in step S24 .

[0182] Fig.15 It is shown Fig.14 The flowchart of steps S12 to S18 of the flowchart of FIG. Fig.14 The same processing as in the flowchart is assigned the same step number. In addition, steps S12, S13, S15 to S19 are the same as Fig.13 The processing of steps S12, S13, and S15 to S19 is the same, so the description is omitted.

[0183] After the pre-processing of step S13, in step S141, orange peel masks are created for each image. In step S142, the created orange peel masks are applied to each image to extract feature points of each image.

[0184] In step S143 , a multi-dimensional feature quantity is calculated for each feature point of each extracted image, and in step S144 , a tentative defect candidate is extracted for each image, and then the process proceeds to step S16 .

[0185] [3] Surface defect detection process

[0186] In the first surface defect detection process, after the provisional defect candidates 30 are extracted from each of the images A11 to A17, the defect candidates are determined, the estimated regions around the defect candidates are calculated, and the plurality of estimated region images C11 to C17 are synthesized to perform defect detection.

[0187] In contrast, in the third surface defect detection process, defects are detected after the plurality of images of the continuous time series acquired from the camera 8 are divided into a plurality of regions and the plurality of images before and after are synthesized with each other in the corresponding regions. However, since the workpiece 1 moves, the imaging range of the workpiece 1 represented in the region of the front image and the imaging range of the workpiece 1 represented in the region of the rear image are different, and the imaging position is different according to the movement amount of the workpiece 1, so the region of the rear image is synthesized with the position offset relative to the region of the front image by the position offset amount corresponding to the movement amount of the workpiece 1. In addition, the position offset amount of the corresponding region of the front image and the rear image is different according to the position of the divided region, so the position offset amount corresponding to the movement amount of the workpiece 1 is set for each divided region.

[0188] Hereinafter, it will be described in detail that the plurality of images continuously captured by the camera 8 and continuously acquired in time series by the defect detection PC 22 are the same as the images acquired in the first surface defect detection process.

[0189] Fig.16 A plurality of images A21 and A22 acquired continuously in time series are shown. In this example, two images are shown, but the number of images is actually greater. In addition, the light and dark patterns that appear in the images are omitted in the images A21 and A22. These images A21 and A22 are taken in a direction orthogonal to the moving direction of the workpiece (in Fig.16 The image A21 is divided into a plurality of regions 1 to p in the vertical direction (in the vertical direction). The regions 1 to p are at the same position (same coordinates) and have the same size in the images A21 and A22.

[0190] Since the workpiece moves, the imaging range corresponding to the image in each area 1 to p in the image A21 obtained from the camera 8 is shifted in the moving direction relative to the original area 1 to p as indicated by the arrows in the subsequent image A22 by the amount of movement of the workpiece 1. Therefore, by shifting the position of each area 1 to p in the image A22 by the position shift amount S corresponding to the amount of movement of the workpiece, each area 1 to p in the image A21 and each area 1 to p after the position shift in the image A22 become the same imaging range on the workpiece 1. Such a relationship occurs between each area 1 to p in the previous and next imaging images, so by sequentially shifting the area 1 to p in each subsequent image by the position shift amount S, the imaging range of each of the original image A21 and each of the areas 1 to p in each subsequent image can be made consistent.

[0191] However, as in Fig.16 As schematically shown in the image A22 of FIG. 8 , the offset amount relative to the original area 1 to p is different for each of the areas 1 to p. For example, when there are straight portions and curved portions of the workpiece 1 within the imaging range of one camera 8, the position offset amount of the area corresponding to the straight portion and the area corresponding to the curved portion in the image are different. In addition, the position offset amount is different depending on the distance relative to the camera 8. Therefore, even if all areas 1 to p are offset by a uniform position offset amount, they may not necessarily become the same imaging range depending on the area.

[0192] Therefore, in this embodiment, the position offset S is calculated and set for each of the areas 1 to p. Specifically, the average magnification information in the area is obtained for each of the areas 1 to p based on the camera information, the camera position information, the three-dimensional shape of the workpiece, and the position information of the workpiece. Then, the position offset S is calculated for each of the areas 1 to p based on the obtained magnification information and the approximate moving speed assumed in advance, and is set as the position offset S for each area 1 to p.

[0193] Here, the calculation of the positional deviation amount is supplemented. Consider a case where a plurality of images of a moving workpiece 1 are taken at equal time intervals. The focus is on the movement of the same point between two consecutive captured images.

[0194] The amount of movement on the image is related to the camera magnification and the speed of the workpiece. The camera magnification depends on (1) the focal length of the lens and (2) the distance from the camera to each part of the workpiece to be photographed. Regarding (2), the part close to the camera moves more than the part far from the camera on the image. When the 3D shape of the workpiece 1 and the installation position of the camera 8 and the position and posture of the workpiece 1 are known, it is possible to calculate where the focus point in the image captured at a certain moment is.

[0195] When the workpiece 1 moves and changes position, it is possible to calculate how many pixels the same focus point has moved in two consecutive images. For example, if a sensor with a focal length of 35 mm and a pixel size of 5.5 μm is used and the workpiece moves 1.7 mm between adjacent images, Fig.17 As shown in the graph, when the distance to the workpiece 1 (Zw) is 600 mm to 1100 mm, the moving distance in the screen moves 18 pixels to 10 pixels.

[0196] If it is assumed that the alignment error required for composite image production is suppressed to ±1 pixel, the distance difference can be set to ±5 cm. The image is divided into regions so that the distance difference from the camera is within ±5 cm. For each divided region, the average position offset between consecutive images is calculated based on the approximate moving speed of the workpiece 1. Three types of position offsets can be set for each of the regions 1 to p, including the position offset and the offset of ±1 pixel. However, the position offset is not limited to three types, and the distance difference is not limited to ±5 cm.

[0197] In addition, the position offset S of each of the set areas 1 to p is stored in association with the areas 1 to p in a table in the storage unit within the defect detection PC 22, and is set by retrieving the position offset from the table for the imaging parts that can set the same position offset, such as the same shape parts of the workpiece 1, or the same type of workpieces.

[0198] Next, a predetermined number of images are synthesized for each of the plurality of regions 1 to p while the positions of the regions 1 to p are shifted by the set position shift amount S. With regard to the synthesis, the images of the regions are overlapped while the positions of the regions 1 to p are shifted by the set position shift amount S, and calculations are performed for each pixel of the corresponding coordinates in the overlapped images to create a synthesized image for each pixel. As an example of a synthesized image, at least one of an image synthesized by calculating a statistical deviation value such as a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image can be cited.

[0199] Next, after performing pre-processing such as background removal and binarization on the composite image, such as the standard deviation image, and extracting defect candidates, surface defects are detected using different operations and composite images from those used to extract defect candidates. The detection criteria for surface defects can be freely selected, and it is also possible to only determine the presence or absence of defects, or to determine the type of defects based on comparison with reference defects, etc. In addition, the criteria for determining the presence or absence of defects and the type of defects can be set based on the characteristics of the workpiece and the defects, and can be changed through machine learning, etc., or a new benchmark can be created.

[0200] The detection result of the surface defect is displayed on the display 25. It is preferable that a development view of the workpiece (vehicle body) 1 is displayed on the display 25, and the position and type of the surface defect are displayed in an understandable manner on the development view.

[0201] Thus, in this embodiment, a plurality of camera images A21 and A22 continuously acquired from the camera in a time series are divided into a plurality of regions 1 to p, and a plurality of images are synthesized for each of the divided regions 1 to p, and defect detection is performed based on the synthesized image, so that information of a plurality of images is included in the synthesized image. Therefore, defect detection can be performed using a large amount of information for one defect candidate, so even a small surface defect can be stably detected with high accuracy while suppressing over-detection and erroneous detection.

[0202] Furthermore, by synthesizing the images of the corresponding regions in a state where the regions 1 to p of the subsequent image A22 are offset in sequence with respect to the regions 1 to p of the previous image A21 by the position offset amount S set according to the movement amount of the workpiece 1, the region of the previous image and the corresponding region of the subsequent image become the same imaging range of the workpiece 1, and a plurality of images can be synthesized in a state where the imaging range of the workpiece 1 is consistent. In addition, since the position offset amount is set for each of the divided regions 1 to p, the error of the imaging range can be suppressed to a minimum compared to the case where a uniform position offset amount is applied to all the divided regions 1 to p. Therefore, it is possible to detect surface defects with higher accuracy.

[0203] [2-1] Modification 1 regarding position shift

[0204] In the above example, the position offset S corresponding to each divided area 1-p is calculated for each area 1-p based on the magnification information of each area 1-p and the approximate moving speed assumed in advance, but the position offset S can also be set based on the result of setting multiple position offsets for each area 1-p.

[0205] For example, for each area 1 to p, positional offset candidates are set under a plurality of conditions, including the assumed moving speed, from slow speed to fast speed. Then, synthetic images are created using each positional offset candidate, and defect detection is performed as needed. Based on the comparison between them, the positional offset S with the highest evaluation is adopted.

[0206] In this way, a plurality of position offset candidates are set for each region 1 to p under different conditions, and based on a comparison when synthesizing an image with each position offset candidate, the position offset candidate with the highest evaluation is used as the position offset S for the region 1 to p. Therefore, an appropriate position offset S can be set for each of the regions 1 to p, enabling higher-precision surface defect detection.

[0207] [2-2] Modification 2 regarding position shift

[0208] The setting of the position shift amount S of each of the regions 1 to p can also be performed as follows. Fig.17 As shown in the figure, if the moving distance of the workpiece 1 between adjacent images is known, the positional deviation amount on the image can be calculated. In the above example, the positional deviation amount is set according to the pre-assumed workpiece moving speed.

[0209] The appropriate positional offset amount for each frame during composite image creation may also be determined based on the workpiece position actually measured. In this case, time and effort for selecting the optimal positional offset amount from a plurality of positional offset amounts can be saved.

[0210] The method for measuring the position of the workpiece is described as follows. A plurality of position-specific cameras arranged in the moving direction of the workpiece 1 are used to capture an image of the same portion of the workpiece 1 or a support member that performs the same movement as the workpiece 1, and the position information of the workpiece is obtained from the image. First, when there is a characteristic hole in the workpiece 1, the hole or a mark provided on a pedestal that moves while holding the workpiece 1 is used as a target for measuring the position or speed of the workpiece 1.

[0211] In order to detect the target, multiple cameras independent of the camera 8 are prepared. For example, they are arranged in a row in the direction of travel of the workpiece 1 in such a way that the side of the workpiece 1 is observed from the side of the workpiece 1. They are arranged so as to cover the entire length of the workpiece 1 when multiple horizontal fields of view are connected. The magnification can be calculated based on the distance from the camera to the workpiece 1 and the focal length of the camera. Based on the magnification, the actual position is calculated based on the position on the image. The positional relationship between each camera is known, and the position of the workpiece 1 is calculated based on the image information of each camera.

[0212] By matching the workpiece position information from the above-mentioned multiple cameras, an appropriate position offset is obtained from the image of the defect extraction camera 8. For example, for each area virtually divided on the workpiece 1 so that the distance difference on the workpiece observed from the camera becomes ±5 cm, the average movement amount on the image between adjacent images corresponding to the movement amount of the workpiece 1 is determined as the position offset amount when overlapping, and a composite image is produced.

[0213] [2-3] Modification 3 regarding position shift

[0214] In variant 2, the position of the workpiece is obtained by arranging a plurality of cameras. Alternatively, a measuring system including a distance sensor, a speed sensor, a vibration sensor, or a combination thereof may be used to measure the same part of the workpiece 1 or a support member that performs the same movement as the workpiece 1 to obtain the workpiece position information.

[0215] The method for measuring the workpiece position is described below. A portion of the workpiece 1 or the same portion of a support member that performs the same movement as the workpiece 1 is used as the target. In detecting the workpiece position, a "sensor for detecting the workpiece position passing through a reference point + a distance sensor" or a "sensor for detecting the workpiece position passing through a reference point + a speed sensor + the time interval between adjacent images" is used. The former directly obtains the workpiece position. The latter can obtain the workpiece position when each image is captured by multiplying the speed information from the speed sensor by the imaging interval.

[0216] By matching the above workpiece position information, an appropriate position offset is obtained from the image of the defect extraction camera 8. For example, for each area virtually divided on the workpiece 1 so that the distance difference on the workpiece observed from the camera becomes ±5 cm, the average movement amount on the image between adjacent images corresponding to the movement amount of the workpiece 1 is determined as the position offset amount when overlapping, and a composite image is produced.

[0217] [2-4] Flowchart

[0218] The overall processing of the workpiece surface inspection system is based on Figure 5 Implementation as shown in the flowchart.

[0219] Fig.182 is a flowchart showing the contents of the third surface defect detection process executed by the defect detection PC 22. The surface defect detection process is shown in more detail. Figure 5 The surface defect detection process is executed by a processor in the defect detection PC 22 in accordance with an operation program stored in a built-in storage device such as a hard disk device.

[0220] In step S31, the host PC 21 obtains the host PC 21 in Figure 5 The initial information includes the individual information obtained in step S02, the setting of the parameters set in step S03, the setting of the inspection range on the vehicle body, etc.

[0221] Next, in step S32, images A21 and A22 captured by the camera 8 are obtained, and then, in step S33, each image A21 and A22 is divided into a plurality of regions 1 to p. On the other hand, based on the position and moving speed of the workpiece 1 (step S34), a plurality of positional offset amount candidates are set for each of the divided regions 1 to p in step S35.

[0222] Next, in step S36, for one region, a plurality of images whose positions are shifted by a plurality of position offset amount candidates are synthesized to create a plurality of synthesized image candidates for each region. Thereafter, in step S37, based on a comparison of the synthesized images of each of the created position offset amount candidates, the position offset amount candidate with the highest evaluation is used as the position offset amount for the regions 1 to p, and the plurality of images are synthesized again for each region according to the position offset amount to create a synthesized image.

[0223] In step S38, background removal, binarization and other pre-processing are performed on the synthesized image, and then defect candidates are extracted in step S39. By performing such processing on each of the plurality of regions 1 to p and on each of the predetermined number of images, a large number of defect candidate image groups from which defect candidates are extracted are produced in step S40, and then the process proceeds to step S41. Steps S32 to S40 are pre-processing.

[0224] In step S41, it is determined whether the vehicle body has exited the inspection range based on the information from the host PC 21. If it has not exited the inspection range (No in step S41), the process returns to step S32 and continues to acquire images from the camera 8. If the vehicle body has exited the inspection range (Yes in step S41), defect detection processing is performed on the defect candidate image group in step S42. Step S42 is a post-processing. After defect detection, the detection result is output to the display 25, etc. in step S43.

[0225] Fig.19This is a diagram showing the front-end processing in the surface defect detection process. Fig.18 Detailed flowchart of steps S32 to S40 of the flowchart of Fig.18 The same processing as in the flowchart is assigned the same step number.

[0226] The camera 8 continuously captures images of a workpiece 1 as it moves from the time the workpiece 1 enters the inspection range until the workpiece 1 exits the inspection range, and the defect detection PC 22 acquires images from the first capture to the last capture in step S32. Here, the case of using images from the nth capture to the (n+m-1)th capture is exemplified.

[0227] In step S33, each image is divided into p image regions, for example, regions 1 to p. In step S35, q positional offset amount candidates are set for each of the p regions, and in step S36, q synthetic image candidates are created for each of the p image regions using the q positional offset amount candidates. That is, q synthetic images are created for each of the regions 1 to p.

[0228] In step S37 - 1 , a synthetic image having the highest evaluation value is selected for each of the regions 1 to p, and a positional shift amount candidate corresponding to the selected synthetic image is determined as the positional shift amount for the image region.

[0229] Then, in step S37 - 2 , the determined positional shift amount is applied to each of the regions 1 to p to create a composite image.

[0230] The subsequent pre-processing (step S38), defect candidate extraction processing (step S39), defect candidate image group creation processing (step S40) and Fig.18 The same, so the description is omitted.

[0231] [4] Production of standard deviation images, etc.

[0232] In the first surface defect detection process and the third surface defect detection process, multiple images of the synthetic object are prepared based on multiple repeated images of the imaging range captured by the camera 8 in a time series when the workpiece is moved under the state of irradiating a light and dark lighting pattern, and these multiple images are synthesized into one image as a synthetic image. As one of the synthetic images, an image synthesized by calculating a statistical deviation value such as a standard deviation image is considered.

[0233] The statistical variation value includes at least one of the variance, the standard deviation, and the half-value width. Although any one of them can be calculated, the case where the standard deviation is calculated and synthesized is described here.

[0234] The standard deviation is calculated for each corresponding pixel of multiple images. Fig.17 is a flowchart showing the process of creating a standard deviation image. Fig. 20 The processing shown in the flowchart thereafter is executed by the defect detection CPU according to the operation program stored in the storage unit or the like.

[0235] In step S51, source images (N images) to be synthesized are generated, and in step S52, the sum of squares of brightness values ​​(hereinafter also referred to as pixel values) is calculated for each pixel of the first source image, and then the sum of pixel values ​​is calculated for each pixel in step S53. In the first image, both the sum of squares and the sum calculation are the results of the first image only.

[0236] Next, in step S54, it is checked whether there is a next image. If there is (yes in step S54), the process returns to step S52, squares the pixel values ​​of each pixel of the second image and adds them to the square values ​​of the corresponding pixel values ​​of the first image. Next, in step S53, the pixel values ​​of the second image are added to the corresponding pixel values ​​of the first image.

[0237] Such processing is sequentially performed on N images, and the sum of squares of the pixel values ​​and the sum of the pixel values ​​are calculated for each corresponding pixel of the N images.

[0238] After the above processing for N images is completed (No in step S54), in step S55, the average of the sums of the pixel values ​​calculated in step S53 is calculated, and then in step S56, the square of the average of the sums is calculated.

[0239] Next, in step S57, the average value of the sum of squares of the pixel values ​​calculated in step S52, i.e., the square mean, is calculated, and then, in step S57, the variance is calculated according to the formula {(square mean) - (square of the mean)}. Then, in step S59, the standard deviation, which is the square root of the variance, is calculated.

[0240] It is preferable to normalize the standard deviation thus obtained and to create a synthetic image based on the result. In addition, when the variance or the half-value width is used as a statistical deviation value, it can be calculated in the same manner.

[0241] The surface defect detection process is performed based on the created synthetic image. The detection process may be performed in the same manner as the first surface defect detection process and the third surface defect detection process.

[0242] In this way, by calculating the statistical deviation value to synthesize the corresponding pixels of multiple images and applying it to all pixels to create a synthetic image, even if the number of images to be synthesized is small, a synthetic image with a high S / N ratio for defect detection can be created. By using this synthetic image, high-precision defect detection can be performed, the detection of unnecessary defect candidates can be reduced, and the detection of necessary defects can be prevented from being missed. In addition, compared with the case of creating a synthetic image using the maximum value, minimum value, etc., the cost is also reduced.

[0243] [4-1] Other embodiments related to standard deviation image 1

[0244] Fig.18 FIG. 6 is a graph showing the illumination intensity of the workpiece 1 by the lighting assembly 6 that performs light and dark pattern illumination. Fig.21 In the graph of FIG. 7 , the top 71 of the waveform represents a light band and the bottom 72 represents a dark band.

[0245] The rising and falling edge portions 73 of the waveform from the light band to the dark band or from the dark band to the light band are not actually vertical but inclined. The pixel value in the image portion corresponding to each midpoint of the rising and falling edge portions 73 becomes an intermediate grayscale, which affects the deviation.

[0246] When multiple images are taken during one cycle of the lighting pattern, for example, Fig.21 When the image is taken 8 times as indicated by the black circle in (A), it is highly likely that 2 pixels out of 8 pixels in each of the 8 images obtained will have a pixel value of the intermediate grayscale corresponding to the intermediate point. Fig.21 When the timed imaging indicated by the black circle in (B) is performed seven times, there is a high possibility that at least one pixel among the seven pixels in each of the seven obtained images will have a pixel value of an intermediate grayscale corresponding to the intermediate point.

[0247] As described above, such pixel values ​​of intermediate grayscales affect the deviation, resulting in a decrease in the accuracy of defect detection. Therefore, it is preferable to exclude such pixel values ​​of intermediate grayscales from the sampling candidates for deviation calculation and only calculate the deviation for the selected best sampling candidate. Specifically, when the number of source images that are the objects of synthesis in one cycle of the lighting pattern is an even number, the deviation is calculated by removing two pixel values ​​of intermediate grayscales from the pixel values ​​of multiple pixels, and when the number of source images is an odd number, the deviation is calculated by removing one pixel value of intermediate grayscales from the pixel values ​​of multiple pixels. In this way, the pixel values ​​of intermediate grayscales are excluded from the sampling candidates for deviation calculation, and only the deviation is calculated for the selected best sampling candidate, so that only the best sampling candidate is used to calculate the statistical deviation value, which can suppress the influence of the pixels excluded from the sampling candidates, so that even when the number of synthesized images is small, a synthetic image that can perform high-precision defect detection can be produced.

[0248] Fig. 22 This is a flowchart showing a process of creating a standard deviation image by excluding pixel values ​​of intermediate grayscale from sampling candidates for deviation calculation and calculating deviation only for selected optimal sampling candidates.

[0249] In step S61, multiple (N) source images are generated, and then in step S62, the sampling data of the pixel values ​​of N images are sorted in each pixel of each image, and 1 value (N is an odd number) or 2 values ​​(N is an even number) of the median are removed.

[0250] Next, in step S63, for each pixel, the standard deviation is calculated using N-1 (N is an odd number) or N-2 (N is an even number) values.

[0251] It is preferable to normalize the standard deviation thus obtained and to create a synthetic image based on the result. In addition, when the variance or the half-value width is used as a statistical deviation value, it can be calculated in the same manner.

[0252] [4-2] Other Implementation Methods 2 Related to Standard Deviation Image

[0253] In this embodiment, imaging is performed a plurality of times (N times) for one cycle of the illumination pattern. N times may be a small number.

[0254] In this embodiment, similar to the case of other embodiment 1 related to the standard deviation image, when the source image of the synthetic object in one cycle of the illumination pattern is an odd number, the standard deviation is calculated for each pixel using N-1 sampling data (pixel values), and when it is an even number, the standard deviation is calculated using N-2 sampling data. That is, in the case of an odd number, the standard deviation is calculated for each pixel using N-1 combinations (NCN-1) selected from N pixel values, and in the case of an even number, the standard deviation is calculated for each pixel using N-2 combinations (NCN-2) selected from N pixel values. Then, from the standard deviations such as (NCN-1) or (NCN-2) obtained for each pixel, the largest standard deviation is determined as the standard deviation for the pixel (maximum value processing).

[0255] Fig.23 The flowchart of FIG. Fig.23 In the processing of , the case where the number N of source images to be synthesized is an odd number is shown, but the same is true for the case of an even number.

[0256] In step S71, source images (N images) to be synthesized are generated, and in step S72, the square sum of pixel values ​​is calculated for each pixel of the first source image, and then the sum of pixel values ​​is calculated for each pixel in step S73. In the first image, both the square sum and the sum calculation are the results of the first image only. In step S74, the square value of each pixel value of the first image is stored, and in step S75, each pixel value (original value) of the first image is stored.

[0257] Next, in step S76, it is checked whether there is a next image. If there is (yes in step S76), the process returns to step S72, squares the pixel value of each pixel of the second image and adds it to the square value of each corresponding pixel value of the first image. Next, in step S73, each pixel value of the second image is added to each corresponding pixel value of the first image. Furthermore, in step S74, the square value of each pixel value of the second image is stored, and in step S75, each pixel value (original value) of the second image is stored.

[0258] Such processing is performed sequentially for N images, and the square sum of each pixel value and the sum of each pixel value are calculated for each corresponding pixel of the N images. In addition, the square value and pixel value (original value) of each image value of the N images are stored respectively.

[0259] After the above processing for N images is completed (No in step S76), in step S77, i is used as a variable, and the square value of each pixel in the first image (i=1) is first subtracted from the sum of the squares of the pixel values ​​of each pixel in all N images calculated in step S72, and the sum of the squares of N-1 images is calculated for each pixel.

[0260] Next, in step S78, the sum of N-1 images is calculated by subtracting the pixel values ​​of the first image from the sum of the pixel values ​​of all images calculated in step S73. In step S79, the average of the sums of N-1 images calculated in step S78 is calculated, and then, in step S80, the square of the average of the sums is calculated.

[0261] Next, in step S81, the square average, which is the average of the sum of squares of the N-1 sheets calculated in step S77, is calculated, and then in step S82, the variance is calculated using the formula {(square average) - (square of the average)}. Then, in step S83, the standard deviation, which is the square root of the variance, is calculated.

[0262] Next, in step S84, maximization processing is performed. Here, since only one standard deviation is currently available for each pixel, this value becomes the maximum.

[0263] Next, in step S85, it is checked whether there is a next image to be subtracted, in other words, whether i=N. If there is a next image, that is, i=N is not (yes in step S85), the process returns to step S77, i=2 is set, the square sum of each pixel value and the pixel value of the second image are subtracted, the standard deviation is calculated in the same way, and maximization processing is performed in step S85. In the maximization processing, the standard deviation when the first image is subtracted and the standard deviation when the second image is subtracted are compared, and the larger standard deviation is adopted.

[0264] In this way, the sum of squares and pixel values ​​of each image from the 1st to the Nth image are sequentially subtracted until i=N is reached, and the standard deviation is calculated for each pixel, and the largest standard deviation is used as the standard deviation of the pixel.

[0265] It is preferable to normalize the standard deviation thus obtained and to create a synthetic image based on the result. In addition, when the variance or the half-value width is used as a statistical deviation value, it can be calculated in the same manner.

[0266] In this embodiment, a predetermined number of images are excluded from the calculation target from a plurality of images in sequence, and the statistical deviation value of each pixel is calculated, so that the best sampling candidate can be easily selected. In addition, the maximum value of the calculated deviation value is used as the deviation value for the pixel, so that a composite image with a higher S / N ratio can be produced.

[0267] In this embodiment, a case is described in which a plurality of images are acquired in one cycle of a light-dark pattern by the illumination unit 6 while the workpiece 1 is moved relative to the illumination unit 6 and the camera at a predetermined speed.

[0268] However, it is also possible to obtain a plurality of images in one cycle of the illumination pattern by moving only the illumination unit 6 relative to the workpiece 1 and the camera 8 , and to create a composite image by calculating deviations such as standard deviations from these plurality of images.

[0269] It should be understood that the terms and expressions used in this specification are used for explanation only and are not used for limiting interpretation. Any equivalents of the features shown and stated herein are not excluded. Various modifications are allowed within the scope of protection claimed by the present invention.

[0270] Illustrated embodiments of the present invention are described herein, but the present invention is not limited to the embodiments described herein, but also includes any embodiments having equivalent elements, corrections, eliminations, combinations (such as combinations of features across various embodiments), improvements and / or changes that can be understood by a person skilled in the art based on this disclosure.

[0271] Industrial Applicability

[0272] The present invention can be used to detect surface defects of workpieces such as vehicle bodies.

[0273] Explanation of symbols

[0274] 1 Workpiece

[0275] 2 Mobile mechanism

[0276] 3 Lighting framework

[0277] 4 Support table

[0278] 6 Lighting components

[0279] 7 Camera Frames

[0280] 8. Camera

[0281] 21 Main PC

[0282] 22 Defect Detection PC

[0283] 30 Tentative defect candidate

[0284] 40 Estimated coordinates

[0285] 221 Image acquisition unit

[0286] 222 Provisional defect candidate extraction department

[0287] 223 Coordinate Estimation Department

[0288] 224 Defect Candidate Decision Department

[0289] 225 Image Group Production Department

[0290] 226 Image Synthesis Department

[0291] 227 Defect Detection Department

Claims

1. A device for detecting surface defects of a workpiece, comprising: An image acquisition unit acquires a plurality of images of a measured portion of a workpiece as a surface defect detection target while a light-dark pattern caused by the lighting device is relatively moved relative to the workpiece; a tentative defect candidate extraction unit, for extracting tentative defect candidates for each image acquired by the image acquisition unit; a defect candidate determining unit that determines the tentative defect candidate as a defect candidate if the number of images including the tentative defect candidate among the plurality of images from which the tentative defect candidate is extracted by the tentative defect candidate extracting unit is greater than a preset threshold; an image group producing unit which, for each defect candidate determined by the defect candidate determining unit, cuts out a predetermined area around the defect candidate from a plurality of images including the defect candidate as an estimated area, and produces a plurality of estimated area image groups; an image synthesis unit that synthesizes the plurality of estimated region images produced by the image group production unit to produce a synthesized image for each defect candidate determined by the defect candidate determination unit; as well as a detection unit that performs defect detection based on the synthesized image produced by the image synthesis unit, The image synthesis unit performs alignment of a plurality of estimated region images when creating a synthesized image so that an evaluation value becomes maximum among a plurality of combinations in which the center coordinates of the images are shifted from each other in the X-coordinate and Y-coordinate directions.

2. The workpiece surface defect detection device according to claim 1, wherein: When the threshold is set to the first threshold, the defect candidate determination unit terminates the process of determining the tentative defect candidate as a defect candidate when the number of images including the tentative defect candidate is less than the first threshold and does not reach the predetermined second threshold in the middle of the process of determining the tentative defect candidate as a defect candidate.

3. The surface defect detection device for a workpiece according to claim 1, wherein: The defect candidate determination unit determines the tentative defect candidate as a defect candidate if the number of images corresponding to the position of the tentative defect candidate among the plurality of images is greater than a preset threshold.

4. The workpiece surface defect detection device according to claim 3, wherein: The position of the tentative defect candidate is represented by the coordinates of the tentative defect candidate. The surface defect detection device for the workpiece includes a coordinate estimation unit, which obtains the coordinates of the provisional defect candidate extracted by the provisional defect candidate extraction unit, and calculates the coordinates to which the coordinates of the provisional defect candidate are to be moved for each of the subsequent multiple images of the image from which the provisional defect candidate is extracted to obtain the estimated coordinates. The defect candidate determination unit determines whether the estimated coordinates calculated by the coordinate estimation unit in each subsequent image correspond to the temporary defect candidate in the image. If the number of images corresponding to the estimated coordinates and the temporary defect candidate in the subsequent image is above a predetermined threshold, the temporary defect candidate is determined as a defect candidate.

5. The workpiece surface defect detection device according to any one of claims 1 to 4, wherein: The synthetic image is at least any one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

6. The workpiece surface defect detection device according to claim 1, wherein: The image synthesis unit performs positional alignment of a plurality of estimated region images when creating a synthesized image by superimposing center coordinates of the images.

7. The workpiece surface defect detection device according to claim 6, wherein: The image synthesis unit corrects the center coordinates of each image based on the position of each center coordinate from the boundary of the light-dark pattern of the image corresponding to the light-dark pattern of the lighting device so as to overlap the corrected center coordinates.

8. The workpiece surface defect detection device according to any one of claims 1 to 4, wherein: The tentative defect candidate extraction unit extracts tentative defect candidates using at least two processing routines of extracting small defects and extracting gradual convex defects, and corrects defect sizes of tentative defect candidates detected in the gradual convex defect routine.

9. The workpiece surface defect detection device according to any one of claims 1 to 4, wherein: The detection unit determines the linear protrusion based on the amount of deviation of the two-dimensional defect signal in the phase image of the defect candidate from the roundness of the two-dimensional shape.

10. A workpiece surface inspection system comprising: an illumination device for irradiating a workpiece as a surface defect detection object with illumination light of a light and dark pattern; an imaging unit for imaging reflected light of the illumination light of the illumination device from a measured portion of the workpiece; A moving unit that moves the light-dark pattern caused by the lighting device relative to the workpiece; a control unit that controls the imaging unit so that the measured portion of the workpiece is imaged while the light and dark pattern of the lighting device is relatively moved relative to the workpiece by the moving unit; as well as The workpiece surface defect detection device according to any one of claims 1 to 9, The image acquisition unit acquires a plurality of images of a measurement target portion of the workpiece from the imaging unit.

11. A method for detecting surface defects of a workpiece, wherein: The surface defect detection device of the workpiece performs: An image acquisition step of acquiring a plurality of images of a measured portion of the workpiece while a light-dark pattern caused by the lighting device is relatively moved relative to the workpiece as a detection target of surface defects; A tentative defect candidate extraction step of extracting tentative defect candidates for each image acquired by the image acquisition step; a defect candidate determination step of determining the tentative defect candidate as a defect candidate if the number of images including the tentative defect candidate among the plurality of images from which the tentative defect candidate is extracted by the tentative defect candidate extraction step is greater than a preset threshold value; an image group preparation step, for each defect candidate determined by the defect candidate determination step, cutting out a predetermined area around the defect candidate from a plurality of images including the defect candidate as an estimation area, and preparing a plurality of estimation area image groups; an image synthesis step of synthesizing a plurality of estimated region images produced in the image group production step to produce a synthesized image for each defect candidate determined in the defect candidate determination step; and a detection step of performing defect detection based on the synthesized image produced by the image synthesis step, In the image synthesis step, the plurality of estimated region images are aligned when creating the synthesized image so that the evaluation value becomes maximum among a plurality of combinations in which the center coordinates of the images are shifted from each other in the X-coordinate and Y-coordinate directions.

12. The method for detecting surface defects of a workpiece according to claim 11, wherein: When the threshold is set to the first threshold, the defect candidate determination unit terminates the process of determining the tentative defect candidate as a defect candidate when the number of images including the tentative defect candidate is less than the first threshold and does not reach the predetermined second threshold in the middle of the process of determining the tentative defect candidate as a defect candidate.

13. The method for detecting surface defects of a workpiece according to claim 11, wherein: In the defect candidate determination step, if the number of images corresponding to the position of the tentative defect candidate among the plurality of images is greater than a preset threshold, the tentative defect candidate is determined as a defect candidate.

14. The method for detecting surface defects of a workpiece according to claim 13, wherein: The position of the tentative defect candidate is represented by the coordinates of the tentative defect candidate. The surface defect detection method of the workpiece includes a coordinate estimation step, in which the coordinates of the tentative defect candidate extracted by the tentative defect candidate extraction step are obtained, and for each of the subsequent images of the image from which the tentative defect candidate is extracted, the coordinates to which the coordinates of the tentative defect candidate are to be moved are calculated to obtain the estimated coordinates, In the defect candidate determination step, it is determined whether the estimated coordinates calculated by the coordinate estimation step in each subsequent image correspond to the temporary defect candidate in the image. If the number of images corresponding to the estimated coordinates and the temporary defect candidate in the subsequent image is above a predetermined threshold, the temporary defect candidate is determined as a defect candidate.

15. The method for detecting surface defects of a workpiece according to any one of claims 11 to 14, wherein: The synthetic image is at least any one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

16. The method for detecting surface defects of a workpiece according to claim 11, wherein: In the image synthesis step, the plurality of estimated region images are aligned when creating a synthesized image by superimposing the center coordinates of the images.

17. The method for detecting surface defects of a workpiece according to claim 16, wherein: In the image synthesis step, the center coordinates of each image are corrected based on the position of each center coordinate from the boundary of the light and dark pattern of the image corresponding to the light and dark pattern of the lighting device so that the corrected center coordinates overlap.

18. The method for detecting surface defects of a workpiece according to any one of claims 11 to 14, wherein: In the tentative defect candidate extraction step, tentative defect candidates are extracted using at least two processing routines, namely, a routine for extracting small defects and a routine for extracting gradual convex defects, and defect sizes of tentative defect candidates detected in the routine for extracting gradual convex defects are corrected.

19. The method for detecting surface defects of a workpiece according to any one of claims 11 to 14, wherein: In the detection step, the linear protrusion is determined based on the amount of deviation of the two-dimensional defect signal in the phase image of the defect candidate from the roundness of the two-dimensional shape. 20 . A computer-readable recording medium storing a program for causing a computer to execute the workpiece surface defect detection method according to claim 11 .

Images