Work surface defect detection device and detection method, work surface inspection system and program

Abstract

To provide a surface defect detection apparatus, or the like, for a workpiece, which can stably detect a small surface defect with high accuracy.SOLUTION: A surface defect detection apparatus acquires a plurality of images of a workpiece having a measurement portion captured at positions sequentially displaced in a moving direction, from imaging means which sequentially images the measurement portion of the workpiece so that the previous and next imaging ranges overlap mostly, divides the images into a plurality of regions, and sets an amount of displacement according to a moving amount of the workpiece for each of the regions. The apparatus synthesizes images of corresponding regions while displacing a corresponding region of the next image with respect to a region of the previous image with the set amount of displacement, and executes the image synthesis for each region to detect a defect on the basis of synthesized images.SELECTED DRAWING: Figure 18

Description

[Technical field]

[0001] The present invention relates to a workpiece surface defect detection device and detection method, a workpiece surface inspection system, and a program for detecting surface defects based on multiple images obtained by continuously imaging the measured portion, such as the painted surface of the workpiece, while the workpiece, such as a car body, for which surface defects are to be detected, is moved relative to a lighting device and a light and dark pattern of illumination light is irradiated onto the measured portion of the workpiece, such as the painted surface of the workpiece, and the measured portion is continuously imaged by an imaging means. [Background technology]

[0002] As described above, Patent Document 1 is known as a technology for detecting surface defects of a workpiece by continuously capturing images of a measurement site of the workpiece with an imaging means while moving the workpiece such as a car body relative to an illumination device. However, the technology described in Patent Document 1 does not disclose a specific defect detection algorithm using images obtained by the imaging means.

[0003] Moreover, Patent Document 2 describes a method of fusing a plurality of images into a single image to detect surface defects.

[0004] Furthermore, Patent Document 3 proposes a defect inspection method for an inspected surface that enables highly accurate defect detection by processing a plurality of images captured in a time series and detecting defects based on the movement state of defect candidate points extracted from each image. Specifically, the inspected surface 1 is moved so that its imaged portion A moves with the passage of time, an image from the imaging means 2 is input at any time, a defect candidate point B is extracted from each of the two input images captured at different times, an error range is set around the defect candidate point B in the image captured later, the defect candidate point B in the image captured earlier is moved according to the movement distance of the inspected surface or the imaging means, among the defect candidate points present within the error range, the defect candidate point closest to the defect candidate point B after this movement is set as a match candidate, and the above process is performed on a plurality of images captured in a time series, and the defect candidate point B that is raised as a match candidate a predetermined number of times or more is determined to be a defect present on the inspected surface 1. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Patent No. 7,639,349 [Patent Document 2] US Patent Application Publication No. 2017 / 0277979 [Patent Document 3] Patent No. 3156602 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the inspection method described in Patent Document 2 has a problem in that the defect detection process is complicated, for example, because it is necessary to perform first to third different conversion processes on the fused image.

[0007] Furthermore, the inspection method described in Patent Document 3 has a problem in that the amount of information is insufficient because the processing is basically based on a single image, and small surface defects cannot be identified with sufficient accuracy.

[0008] This invention has been made to solve such problems, and its object is to provide a work surface defect detection device and detection method, a work surface inspection system, and a program that are capable of stably detecting small surface defects with high accuracy without requiring multiple complex conversion processes, etc. [Means for solving the problem]

[0009] The above object can be achieved by the following means: (1) An image acquisition means for acquiring a plurality of images of a measurement portion of a workpiece, which is a target for detecting surface defects, while a light and dark pattern generated by an illumination device is moved relative to the workpiece; a division means for dividing each image acquired by the image acquisition means into a plurality of regions; and Taking into account the three-dimensional shape of the workpiece in the area a position shift amount setting means for setting a position shift amount according to the amount of movement of the workpiece; an image synthesis means for synthesizing images of corresponding areas in a state in which corresponding areas of a subsequent image are shifted in sequence relative to the area of ​​a previous image by the position shift amount set for each area by the position shift amount setting means, and performing this synthesis for each area; and a detection means for detecting defects based on the synthesized image for each area created by the image synthesis means. (2) the light-dark pattern generated by the lighting device is moved relative to the workpiece as the target for surface defect detection, and an image acquisition means acquires a plurality of images of a measurement site of the workpiece; a division means divides each image acquired by the image acquisition means into a plurality of regions; a position shift amount setting means sets a position shift amount corresponding to the amount of movement of the workpiece for each region divided by the division means; an image synthesis means synthesizes images of corresponding regions with corresponding regions of a subsequent image being shifted in sequence relative to a region of a previous image by the position shift amount set for each region by the position shift amount setting means, and performs this synthesis for each region; and a detection means performs defect detection based on the synthesized image for each region created by the image synthesis means, The position shift amount setting means sets a plurality of different position shift amount candidates for each of the divided regions, and when the image synthesis means applies each of the set position shift amount candidates to synthesize the image, the position shift amount candidate with the maximum evaluation value is set as the position shift amount for that region. Characterized by A device for detecting surface defects on workpieces. (3) The position shift amount setting means captures images of the workpiece or the same portion of a support member that moves in the same manner as the workpiece using a plurality of position-specific cameras arranged in the direction of movement of the workpiece, obtains position information of the workpiece from the images, and ,mosquitoA surface defect detection device for a workpiece as described in the preceding paragraph 1, wherein an average positional shift amount for each area is calculated from the positional information of the camera and the workpiece and the optical information of the camera, and this is used as the positional shift amount for each area. (4) The position shift amount setting means measures the same portion of the workpiece or a support member that moves in the same manner as the workpiece with a measurement system including a distance sensor, a speed sensor, and a vibration sensor, either alone or in combination, to obtain position information of the workpiece, and ,mosquito A surface defect detection device for a workpiece as described in the preceding paragraph 1, wherein an average positional shift amount for each area is calculated from the positional information of the camera and the workpiece and the optical information of the camera, and this is used as the positional shift amount for each area. (5) The workpiece surface defect detection device according to any one of items 1 to 4, wherein the composite image is at least one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image. (6) A workpiece surface defect detection device according to any one of the preceding paragraphs 1 to 5, wherein the detection means performs defect discrimination based on discrimination criteria set according to characteristics of the workpiece and the defect. (7) A work surface inspection system comprising: an illumination device which irradiates illumination light of a light and dark pattern onto a workpiece which is to be detected for surface defects; an imaging means which images the reflected light of the illumination light from the illumination device from a measured portion of the workpiece; a moving means which moves the light and dark pattern of the illumination device relative to the workpiece; a control means which controls the imaging means so as to image the measured portion of the workpiece while moving the light and dark pattern of the illumination device relative to the workpiece by the moving means; and a work surface defect detection device as described in any of paragraphs 1 to 6 above, wherein the image acquisition means acquires a plurality of images of the measured portion of the workpiece from the imaging means. (8) an image acquiring step of acquiring a plurality of images of a measurement portion of a workpiece, which is a target for detecting surface defects, while a light-dark pattern generated by an illumination device is moved relatively to the workpiece; a division step of dividing each image acquired by the image acquiring step into a plurality of regions; and Taking into account the three-dimensional shape of the workpiece in the areaA method for detecting surface defects of a workpiece, characterized in that a workpiece surface defect detection device executes the following steps: a position shift amount setting step for setting a position shift amount corresponding to the amount of movement of the workpiece; an image synthesis step for synthesizing images of corresponding areas with respect to the area of ​​a previous image, with the corresponding area of ​​a subsequent image being shifted in sequence by the position shift amount set for each area by the position shift amount setting step, and performing this synthesis for each area; and a detection step for detecting defects based on the synthesized image created by the image synthesis step. (9) a work surface defect detection device that executes the following steps: an image acquisition step of acquiring a plurality of images of a measurement portion of a workpiece while a light and dark pattern generated by an illumination device is moved relative to the workpiece as a surface defect detection target; a division step of dividing each image acquired by the image acquisition step into a plurality of regions; a position shift amount setting step of setting a position shift amount according to the amount of movement of the workpiece for each region divided by the division step; an image synthesis step of synthesizing images of corresponding regions with corresponding regions of a subsequent image being shifted in sequence relative to a region of a previous image by the position shift amount set for each region by the position shift amount setting step, and performing this synthesis for each region; and a detection step of detecting defects based on the synthesized image created by the image synthesis step; In the position shift amount setting step, a plurality of different position shift amount candidates are set for each of the divided plurality of regions, and in the image synthesis step, a position shift amount candidate with a maximum evaluation value is set as the position shift amount for that region based on a comparison when the set position shift amount candidates are applied and the image is synthesized. Characterized by A method for detecting surface defects on a workpiece. (10) In the position shift amount setting step, the same portion of the workpiece or a support member that moves in the same manner as the workpiece is imaged by a plurality of position-specific cameras arranged in the moving direction of the workpiece, and the position information of the workpiece is obtained from the images. ,mosquito 9. A method for detecting surface defects of a workpiece as described in paragraph 8 above, in which an average positional shift amount for each area is calculated from the positional information of the camera and the workpiece and the optical information of the camera, and this is used as the positional shift amount for each area. (11) In the position shift amount setting step, the same portion of the workpiece or a support member that moves in the same manner as the workpiece is measured by a measurement system including a distance sensor, a speed sensor, and a vibration sensor, either alone or in combination, to obtain position information of the workpiece, and ,mosquito 9. A method for detecting surface defects of a workpiece as described in paragraph 8 above, in which an average positional shift amount for each area is calculated from the positional information of the camera and the workpiece and the optical information of the camera, and this is used as the positional shift amount for each area. (12) The method for detecting surface defects of a workpiece according to any one of items 8 to 11 above, wherein the composite image is at least one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image. (13) A method for detecting surface defects of a workpiece according to any one of claims 8 to 12, wherein in the detection step, defect discrimination is performed based on discrimination criteria set according to characteristics of the workpiece and the defect. (14) A program for causing a computer to execute the method for detecting surface defects of a workpiece according to any one of items 8 to 13 above. Effect of the Invention

[0010] According to the invention described in the preceding paragraphs (1), (7) and (8), a light and dark pattern generated by an illumination device is moved relative to a workpiece for which surface defects are to be detected, and multiple images of a measured portion of the workpiece are acquired, the multiple acquired images are divided into multiple regions, and multiple images for each divided region are synthesized, and defect detection is performed based on this synthesized image, so that the synthesized image contains information on multiple images. Therefore, defect judgment can be performed using a large amount of information about one defect candidate, and even if the surface defect is small, it is possible to perform stable detection with high accuracy while suppressing overdetection and erroneous detection.

[0011] Moreover, Taking into account the three-dimensional shape of the workpiece in the divided areas Since the area of ​​the subsequent image is shifted in sequence relative to the area of ​​the previous image by a position shift amount set according to the movement amount of the workpiece, the images of the corresponding areas are synthesized, so that the area of ​​the previous image and the corresponding area of ​​the subsequent image become the same imaging range of the workpiece, and multiple images can be synthesized with the imaging range of the workpiece being matched. Also, since the position shift amount is set for each divided area, it is possible to minimize errors in the imaging range compared to when a uniform position shift amount is applied to all divided areas. This allows for more accurate detection of surface defects.

[0012] Moreover, there is no need to perform a plurality of different conversion processes on the fused image, as described in Patent Document 2.

[0013] According to the invention described in the preceding paragraphs (2) and (9), a plurality of different candidate position shift amounts are set for each of the divided regions, and when each candidate position shift amount is applied and compared to synthesize the image, the candidate position shift amount that has the maximum evaluation value is set as the position shift amount for that region. Therefore, it is possible to set a position shift amount appropriate for each region, and to perform more accurate detection of surface defects.

[0014] According to the inventions described in the above items (3) and (10), the amount of positional shift can be easily calculated and set.

[0015] According to the inventions described in the above items (4) and (11), the amount of positional shift can be easily calculated and set.

[0016] According to the inventions described in the preceding paragraphs (5) and (12), surface defects can be identified based on a composite image of at least any one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

[0017] According to the inventions described in the preceding paragraphs (6) and (13), defect discrimination is performed based on discrimination criteria set according to the characteristics of the workpiece and the defect.

[0018] According to the invention described in the preceding paragraph (14), a computer can be caused to execute a process of dividing a plurality of images containing defect candidates into a plurality of image regions, synthesizing a plurality of images for each divided region, and performing defect determination based on this composite image. [Brief description of the drawings]

[0019] [Figure 1] 1 is a plan view showing an example of the configuration of a workpiece surface inspection system according to an embodiment of the present invention; [Diagram 2] 1 is a vertical cross-sectional view of the lighting frame as viewed from the front in the direction of travel of the workpiece. [Diagram 3] 1 is a vertical cross-sectional view of the camera frame as viewed from the front in the direction of travel of the workpiece. [Figure 4] 2 is a plan view showing an electrical configuration of the workpiece surface inspection system shown in FIG. 1. [Diagram 5] 4 is a flowchart showing the overall processing of a workpiece surface defect inspection system. [Figure 6] (A) is images acquired continuously in time series from a single camera, (B) is a diagram showing the state in which the coordinates of a tentative defect candidate are estimated in an image subsequent to the first image in (A), (C) is a diagram showing a process of superimposing each image of a group of estimated area images to create a composite image, and (D) is a diagram showing another process of superimposing each image of a group of estimated area images to create a composite image. [Figure 7] 13 is a diagram for explaining a process of correcting the center coordinates of an estimated area image from a boundary between a bright band portion and a dark band portion in the image in accordance with the position of a defect candidate. FIG. [Figure 8] 13(A) to 13(D) are diagrams showing a process of creating a composite image by superimposing each image of a group of estimated area images in different modes. [Figure 9] 11A and 11B are diagrams for explaining an example of a process for extracting a tentative defect candidate; [Figure 10] 13 is a flowchart showing the contents of a first surface defect detection process executed by the defect detection PC. [Figure 11] 11 is a flowchart for explaining the matching process in step S17 of FIG. 10 in more detail. [Figure 12] 11 is a flowchart for explaining a modified example of the matching process in step S17 of FIG. [Figure 13] 11 is a flowchart showing the details of steps S12 to S18 in the flowchart of FIG. [Figure 14] 13 is a flowchart showing a second surface defect detection process executed by the defect detection PC. [Figure 15] 15 is a flowchart showing the details of steps S12 to S18 in the flowchart of FIG. [Figure 16] FIG. 13 is a diagram for explaining the third surface defect detection process, showing a plurality of images (two in this example) acquired continuously in time series. [Figure 17] 1 is a graph showing an example of a relationship between the position of a workpiece (vehicle body) and an image plane displacement amount. [Figure 18] 13 is a flowchart showing the contents of a third surface defect detection process executed by the defect detection PC. [Figure 19] 19 is a flowchart showing the details of steps S32 to S40 in the flowchart of FIG. [Figure 20] 13 is a flowchart showing a process for creating a standard deviation image. [Figure 21] FIG. 13 is a graph showing the illuminance of a lighting device that provides light and dark pattern illumination on a workpiece. [Figure 22] 13 is a flowchart showing another example of the process of creating a standard deviation image. [Diagram 23] 13 is a flowchart showing another example of the process of creating a standard deviation image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0021] FIG. 1 is a plan view showing an example of the configuration of a work surface inspection system according to an embodiment of the present invention. In this embodiment, the work 1 is a car body, the measured portion of the work 1 is a painted surface of the car body, and a case is shown in which surface defects of the painted surface are detected. Generally, the surface of a car body is subjected to a base treatment, a metallic paint, a clear paint, etc., to form a multi-layered paint film layer, but uneven defects occur in the top clear layer due to the influence of foreign matter during painting. This embodiment is applied to the detection of such defects, but the work 1 is not limited to a car body, and may be a work other than a car body. In addition, the measured portion may be a surface other than a painted surface.

[0022] This inspection system is equipped with a work moving mechanism 2 that continuously moves a work 1 at a predetermined speed in the direction of arrow F. Two lighting frames 3, 3 are attached to the front and rear of the work moving direction in the longitudinal middle of the work moving mechanism 2, with both lower ends in a direction perpendicular to the work moving direction fixed to support bases 4, 4. In addition, the lighting frames 3, 3 are interconnected by two connecting members 5, 5. The number of lighting frames is not limited to two.

[0023] Each lighting frame 3 is formed in a gate shape as shown in the vertical cross section of Fig. 2 when viewed from the front in the traveling direction of the vehicle body, and a lighting unit 6 for illuminating the workpiece 1 is attached to each lighting frame 3. In this embodiment, the lighting unit 6 has a linear lighting section attached to the lighting frame 3 so as to surround the peripheral surface of the workpiece 1 except for the lower surface along the inner shape of the lighting frame 3, and a plurality of the linear lighting sections are attached to the lighting frame 3 at equal intervals in the moving direction of the workpiece 1. Therefore, the lighting unit 6 diffusely illuminates the workpiece with illumination light in a light and dark stripe pattern composed of lighting sections and non-lighting sections that alternate in the moving direction of the workpiece 1. The lighting unit may be curved.

[0024] A camera frame 7 is attached to the middle of the two front and rear lighting frames 3, 3 with both lower ends in a direction perpendicular to the direction of movement of the work fixed to the support bases 4, 4. When viewed from the front in the traveling direction of the work 1, the camera frame 7 is formed in a gate shape as shown in the vertical cross section of Fig. 3, and a plurality of cameras 8 as imaging means are attached to the camera frame 7 along its inner shape so as to surround the peripheral surface of the work 1 except for the underside thereof.

[0025] With this configuration, while the workpiece 1 is moved at a predetermined speed by the workpiece moving mechanism 2 and the workpiece 1 is diffusely illuminated with illumination light of a light and dark stripe pattern by the illumination unit 6, multiple cameras 8 attached to the camera frame 7 continuously capture images of each part in the circumferential direction of the workpiece 1 as the measurement part. The images are captured so that most of the image capturing ranges overlap in the previous and subsequent images. As a result, multiple images are output from each camera 8 in which the position of the measurement part of the workpiece 1 is continuously shifted in the movement direction of the workpiece 1.

[0026] FIG. 4 is a plan view showing an electrical configuration of the workpiece surface inspection system shown in FIG.

[0027] In the movement area of ​​the workpiece 1, a first position sensor 11, a vehicle type information detection sensor 12, a second position sensor 13, a vehicle speed sensor 14, and a third position sensor 15 are provided in order from the entry side along the movement direction of the workpiece 1.

[0028] The first position sensor 11 is a sensor that detects that the next work 1 approaches the inspection area. The vehicle body information detection sensor 12 is a sensor that detects the ID, model, color, destination information, etc. of the vehicle body to be inspected. The second position sensor 13 is a sensor that detects that the work 1 has entered the inspection area. The vehicle body speed sensor 14 detects the moving speed of the work 1 and monitors the position of the work 1 by calculation, but the work position may also be directly monitored by a position sensor. The third position sensor 15 is a sensor that detects that the work 1 has left the inspection area.

[0029] The workpiece surface defect inspection system further includes a master PC 21, a defect detection PC 22, a HUB 23, a NAS (Network Attached Storage) 24, a display 25, and the like.

[0030] The master PC 21 is a personal computer that comprehensively controls the entire workpiece surface defect inspection system, and includes a processor such as a CPU, a memory such as a RAM, a storage device such as a hard disk, and other hardware and software. The master PC 21 includes a movement control unit 211, a lighting unit control unit 212, a camera control unit 213, etc. as one of the functions of the CPU.

[0031] A movement control unit 211 controls the movement stop and movement speed of the moving mechanism 2, a lighting unit control unit 212 controls the lighting of the lighting unit 6, and a camera control unit 213 controls imaging of the camera 8. Imaging by the camera 8 is continuously performed in response to a trigger signal continuously transmitted from the master PC 21 to the camera 8.

[0032] The defect detection PC 22 is a surface defect detection device that executes detection processing of surface defects, and is configured by a personal computer equipped with a processor such as a CPU, a memory such as a RAM, a storage device such as a hard disk, and other hardware and software. The defect detection PC 22 includes an image acquisition unit 221, a tentative defect candidate extraction unit 222, a coordinate estimation unit 223, a defect candidate determination unit 224, an image group creation unit 225, an image synthesis unit 226, a defect detection unit 227, etc. as functions of the CPU.

[0033] The image acquisition unit 221 acquires a plurality of images captured by the camera 8 in a time series and transmitted from the camera 8 via GigE (Gigabit Ethernet (registered trademark)). The tentative defect candidate extraction unit 222 extracts tentative defect candidates based on the plurality of images captured by the image acquisition unit 221 from the camera 8, and the coordinate estimation unit 223 estimates the coordinates of the extracted tentative defect candidates in the subsequent images. The defect candidate determination unit 224 determines the defect candidates by matching the coordinates of the estimated tentative defect candidates with the actual tentative defect candidates, and the image group creation unit 225 cuts out the area around the determined defect candidate and creates an image group consisting of a plurality of images for image synthesis. The image synthesis unit 226 synthesizes each image of the created image group into one image, and the defect detection unit 227 detects and identifies defects from the synthesized image. Specific surface defect detection processing by these units in the defect detection PC 22 will be described later.

[0034] The NAS 24 is a storage device on the network and stores various data. The display 25 displays the surface defects detected by the defect detection PC 22 in association with the position information of the vehicle body, which is the workpiece 1. The HUB 23 has a function of transmitting and receiving data between the master PC 21, the defect detection PC 22, the NAS 24, the display 25, etc.

[0035] Next, the defect detection process performed by the defect detection PC 22 will be described.

[0036] While the workpiece 1 is moved at a predetermined speed by the moving mechanism 2 and illuminated from all around with a light with a light and dark stripe pattern by the lighting unit 6, the master PC 21 continuously transmits trigger signals to each camera 8, and each camera 8 continuously captures images of the measured portion of the workpiece 1. The master PC 21 sets the imaging interval, in other words, the interval between trigger signals, so that most of the imaging ranges overlap in the previous and subsequent images. By capturing images in this manner, each camera 8 obtains multiple images in which the position of the measured portion of the workpiece 1 is continuously shifted in the movement direction according to the movement of the workpiece 1.

[0037] Such a plurality of images can be obtained not only when the workpiece 1 moves relative to the fixed lighting unit 6 and camera 8 as in this embodiment, but also when the workpiece 1 is fixed and the lighting unit 6 and camera 8 are moved relative to the workpiece 1, or when the workpiece 1 and camera 8 are fixed and the lighting unit 6 is moved, from the camera 8. In other words, it is sufficient that the light and dark pattern of the lighting unit 6 moves relatively to the workpiece 1 by moving at least one of the workpiece 1 and the lighting unit 6.

[0038] The multiple images acquired by each camera 8 are transmitted to the defect detection PC 22, and the image acquisition unit 221 of the defect detection PC 22 acquires the multiple images transmitted from each camera 8. The defect detection PC 22 executes a detection process for surface defects using these images.

[0039] The overall process of the workpiece surface inspection system is shown in the flowchart of Figure 5.

[0040] In step S01, the master PC 21 judges whether the work 1 has approached the inspection range based on the signal of the first position sensor 11, and if the work 1 has not approached (NO in step S01), the process remains in step S01. If the work 1 has approached (YES in step S01), in step S02 the master PC 21 acquires individual information such as the ID, model, color, and destination information of the vehicle to be inspected based on the signal from the vehicle information detection sensor 12, and in step S03 performs initial information setting such as setting parameters of the inspection system and setting the inspection range on the vehicle body.

[0041] In step S04, the master PC determines whether the workpiece 1 has entered the inspection range based on the signal of the second position sensor 13, and if it has not entered (NO in step S04), it remains in step S04. If it has entered (YES in step S04), in step S05, the moving workpiece 1 is imaged in chronological order by the camera 8 with most of the imaging range overlapping. Next, in step S06, the defect detection PC 22 performs pre-processing in the surface defect detection process. The pre-processing will be described later.

[0042] In step S07, it is determined whether the workpiece 1 has left the inspection range based on the signal of the third position sensor 15. If the workpiece 1 has not left the inspection range (NO in step S07), the process returns to step S05 to continue imaging and the previous stage processing. If the workpiece 1 has left the inspection range (YES in step S07), in step S08, the subsequent stage processing of the surface defect detection processing by the defect detection PC 22 is performed. That is, in this embodiment, the subsequent stage processing is performed after all images of the workpiece 1 have been taken. The subsequent stage processing will be described later.

[0043] After the post-processing, in step S09, the results of the surface defect detection processing are displayed on the display 25 or the like.

[0044] Next, the surface defect detection process including the pre-processing of step S06 and the post-processing of step S08, which is performed by the defect detection PC 22, will be specifically described. [1] First surface defect detection process As described above, the defect detection PC 22 acquires multiple images from each camera 8 in which the position of the measured portion of the workpiece 1 is continuously shifted in the movement direction. This is shown in FIG. 6. Images A11 to A17 in FIG. 6(A) are images acquired continuously in time series from one camera 8. The light and dark pattern displayed in the image, in which vertically extending light bands (white parts) and dark bands (black parts) alternate horizontally, corresponds to the light and dark stripe pattern of the illumination light from the illumination unit 6.

[0045] The tentative defect candidate extraction unit 222 of the defect detection PC 22 extracts tentative defect candidates from each image. The extraction of the tentative defect candidates is performed by performing processes such as background removal and binarization. In this example, it is assumed that tentative defect candidates 30 are extracted from all images A11 to A17.

[0046] Next, the coordinate estimation unit 223 calculates representative coordinates, which are the positions of the tentative defect candidates 30, for the tentative defect candidates 30 in each of the extracted images, and sets a predetermined area around the representative coordinates as a tentative defect candidate area. Furthermore, the coordinate estimation unit 223 calculates to which coordinates the calculated representative coordinates of the tentative defect candidates will move in each of the subsequent images A12 to A17 based on the movement amount of the work 1, etc., to obtain estimated coordinates in each image. For example, the coordinate estimation unit 223 calculates to which coordinates the tentative defect candidate 30 extracted in the image A11 will move in each of the subsequent images A12 to A17, and obtains estimated coordinates in each image.

[0047] The states in which the estimated coordinates 40 of the tentative defect candidate 30 are estimated in the images A12 to A17 subsequent to the image A11 are shown in the images B12 to B17 in Fig. 6(B). Note that the images B12 to B17 are the same as the images A12 to A17 from which the tentative defect candidate 30 has been removed. Some of the intermediate images are omitted in Fig. 6(B). Also, light and dark patterns that appear in the images are omitted.

[0048] Next, the defect candidate determination unit 224 performs matching between corresponding images, such as image A12 and image B12, image A13 and image B13, . . . image A17 and image B17, among the images A12 to A17 subsequent to the image A11 shown in FIG. 6A and the images B12 to B17 in FIG. 6B for which the estimated coordinates 40 of the tentative defect candidate 30 are obtained. The matching is performed by determining whether the estimated coordinates 40 correspond to the actual tentative defect candidate 30 in the image. Specifically, the matching is performed by determining whether the estimated coordinates 40 are included in a predetermined tentative defect candidate area for the actual tentative defect candidate 30 in the image. Note that the correspondence between the estimated coordinates 40 and the actual tentative defect candidate 30 in the image may be determined by determining whether the tentative defect candidate 30 exists within a predetermined range from the estimated coordinates 40, or by determining whether the estimated coordinates 40 of the corresponding image exist within a predetermined range from the representative coordinates of the tentative defect candidate 30. When the estimated coordinates 40 correspond to the tentative defect candidate 30 in that image, the tentative defect candidate 30 included in the original image A11 and the tentative defect candidate 30 included in the subsequent image can be considered to be the same. Next, as a result of the matching, the number of images in which the estimated coordinates 40 correspond (match) with the actual temporary defect candidates 30 in the images is checked, and it is determined whether the number is equal to or greater than a preset threshold. If the number is equal to or greater than the threshold, the probability that the temporary defect candidates 30 actually exist is high, and the temporary defect candidates 30 in each image are determined as defect candidates. In the example of FIGS. 6A and 6B, matching is achieved for all of the subsequent images A12 to A17 of the image A11. That is, the estimated coordinates 40 are included in the temporary defect candidate area for the temporary defect candidate 30 in that image. If the number of images in which the estimated coordinates 40 correspond with the actual temporary defect candidates 30 is not equal to or greater than a preset threshold, it is considered that the possibility that the temporary defect candidate 30 is a defect candidate is not high, so matching is stopped and the next temporary defect candidate 30 is extracted.

[0049] Next, the image group creation unit 225 cuts out a predetermined area around the representative coordinates of the defect candidate as an estimated area for all images containing the defect candidate, as shown by a rectangular frame in each image A11 to A17 in FIG. 6(A), and creates an estimated area image group consisting of multiple estimated area images C11 to C17 as shown in FIG. 6(C). Note that it is not necessary to cut out all images containing the defect candidate, but it is possible to cut out multiple images among them, but it is preferable to have a large number of images in order to increase the amount of information and perform a highly accurate surface inspection. In addition, the estimated area may be calculated by first finding the estimated area of ​​the original image A11 and then calculating the position of the estimated area in each image from the movement amount of the work 1.

[0050] The image synthesis unit 226 superimposes and synthesizes each of the estimated area images C11 to C17 of the estimated area image group thus created, to create one synthetic image 51 shown in Fig. 6(C). The superimposition is performed at the center coordinates of each of the estimated area images C11 to C17. Examples of the synthetic image 51 include at least one of an image synthesized by calculating a statistical variation value such as a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image. Images synthesized by calculating a statistical variation value such as a standard deviation image will be described later.

[0051] Next, the defect detection unit 227 detects surface defects using the created composite image 51. The criteria for detecting surface defects may be freely selected. For example, as shown by the signal graph 61 in FIG. 6C, only the presence or absence of a defect may be detected by determining that a defect exists when the signal is equal to or greater than a reference value. Alternatively, the type of defect may be determined by comparison with a reference defect or the like. Note that the criteria for determining the presence or absence of a defect and the type of defect may be changed by machine learning or the like, or new criteria may be created.

[0052] The results of detection of surface defects are displayed on the display 25. It is preferable to display a development of the work (car body) 1 on the display 25, and to display the positions and types of surface defects on the development so that they can be seen.

[0053] In this embodiment, a plurality of estimated area images C11 to C17 cut out from a plurality of images A11 to A17 containing defect candidates are synthesized into one synthesized image 51, and defect detection is performed based on this synthesized image 51, so that information of a plurality of images is included in the synthesized image 51. Therefore, defect detection can be performed using a large amount of information about one defect candidate, and therefore even for small surface defects, stable detection can be performed with high accuracy while suppressing overdetection and erroneous detection. Furthermore, when there are images in which the estimated coordinates 40 correspond to the actual temporary defect candidates 30 in that image in a number equal to or greater than a preset threshold, a composite image is created and defect detection is performed. This allows defect detection to be performed when there is a high possibility that a defect exists, reduces the processing load, improves detection efficiency, and improves detection accuracy.

[0054] Moreover, there is no need to perform multiple different transformations on the fused image. [1-1] Modification 1 when creating a composite image However, there are cases where accuracy cannot be achieved by simply superimposing and synthesizing the multiple estimated area images C11 to C17 at the center coordinates of each image.

[0055] Therefore, it is desirable to correct the center coordinates of each estimated area image C11 to C17 and superimpose them. One example of the correction of the center coordinates is based on the relative position in the light and dark patterns in each image. Specifically, when a defect exists in the center of a light band or a dark band of a light and dark pattern, the shape is likely to be symmetrical, but as shown in FIG. 7, for example, in the estimated image C14, if the defect candidate 30 is close to the boundary with the dark band 110 in the light band 120, the boundary side becomes dark. Conversely, if the defect candidate 30 is close to the boundary in the dark band 110, the boundary side becomes bright. For this reason, for example, when the center of gravity position is calculated, the defect candidate 30 is deviated from the center position 30a. Since the position of the deviation is correlated with the position from the boundary, the center coordinates of the image are corrected according to the position L from the boundary.

[0056] 6(D) is a diagram showing how each of the estimated area images C11 to C17, whose center positions have been corrected, are superimposed at their center positions and synthesized to create a synthetic image 52. Compared to the synthetic image 51 and signal graph 61 in FIG. 6(C), a sharper synthetic image 52 is obtained, and the signal height in the signal graph 62 is also higher. For this reason, a highly accurate synthetic image 52 can be created, and thus highly accurate surface defect detection can be performed. [1-2] Modification 2 when creating a composite image Another composite image creation method will be described with reference to FIG. 8 when accuracy cannot be achieved by simply superimposing and synthesizing a plurality of estimated area images C11 to C17 at the center coordinates of each image.

[0057] The process is the same up to the creation of the estimated area images C11 to C17 in FIG. 6(C). In this example, the alignment of the estimated area images C11 to C17 is attempted with multiple combinations in which the center coordinates of each image are shifted in at least one of the left-right direction (x direction) and the up-down direction (y direction) by various alignment amounts. Then, the combination with the highest evaluation value is adopted from among these. In FIG. 8, four types of overlapping are performed, (A) to (D). The obtained composite images are shown at 53 to 56, respectively, and signal graphs based on the composite images are shown at 63 to 66. In the example of FIG. 8, (B), which obtained the highest signal, is adopted.

[0058] In this way, when creating a composite image, the multiple estimated area images C11 to C17 are aligned so as to maximize the evaluation value from multiple combinations in which the center coordinates of each image are shifted in at least one of the X-coordinate and Y-coordinate directions, so that a more accurate composite image can be created, and thus surface defect detection can be performed with high accuracy. [1-3] An example of temporary defect candidate extraction processing An example of the process of extracting temporary defect candidates that are large in size and have a gradual change in curvature, performed by the temporary defect candidate extracting unit 222, will be described.

[0059] First, we return to the principle of this method using illumination with a light and dark stripe pattern.

[0060] The illumination light reflects off the surface of the workpiece 1 and enters each pixel of the camera 8. Conversely, the light that enters each pixel is the light from the area that each pixel gazes at, where the line of sight from each pixel is reflected off the surface of the workpiece 1 and reaches it. If there is no illumination, a dark pixel signal is obtained, and if there is illumination, a bright pixel signal is obtained. If there are no defects and the workpiece 1 is flat, the area on the illumination that corresponds to each pixel is close to a point. If there is a defect, there are two types of changes to the surface of the workpiece 1: (1) a change in curvature, and (2) a change in the inclination of the surface. (1) As shown in FIG. 9(A), when the surface of the workpiece 1 has a curvature change due to the tentative defect candidate 30, the direction of the line of sight changes, and the area that each pixel looks at expands. As a result, the area that each pixel corresponds to becomes a expanded area rather than a point, and the average brightness within that area corresponds to the pixel signal. In other words, when the shape of the tentative defect candidate 30 changes suddenly, the curvature change becomes large within the area that each pixel looks at, and the area expansion in addition to the line of sight inclination cannot be ignored. The expansion of the area that is looked at results in the averaging of the illumination distribution of the signal. When the area expands with light and dark stripe pattern illumination (in FIG. 9, the white parts are light and the black parts are dark), the average value of both the light and dark areas according to the expansion is obtained. When the light and dark stripe pattern in the area where this phenomenon occurs moves sequentially, the effect can be captured in the standard deviation image. (2) As shown in FIG. 9(B), when the tentative defect candidate 30 tilts the surface of the workpiece 1 while remaining almost flat, the corresponding area remains a point, but faces a different direction from the non-tilted surface. When the tentative defect candidate 30 is large (the shape change is gradual), the area that each pixel gazes at is the same, and the change in gaze direction becomes dominant, and the change in curvature becomes gradual. The standard deviation image cannot capture this change. When the defect is large, the difference in the surface tilt between the defect-free part and the defective part can be detected by the phase image. If it is a defect-free part, the phase in the direction parallel to the stripes is the same, and the direction perpendicular to the stripes causes a certain phase change according to the period of the stripes. If it is a defective part, the regularity of the above phase is disrupted in the phase image. For example, by looking at the phase images in the X and Y directions, tentative defect candidates with gradual curved surface changes can be detected.

[0061] Any tentative defect candidate can be extracted using two types of routines: one for small tentative defect candidates and one for large tentative defect candidates. The candidate extracted by either routine can be set as the tentative defect candidate.

[0062] Now, consider a case where the size of the detected tentative defect candidate 30 is to be reported as a result. The correlation between the size of the defect as determined by visual inspection and the size of the defect detected from the image is determined from an approximation circle of the part where the inclination of the surface of the defect is at a certain angle. When the defect size is small, a certain linear relationship is observed, but when the defect size is large and the surface inclination is gentle, the relationship is nonlinear. Therefore, the gentle tentative defect candidate 30 detected in the phase image needs to be corrected from a separately obtained calibration curve, since there is no linear relationship between the defect signal and the defect size. [1-4] Detection of thread defects As an example of defect detection by the defect detection unit 227, a process of detecting thread defects will be described.

[0063] Thread defects are defects in which thread-like foreign matter is trapped beneath the paint; they are long and thin, not circular. Some are small in the line width direction (e.g., less than 0.2 mm) but long in the longitudinal direction (e.g., 5 mm or more). Because they are very narrow in the width direction, they are small, and have a gradual change in curvature in the longitudinal direction. They may be overlooked using detection methods for small defects and large defects (defects with gentle slopes) similar to those used to extract tentative defect candidates. After the prescribed processing, they are binarized and granulated, and it is determined whether they are defective or not based on the area of ​​each part.

[0064] Since the granules are narrow but long, if they are detected properly, the specified area can be obtained. However, the granules are easy to detect when their length is parallel to the direction in which the light and dark pattern extends, but difficult to find when they are perpendicular. Defects occur in the length direction, making the granules appear shorter than they actually are, meaning that the granules' area tends to be smaller.

[0065] Therefore, based on the shape information of the defect obtained from the phase image, if there is a certain amount of elongation in the longitudinal direction, that is, if the circularity is lower than a predetermined value, the threshold value for area judgment is reduced to prevent thread particles from going undetected. [1-5] Flowchart Fig. 10 is a flowchart showing the contents of the surface defect detection process executed by the defect detection PC 22. This surface defect detection process shows in more detail the contents of the first stage process of step S06 and the second stage process of step S08 in Fig. 5. Furthermore, this surface defect detection process is executed by a processor in the defect detection PC 22 operating in accordance with an operating program stored in an internal storage device such as a hard disk device.

[0066] In step S11, the individual information acquired by the master PC 21 in step S02 of FIG. 5, and initial information such as the parameter settings and the inspection range on the vehicle body set in step S03 are acquired from the master PC 21.

[0067] Next, in step S12, an image captured by the camera 8 is acquired, and then in step S13, pre-processing is performed, for example, setting of position information for the image based on initial setting information or the like.

[0068] Next, in step S14, tentative defect candidates 30 are extracted from each image, and then in step S15, the amount of movement of the workpiece 1 is calculated for one tentative defect candidate 30, and in step S16, the coordinates of that tentative defect candidate 30 in the subsequent image are estimated as estimated coordinates 40.

[0069] In step S17, matching is performed. That is, it is determined whether the estimated coordinates 40 are present within a predetermined temporary defect candidate area for the actual temporary defect candidates 30 in the image, and if the number of images with matching is equal to or greater than a preset threshold, in step S18, the temporary defect candidates 30 in each image are determined to be defect candidates.

[0070] In step S19, for all images having defect candidates, a predetermined area around the representative coordinates of the defect candidate is cut out as an estimated area, and a group of estimated area images C11 to C17 is created, and then the process proceeds to step S20. Steps S12 to S19 are the pre-stage process.

[0071] In step S20, it is determined whether the vehicle body, which is the work 1, has left the inspection range based on information from the master PC 21. If the vehicle body has not left the inspection range (NO in step S20), the process returns to step S12, and image acquisition from the camera 8 continues. If the vehicle body has left the inspection range (YES in step S20), the alignment amount of each estimated area image C11 to C17 is set in step S21. Then, in step S22, each estimated area image C11 to C17 is synthesized to create a synthesized image, and then in step S23, defect detection processing is performed. Steps S21 to S23 are the latter stage processing. After defect detection, the detection result is output to the display 25 or the like in step S24.

[0072] The matching process in step S17 will be described in detail with reference to the flowchart in FIG.

[0073] In step S201, a variable K representing the number of images that match the tentative defect candidate 30 is set to zero, and in step S202, a variable N representing the number of images to be judged as to whether they match the tentative defect candidate 30 is set to zero.

[0074] In step S203, the tentative defect candidate 30 is extracted, and then in step S204, N is set to N+1. Next, in step S205, it is determined whether the tentative defect candidate 30 and the estimated coordinate 40 match. If they match (YES in step S205), then in step S206, K is set to K+1, and the process proceeds to step S207. If in step S205, the tentative defect candidate 30 and the estimated coordinate 40 do not match (NO in step S205), the process proceeds to step S207.

[0075] In step S207, it is checked whether N has reached a predetermined number of images (7 in this case), and if not (NO in step S207), the process returns to step S203, and a tentative defect candidate 30 is extracted for the next image. If N has reached the predetermined number of images (YES in step S207), the process returns to step S208, and it is determined whether K is equal to or greater than a predetermined threshold value (5 in this case). If it is not equal to or greater than the threshold value (NO in step S208), the process returns to step S201. Therefore, in this case, no further cut-out process of the estimated area image or image synthesis process is performed, N and K are reset, and the next tentative defect candidate 30 is extracted.

[0076] If K is equal to or greater than the threshold (YES in step S208), in step S209, the tentative defect candidate 30 is determined as a defect candidate and the information is stored, and then in step S210, estimated area images are cut out for the K images that match. Then, in step S211, the cut out K estimated area images are synthesized, and then in step S212, it is determined whether or not a surface defect has been detected. If a surface defect is detected (YES in step S212), in step S213, it is determined to be a surface defect and the information is stored, and then the process proceeds to step S214. If a surface defect is not detected (NO in step S212), the process proceeds directly to step S214.

[0077] In step S214, it is checked whether the detection process has been performed for all the inspection target portions of the workpiece, and if not (NO in step S214), the process returns to step S201, N and K are reset, and the next tentative defect candidate 30 is extracted. If the detection process has been performed for all the inspection target portions (YES in step S214), the process ends.

[0078] In this embodiment, when the number K of images in which the tentative defect candidate 30 corresponds (matches) with the estimated coordinates 40 is not equal to or greater than a threshold value, the number of matching images is small and the possibility that the tentative defect candidate 30 is a defect candidate is low, so subsequent processing is stopped, whereas when the number of matching images is K or more, the possibility that the tentative defect candidate 30 is a defect candidate is high, so clipping of estimated area images, image composition, and defect detection are performed. Therefore, compared to when clipping of estimated area images, image composition, and defect detection are performed regardless of the number of matching images, the processing load is small, detection efficiency is improved, and detection accuracy is also improved.

[0079] Fig. 12 is a flowchart for explaining a modified example of the matching process in step S17 in Fig. 10. In this example, if the number of matched images K does not reach a fixed value before the number of images N reaches a predetermined number, it is determined that the tentative defect candidate 30 is not likely to be a defect candidate, and the subsequent process is stopped at that point.

[0080] In step S221, a variable K representing the number of images that match the tentative defect candidate 30 is set to zero, and in step S222, a variable N representing the number of images to be judged as to whether they match the tentative defect candidate 30 is set to zero.

[0081] In step S223, the tentative defect candidate 30 is extracted, and then in step S224, N is set to N+1. Next, in step S225, it is determined whether the tentative defect candidate 30 and the estimated coordinate 40 match. If they match (YES in step S225), K is set to K+1 in step S226, and the process proceeds to step S227. If the tentative defect candidate 30 and the estimated coordinate 40 do not match (NO in step S225), the process proceeds to step S227.

[0082] In step S227, it is checked whether N has reached a second predetermined number of images (8 in this case), and if so (YES in step S227), it is checked in step S228 whether K has reached a second preset threshold value (4 in this case), and if not (NO in step S228), the process returns to step S221. Therefore, in this case, the subsequent cutout process of the estimated area image and the image synthesis process are not performed, N and K are reset, and the next tentative defect candidate 30 is extracted.

[0083] In step S228, if K reaches the second threshold value (YES in step S228), the process proceeds to step S229. Also, in step S227, if N does not reach the second predetermined number of images (8) (NO in step S227), the process proceeds to step S229.

[0084] In step S229, it is checked whether N has reached a first predetermined number (9 in this case), and if not (NO in step S229), the process returns to step S223, and a tentative defect candidate 30 is extracted for the next image. If N has reached the first predetermined number (YES in step S229), then in step S230, it is determined whether K is equal to or greater than a first threshold value (5 in this case). If K is not equal to or greater than the first threshold value (NO in step S230), the process returns to step S201. Therefore, in this case, no further cut-out process of the estimated area image or image synthesis process is performed, N and K are reset, and the next tentative defect candidate 30 is extracted.

[0085] If K is equal to or greater than the first threshold (YES in step S230), in step S231, the tentative defect candidate 30 is determined as a defect candidate and the information is stored, and then in step S232, estimated area images are cut out for the K images that match. Then, in step S233, the cut out K estimated area images are synthesized, and then in step S234, it is determined whether or not a surface defect has been detected. If a surface defect is detected (YES in step S234), in step S235, it is determined to be a surface defect and the information is stored, and then the process proceeds to step S236. If a surface defect is not detected (NO in step S234), the process proceeds directly to step S236.

[0086] In step S236, it is checked whether the detection process has been performed for all the inspection target portions of the workpiece, and if not (NO in step S236), the process returns to step S201, N and K are reset, and the next tentative defect candidate 30 is extracted. If the detection process has been performed for all the inspection target portions (YES in step S236), the process ends.

[0087] In this embodiment, in addition to the same effects as those of the embodiment shown in the flowchart of Fig. 11, the following effects are also achieved. That is, if the number K of images in which the tentative defect candidates 30 correspond (match) with the estimated coordinates 40 does not reach the first threshold value less than the second threshold value at the stage where the number N of images from which the tentative defect candidates 30 have been extracted is the first set value less than the second set value, i.e., at an intermediate stage, it is determined that there are few matching images and that the tentative defect candidates 30 are not highly likely to be defect candidates, and the matching process is not continued until the final image, and subsequent processes are stopped. As a result, unnecessary processes are not continued, so that the processing load can be further reduced and the detection accuracy can be further improved.

[0088] FIG. 13 is a flowchart showing the details of steps S12 to S18 in the flowchart of FIG. 10, which is the first stage of the surface defect detection process, and the same steps as in the flowchart of FIG. 10 are given the same step numbers.

[0089] As one workpiece 1 moves from the time it enters the inspection range until it leaves the inspection range, the camera 8 continuously captures images of the workpiece 1, and the defect detection PC 22 acquires images from the first image capture to the final image capture in step S12. Here, it is assumed that the images in which one tentative defect candidate 30 is captured are the nth image capture to the (n+m-1)th image capture.

[0090] In step S13, each image is pre-processed, and then in step S14, tentative defect candidates 30 are extracted for each image captured from the nth image to the (n+m-1)th image, and representative coordinates and tentative defect candidate areas of the extracted tentative defect candidates 30 are obtained. Next, in step S16, the coordinates to which the representative coordinates of the tentative defect candidates move for each of the subsequent images are calculated based on the calculation of the movement amount of the work 1 in step S15, and estimated coordinates 40 in each image are obtained.

[0091] In step S17, matching is performed for each subsequent image, and if the number of images with matching is equal to or greater than a threshold (e.g., m), the tentative defect candidate 30 is determined as a defect candidate in step S18. In step S19, an estimated area is calculated for each image, and an estimated area image group consisting of multiple estimated area images C11 to C17 is created. [2] Second surface defect detection process In the above-described first surface defect detection process, the defect detection PC 22 extracts tentative defect candidates 30 from images continuously acquired in time series by the camera 8.

[0092] Although the method of extracting the tentative defect candidates 30 is not limited, it is preferable to perform the following processing to extract the tentative defect candidates 30, since this emphasizes the defective areas and enables more accurate extraction of the tentative defect candidates 30.

[0093] That is, after performing binarization processing on each of the images A11 to A17 (shown in FIG. 6) acquired by the camera 8, a threshold value is applied or a corner detection function is applied to extract feature points of the images. Then, the tentative defect candidates 30 may be extracted by calculating multidimensional feature amounts for each of the extracted feature points.

[0094] More preferably, before extracting the feature points, each image acquired by the camera 8 is binarized, its contour is extracted, and then an image that has been expanded and contracted a predetermined number of times is subtracted to create an orange peel mask that removes the boundary between the light and dark bands. It is preferable to apply this created mask to extract feature points from each image after masking the boundary between the light and dark bands, thereby enabling the extraction of tentative defect candidates with even greater accuracy.

[0095] In addition, the extraction of the tentative defect candidates 30 may be performed by extracting feature points of an image, and then calculating multidimensional feature amounts for each extracted feature point for all pixels in a specific range surrounding the extracted feature point, based on brightness gradient information in all vertical, horizontal and diagonal directions from the pixel.

[0096] After the extraction of the tentative defect candidate 30, a group of estimated area images consisting of multiple estimated area images C11 to C17 is created in the same manner as the first surface defect detection process described above, and then defect detection is performed for each tentative defect candidate using this group of estimated area images.

[0097] In this way, in the second surface defect detection process, feature points are extracted from multiple images obtained by camera 8 in which the position of the measured part of workpiece 1 is continuously shifted, and multidimensional feature quantities are calculated for each extracted feature point to extract tentative defect candidates 30. This enables highly accurate extraction of tentative defect candidates 30, and ultimately enables highly accurate surface defect detection.

[0098] Moreover, the coordinates of the extracted tentative defect candidate 30 are obtained, and for each of a plurality of images following the image from which the tentative defect candidate 30 is extracted, the coordinates to which the tentative defect candidate 30 will move are calculated to obtain estimated coordinates 40, and it is determined whether or not the estimated coordinates 40 correspond to the tentative defect candidate 30 in that image, and if there are a number of images among the subsequent images in which the estimated coordinates 40 correspond to the tentative defect candidate that is equal to or greater than a preset threshold, the tentative defect candidate 30 is determined as a defect candidate. Then, for each determined defect candidate, a predetermined area around the defect candidate is cut out as an estimated area from among the plurality of images including the defect candidate, and an estimated area image group consisting of a plurality of estimated area images C11 to C17 is created, and defect discrimination is performed based on the created estimated area image group.

[0099] In other words, since the estimated area images C11 to C17 containing defect candidates each contain multiple pieces of information about one defect candidate, it is possible to perform defect detection using more information. Therefore, even for small surface defects, it is possible to perform stable detection with high accuracy while suppressing overdetection and erroneous detection. [2-1] Flowchart Fig. 14 is a flowchart showing the second surface defect detection process executed by the defect detection PC. Note that steps S11 to S13 and steps S15 to S20 are the same as steps S11 to S13 and steps S15 to S20 in Fig. 10, so the same step numbers are used and the description will be omitted.

[0100] After the preprocessing in step S13, a citron skin mask is created in step S141, and in step S142, the created citron skin mask is applied to extract feature points.

[0101] Next, in step S143, multidimensional feature quantities are calculated for each of the extracted feature points, and in step S144, tentative defect candidates 30 are extracted, after which the process proceeds to step S16.

[0102] If the vehicle body, which is the work 1, exits the inspection range in step S20 (YES in step S20), a defect discrimination process is executed using the created estimated area image group in step S23, and the discrimination result is displayed in step S24.

[0103] Fig. 15 is a flowchart showing the details of steps S12 to S18 in the flowchart of Fig. 14, and the same step numbers are used for the same processes as in the flowchart of Fig. 14. Note that steps S12, S13, and S15 to S19 are the same as the processes in steps S12, S13, and S15 to S19 in Fig. 13, so their explanation will be omitted.

[0104] After the preprocessing in step S13, a citron skin mask is created for each image in step S141. In step S142, the created citron skin mask is applied to each image to extract feature points of each image.

[0105] In step S143, multidimensional feature amounts are calculated for each feature point of each extracted image, and in step S144, tentative defect candidates are extracted for each image, after which the process proceeds to step S16. [3] Third surface defect detection process The first surface defect detection process described above extracts tentative defect candidates 30 from each of the images A11 to A17, determines the defect candidates, calculates estimated areas around the defect candidates, and combines the multiple estimated area images C11 to C17 to perform defect detection.

[0106] In contrast, in the third surface defect detection process, a plurality of continuous time-series images acquired from the camera 8 are each divided into a plurality of regions, and a plurality of previous and subsequent images are synthesized in the corresponding regions, and then defects are detected. However, since the workpiece 1 is moving, the imaging range of the workpiece 1 shown in the region of the previous image and the imaging range of the workpiece 1 shown in the region of the subsequent image are not the same, and the imaging position differs according to the amount of movement of the workpiece 1, so the position of the region of the subsequent image is shifted from the region of the previous image by an amount of position shift according to the amount of movement of the workpiece 1, and then the images are synthesized. In addition, the amount of position shift between the region of the previous image and the corresponding region of the subsequent image differs depending on the position of the divided region, so the amount of position shift according to the amount of movement of the workpiece 1 is set for each divided region.

[0107] As will be described in detail below, the multiple images captured continuously by the camera 8 and continuously acquired in time series by the defect detection PC 22 are the same as the images acquired in the first surface defect detection process.

[0108] FIG. 16 shows multiple images A21, A22 that are continuously acquired in time series. Two images are shown in this example, but the number of images is actually much greater. Note that light and dark patterns that appear in the images A21, A22 are omitted. These images A21, A22 are divided into multiple regions 1 to p in a direction perpendicular to the movement direction of the workpiece (the up and down direction in FIG. 16). Regions 1 to p are the same size and at the same position (same coordinates) in images A21, A22.

[0109] Since the workpiece is moving, the imaging range corresponding to the image in each of the regions 1 to p in the image A21 acquired from the camera 8, for example, is shifted in the movement direction by the amount of movement of the workpiece 1 with respect to the original regions 1 to p in the subsequent image A22, as shown by the arrow. Therefore, by shifting the position of each of the regions 1 to p in the image A22 by the position shift amount S corresponding to the amount of movement of the workpiece, each of the regions 1 to p in the image A21 and each of the regions 1 to p after the position shift in the image A22 become the same imaging range on the workpiece 1. Since such a relationship occurs between each of the regions 1 to p in the previous and subsequent captured images, the imaging range of each of the regions 1 to p in the original image A21 and each of the subsequent images can be matched by sequentially shifting the regions 1 to p in each of the subsequent images by the position shift amount S.

[0110] However, as shown in image A22 of Fig. 16, the amount of deviation from the original regions 1 to p differs for each of the regions 1 to p. For example, if a straight portion and a curved portion of the workpiece 1 are present in the imaging range of one camera 8, the amount of positional deviation of the region corresponding to the straight portion and the region corresponding to the curved portion in the image is not the same. It also differs depending on the perspective of the camera 8. For this reason, even if all the regions 1 to p are shifted by a uniform amount of positional deviation, some regions do not necessarily have the same imaging range.

[0111] Therefore, in this embodiment, a position shift amount S is calculated and set for each of the regions 1 to p. Specifically, average magnification information within each of the regions 1 to p is obtained from the camera information, camera position information, the three-dimensional shape of the workpiece, and the position information of the workpiece. Then, a position shift amount S is calculated for each of the regions 1 to p from the obtained magnification information and an approximate moving speed assumed in advance, and this is set as the position shift amount S for each of the regions 1 to p.

[0112] Here, we will provide additional information on the calculation of the position shift amount. Consider the case where multiple images of a moving workpiece 1 are captured at equal time intervals. We will focus on the movement of the same point between two consecutive captured images.

[0113] The amount of movement on the image is related to the imaging magnification of the camera and the speed of the workpiece. The imaging magnification of the camera depends on (1) the lens focal length and (2) the distance from the camera to each part of the workpiece being imaged. Regarding (2), parts closer to the camera in the image have a greater amount of movement than parts farther away. If the 3D shape of workpiece 1, as well as the installation position of camera 8 and the position and orientation of workpiece 1 are known, it is possible to calculate where the point of interest will appear in the image captured at a given moment.

[0114] When the workpiece 1 moves and its position changes, it is possible to calculate how many pixels the same point of interest moves on the image in two consecutive images. For example, if we consider a case where the workpiece moves 1.7 mm between adjacent images using a sensor with a focal length of 35 mm and a pixel size of 5.5 μm, as shown in the graph in Figure 17, the distance to the workpiece 1 (Zw) is 600 to 1100 mm, so the movement distance on the screen moves 18 to 10 pixels.

[0115] If the registration error required to create a composite image is to be kept to ±1 pixel, then the distance difference should be ±5 cm. Areas are divided on the image so that the distance difference from the camera is within ±5 cm. For each divided area, the average positional deviation between successive images is calculated from the approximate moving speed of the work 1. For each area 1 to p, three types of positional deviation can be set, combining that positional deviation and a deviation of ±1 pixel. However, the positional deviation is not limited to three types, and the distance difference is not limited to ±5 cm.

[0116] The position shift amount S for each of the set regions 1 to p is stored in association with the region 1 to p in a table in the memory section of the defect detection PC 22, and for imaging parts for which the same position shift amount can be set, such as parts of the same shape of the work 1 or workpieces of the same type, the position shift amount is set by retrieving the position shift amount from the table.

[0117] Next, a predetermined number of continuous images are synthesized for each of the regions 1 to p with the position of each region 1 to p shifted by the set position shift amount S. The synthesis involves superimposing the images of each region with the position of each region 1 to p shifted by the set position shift amount S, and performing an operation for each pixel of the superimposed images at corresponding coordinates to create a synthetic image for each pixel. Examples of synthetic images include at least one of an image synthesized by calculating a statistical variation value such as a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

[0118] Next, preprocessing such as background removal and binarization is performed on the composite image, for example the standard deviation image, to extract defect candidates, and then, if necessary, surface defects are detected using a calculation different from the processing used for extracting defect candidates or the composite image. The criteria for detecting surface defects may be freely selected, and may determine only the presence or absence of a defect, or may determine the type of defect by comparison with a reference defect, etc. The criteria for determining the presence or absence of a defect or the type of defect may be set according to the characteristics of the workpiece and the defect, and may be changed by machine learning, etc., or new criteria may be created.

[0119] The results of detection of surface defects are displayed on the display 25. It is preferable to display a development of the work (car body) on the display 25 and to display the positions and types of surface defects on the development so that they can be seen.

[0120] In this embodiment, multiple captured images A21, A22 continuously acquired in time series from the camera are divided into multiple regions 1 to p, multiple images for each divided region 1 to p are synthesized, and defect detection is performed based on this synthesized image, so that the synthesized image contains information from multiple images. Therefore, defect detection can be performed using a large amount of information about one defect candidate, and even for small surface defects, highly accurate and stable detection can be performed while suppressing overdetection and erroneous detection.

[0121] Moreover, the regions 1 to p of the subsequent image A22 are shifted in order relative to the regions 1 to p of the previous image A21 by a position shift amount S set according to the movement amount of the work 1, and the images of the corresponding regions are synthesized, so that the regions of the previous image and the corresponding regions of the subsequent image become the same imaging range of the work 1, and multiple images can be synthesized with the imaging range of the work 1 matched. Also, because the position shift amount is set for each divided region 1 to p, errors in the imaging range can be suppressed to a minimum compared to the case where a uniform position shift amount is applied to all divided regions 1 to p. This allows for more accurate detection of surface defects. [2-1] Variation 1 regarding the amount of position shift In the above example, the position shift amount S corresponding to each divided region 1 to p was calculated for each region 1 to p from the magnification information of each region 1 to p and the approximate movement speed estimated in advance, but the position shift amount S may be set based on the result of setting multiple position shift amounts for each region 1 to p.

[0122] For example, for each of the regions 1 to p, position shift amount candidates are set under multiple conditions, from slow to fast, including the expected moving speed. Then, each position shift amount candidate is applied to create a composite image, and further defect detection is performed as necessary. From the comparison of these, the position shift amount S with the highest evaluation is adopted.

[0123] In this way, multiple candidate position shift amounts are set under different conditions for each of the regions 1 to p, and the candidate position shift amount with the highest evaluation is adopted as the position shift amount S for that region 1 to p based on a comparison of images synthesized using each candidate position shift amount. This makes it possible to set an appropriate position shift amount S for each of the regions 1 to p, thereby enabling surface defects to be detected with even higher accuracy. [2-2] Variation 2 regarding the amount of position shift The position shift amount S for each of the regions 1 to p may be set as follows. That is, as shown in the graph of Fig. 17, if the movement distance of the work 1 between adjacent images is known, the position shift amount on the image can be calculated. In the above example, the position shift amount was set based on the work movement speed assumed in advance.

[0124] The appropriate amount of position shift for each frame when creating a composite image may be determined based on the actually measured position of the workpiece, which eliminates the need to select an optimal amount of position shift from a number of possible amounts of position shift.

[0125] The method for measuring the workpiece position is as follows: Workpiece 1 or the same part of a support member that moves in the same way as workpiece 1 is imaged with multiple position-specific cameras lined up in the direction of workpiece 1 movement, and the position information of the workpiece is obtained from the images. First, if workpiece 1 has a characteristic hole, that hole, or a mark placed on the platform that holds and moves workpiece 1, is used as the target for measuring the position or speed of workpiece 1.

[0126] To detect targets, multiple cameras in addition to camera 8 are prepared. For example, they are arranged in a row in the direction of travel of workpiece 1, peering at the side of the workpiece from the side. They are arranged so that the horizontal fields of view of multiple cameras, when joined together, cover the entire length of workpiece 1. The magnification can be calculated from the distance from the camera to workpiece 1 and the focal length of the camera. Based on the magnification, the actual position can be found from the position on the image. The positional relationship between each camera is known, and the position of workpiece 1 can be found from the image information of each camera.

[0127] By associating the workpiece position information from the multiple cameras, an appropriate amount of position shifting can be obtained from the image of the defect extraction camera 8. For example, for each virtually divided area on the workpiece 1 so that the distance difference on the workpiece as seen by the camera is ±5 cm, an average amount of movement on the image between adjacent images according to the amount of movement of the workpiece 1 is determined, and a composite image is created as the amount of position shifting when superimposing. [2-3] Variation 3 regarding the amount of position shift In the second modification, the position of the workpiece is obtained using multiple cameras arranged in a row. Instead, the workpiece 1 or the same part of a support member that moves in the same way as the workpiece 1 may be measured by a measurement system including a distance sensor, a speed sensor, or a vibration sensor, or a combination of these, to obtain the workpiece position information.

[0128] We will explain how to measure the workpiece position. The target is a part of the workpiece 1, or the same part of a support member that moves in the same way as the workpiece 1. To detect the workpiece position, a "sensor that detects the workpiece position passing a reference point + distance sensor" or a "sensor that detects the workpiece position passing a reference point + speed sensor + imaging time interval between adjacent images" are used. The former method finds the workpiece position directly. The latter method multiplies the speed information from the speed sensor by the imaging interval to find the workpiece position at the time each image was captured.

[0129] By associating the above workpiece position information, an appropriate amount of position shifting can be obtained from the image of the defect extraction camera 8. For example, for each virtually divided area on the workpiece 1 so that the distance difference on the workpiece as seen by the camera is ±5 cm, an average amount of movement on the image between adjacent images according to the amount of movement of the workpiece 1 is determined, and a composite image is created as the amount of position shifting when superimposing. [2-4] Flowchart The overall processing of the workpiece surface inspection system is carried out according to the flowchart shown in FIG.

[0130] Fig. 18 is a flowchart showing the contents of the third surface defect detection process executed by the defect detection PC 22. This surface defect detection process shows in more detail the contents of the first stage process of step S06 and the second stage process of step S08 in Fig. 5. This surface defect detection process is executed by a processor in the defect detection PC 22 operating in accordance with an operating program stored in an internal storage device such as a hard disk device.

[0131] In step S31, the individual information acquired by the master PC 21 in step S02 of FIG. 5, and initial information such as the parameter settings and the inspection range on the vehicle body set in step S03 are acquired from the master PC 21.

[0132] Next, in step S32, images A21 and A22 captured by camera 8 are acquired, and then in step S33, each of images A21 and A22 is divided into a plurality of regions 1 to p. Meanwhile, in step S35, a plurality of position shift amount candidates are set for each of the divided regions 1 to p based on the position and moving speed of work 1 (step S34) and the like.

[0133] Next, in step S36, for one region, multiple images each shifted by multiple candidate position shift amounts are composited to create multiple composite image candidates for each region. In step S37, the composite images created for each candidate position shift amount are compared, and the most highly evaluated candidate position shift amount is set as the position shift amount for that region 1 to p. The multiple images are then composited again for each region using that position shift amount to create a composite image.

[0134] In step S38, pre-processing such as background removal and binarization is performed on the composite image, and then defect candidates are extracted in step S39. By performing such processing for each of the multiple regions 1 to p and for each of a predetermined number of images, in step S40, a large number of defect candidate images from which defect candidates have been extracted are created, and then the process proceeds to step S41. Steps S32 to S40 are the pre-stage processing.

[0135] In step S41, it is determined whether the vehicle body has left the inspection range based on information from the master PC 21. If the vehicle body has not left the inspection range (NO in step S41), the process returns to step S32, and image acquisition from the camera 8 continues. If the vehicle body has left the inspection range (YES in step S41), in step S42, defect detection processing is performed on the defect candidate image group. Step S42 is the latter stage processing. After defect detection, in step S43, the detection result is output to the display 25 or the like.

[0136] FIG. 19 is a flowchart showing the details of steps S32 to S40 in the flowchart of FIG. 18, which is the first stage of the surface defect detection process, and the same steps as in the flowchart of FIG. 18 are given the same step numbers.

[0137] As one work 1 moves from entering the inspection range to leaving the inspection range, the camera 8 continuously captures images of the work 1, and the defect detection PC 22 acquires images from the first image to the final image in step S32. Here, an example is shown in which images from the nth image to the (n+m-1)th image are used.

[0138] In step S33, each image is divided into p image regions, for example, regions 1 to p. In step S35, q position shift amount candidates are set for each of the p regions, and in step S36, q composite image candidates are created by applying the q position shift amount candidates to each of the p image regions. In other words, q composite images are created for each of the regions 1 to p.

[0139] In step S37-1, for each of regions 1 to p, the composite image with the highest evaluation value is selected, and the position shift candidates corresponding to the selected composite image are determined as the position shift amount for that image region.

[0140] Then, in step S37-2, the determined position shift amount is applied to each of the regions 1 to p to create a composite image.

[0141] The subsequent pre-processing (step S38), defect candidate extraction processing (step S39), and defect candidate image group creation processing (step S40) are similar to those in FIG. 18, and therefore description thereof will be omitted. [4] Creation of standard deviation images, etc. In the first surface defect detection process and the third surface defect detection process, when the workpiece is moved while being irradiated with a light and dark illumination pattern, a plurality of images to be combined are created based on a plurality of images of overlapping imaging ranges captured in time series by the camera 8, and these plurality of images are combined into one image to create a composite image. One such composite image is an image combined by calculating a statistical variation value such as a standard deviation image.

[0142] The statistical variation value may be at least one of the variance, standard deviation, and half-width. Any of them may be calculated, but here, the case where the standard deviation is calculated and combined will be described.

[0143] The standard deviation is calculated for each corresponding pixel of multiple images. Fig. 17 is a flowchart showing the process of creating a standard deviation image. The processes shown in Fig. 20 and the subsequent flowcharts are executed by the defect detection CPU operating according to an operation program stored in a storage unit or the like.

[0144] In step S51, original images (N images) to be synthesized are generated, and in step S52, the sum of squares of brightness values ​​(hereinafter also referred to as pixel values) is calculated for each pixel of the first original image, and then in step S53, the sum of pixel values ​​is calculated for each pixel. For the first image, the sum of squares and the sum calculation are the results for only the first image.

[0145] Next, in step S54, it is checked whether there is a next image, and if there is (YES in step S54), the process returns to step S52, where the pixel value of each pixel in the second image is squared and added to the squared value of each corresponding pixel value in the first image. Next, in step S53, each pixel value in the second image is added to each corresponding pixel value in the first image.

[0146] This process is performed for N images in order, and the sum of the squares of each pixel value and the sum of each pixel value are calculated for each corresponding pixel in the N images.

[0147] When the above process for N images is completed (NO in step S54), the average of the sums of the pixel values ​​calculated in step S53 is calculated in step S55, and then the square of the average of the sums is calculated in step S56.

[0148] Next, in step S57, the mean square, which is the average value of the square sums of the pixel values ​​calculated in step S52, is calculated, and then in step S57, the variance is calculated from the formula {(mean square)-(square of the mean)}. Then, in step S59, the standard deviation, which is the square root of the variance, is calculated.

[0149] The standard deviation thus obtained is desirably normalized, and a composite image is created based on the result. Note that, when using variance or half-width as a statistical variation value, the calculation can be performed in the same manner.

[0150] A surface defect detection process is performed based on the created composite image. The detection process may be performed in the same manner as the first surface defect detection process and the third surface defect detection process.

[0151] In this way, corresponding pixels of multiple images are combined by calculating statistical variation values ​​and applying this to all pixels to create a composite image, so even if the number of images to be combined is small, a composite image with a high S / N ratio for defect detection can be created, and this composite image can be used to perform highly accurate defect detection, reducing the detection of unnecessary defect candidates and preventing overlooking the detection of necessary defects. Moreover, the cost is lower than when a composite image is created using maximum and minimum values, etc. [4-1] Other embodiment 1 regarding standard deviation image A graph of the illuminance of the lighting unit 6 that provides illumination with a light and dark pattern on the workpiece 1 is shown in Fig. 18. In the graph of Fig. 21, the peak 71 of the waveform indicates a light band, and the bottom 72 indicates a dark band.

[0152] The rising and falling portions 73 of the waveform from the light band to the dark band or from the dark band to the light band are not actually vertical but are inclined. In the image portions corresponding to the midpoints of the rising and falling portions 73, the pixel values ​​become intermediate gradations, which affects the variation.

[0153] When capturing images multiple times in one period of the illumination pattern, for example, capturing images eight times as shown by the black circles in Fig. 21(A), it is highly likely that two pixels out of eight pixels in each of the eight images obtained will have an intermediate gradation pixel value corresponding to the midpoint. On the other hand, when capturing images seven times at the timings shown by the black circles in Fig. 21(B), it is highly likely that at least one pixel out of seven pixels in each of the seven images obtained will have an intermediate gradation pixel value corresponding to the midpoint.

[0154] As described above, such pixel values ​​of intermediate gradations affect the variation and cause a decrease in the accuracy of defect detection. Therefore, it is preferable to exclude such pixel values ​​of intermediate gradations from the sampling candidates for the variation calculation, and to perform the variation calculation only for the selected optimal sampling candidate. Specifically, when the number of original images to be synthesized in one cycle of the illumination pattern is an even number, two pixel values ​​of intermediate gradations are thinned out from the pixel values ​​of the multiple pixels to calculate the variation, and when the number of original images is an odd number, one pixel value of intermediate gradations is thinned out from the pixel values ​​of the multiple pixels to calculate the variation. In this way, by excluding pixel values ​​of intermediate gradations from the sampling candidates for the variation calculation and performing the variation calculation only for the selected optimal sampling candidate, the statistical calculation of the variation value is performed only for the optimal sampling candidate, and the influence of the pixels excluded from the sampling candidates can be suppressed, so that it is possible to create a synthetic image that can perform high-precision defect detection even when the number of images to be synthesized is small.

[0155] FIG. 22 is a flowchart showing a process for creating a standard deviation image by excluding pixel values ​​of intermediate gradations from sampling candidates for the variation calculation and performing the variation calculation only for the selected optimal sampling candidates.

[0156] In step S61, multiple (N) original images are generated, and then in step S62, sampling data, which is the pixel values ​​of N images, for each pixel in each image are sorted, and the median value of 1 (N is an odd number) or 2 (N is an even number) is removed.

[0157] Next, in step S63, the standard deviation is calculated for each pixel using N-1 (N is an odd number) or N-2 (N is an even number) values.

[0158] The standard deviation thus obtained is desirably normalized, and a composite image is created based on the result. Note that, when using variance or half-width as a statistical variation value, the calculation can be performed in the same manner. [4-2] Other embodiment 2 regarding standard deviation image In this embodiment as well, imaging is performed a plurality of times (N times) for one period of the illumination pattern. N times may be a small number.

[0159] In this embodiment, similarly to the first embodiment related to the standard deviation image, when there are an odd number of original images to be synthesized in one period of the illumination pattern, the standard deviation is calculated for each pixel using N-1 pieces of sampling data (pixel values), and when there are an even number of original images, the standard deviation is calculated for each pixel using N-2 pieces of sampling data. In other words, when there is an odd number, the standard deviation is calculated for each pixel using N-1 combinations (NCN-1)) selected from N pixel values, and when there is an even number, the standard deviation is calculated for each pixel using N-2 combinations (NCN-2)) selected from N pixel values. Then, from the (NCN-1)) or (NCN-2)) standard deviations obtained for each pixel, the maximum standard deviation is determined as the standard deviation for that pixel (maximum value processing).

[0160] The above process is shown in the flowchart of Fig. 23. The process of Fig. 23 shows a case where the number N of original images to be composited is an odd number, but the same applies when it is an even number.

[0161] In step S71, the original images (N images) to be synthesized are generated, and in step S72, the sum of squares of pixel values ​​is calculated for each pixel of the first original image, and then in step S73, the sum of pixel values ​​is calculated for each pixel. For the first image, the sum of squares and the sum calculation are the results for the first image only. In step S74, the squared values ​​of each pixel value of the first image are stored, and in step S75, each pixel value of the first image (original) is stored.

[0162] Next, in step S76, it is checked whether there is a next image, and if there is (YES in step S76), the process returns to step S72, where the pixel value of each pixel in the second image is squared and added to the squared value of each corresponding pixel value in the first image. Next, in step S73, each pixel value in the second image is added to each corresponding pixel value in the first image. Furthermore, in step S74, the squared value of each pixel value in the second image is stored, and in step S75, each pixel value (original) of the second image is stored.

[0163] This process is performed for N images in order, and the sum of the squares of each pixel value and the sum of each pixel value are calculated for each corresponding pixel in the N images. In addition, the squared value and pixel value (original) of each image value for each of the N images are stored.

[0164] When the above process for N images is completed (NO in step S76), in step S77, i is used as a variable and the squared value of each pixel in the first image (i=1) is subtracted from the sum of squares of the pixel values ​​of each pixel in all N images calculated in step S72, to calculate the sum of squares for each pixel in the N-1 images.

[0165] Next, in step S78, the pixel value of the first image is subtracted from the sum of the pixel values ​​of all images calculated in step S73 to calculate the sum for N-1 images. In step S79, the average of the sums for the N-1 images calculated in step S78 is calculated, and then in step S80, the square of the average of the sums is calculated.

[0166] Next, in step S81, the mean square is calculated, which is the average value of the square sums for the N-1 sheets calculated in step S77, and then in step S82, the variance is calculated from the formula {(mean square)-(square of the mean)}. Then, in step S83, the standard deviation, which is the square root of the variance, is calculated.

[0167] Next, in step S84, a maximization process is performed. Here, since only one standard deviation has been obtained for each pixel, this value becomes the maximum.

[0168] Next, in step S85, it is checked whether there is a next image to subtract, in other words whether i=N, and if there is a next image, in other words if i=N is not true (YES in step S85), the process returns to step S77, i=2 is set, the sum of squares of the pixel values ​​and pixel values ​​of the second image are subtracted, the standard deviation is calculated in the same manner, and maximization processing is performed in step S85. In the maximization processing, the standard deviation when the first image is subtracted is compared with the standard deviation when the second image is subtracted, and the larger standard deviation is adopted.

[0169] In this way, the square sums and pixel values ​​of each image are subtracted in turn from the first image to the Nth image until i=N is reached, and the standard deviation is calculated for each pixel, and the maximum standard deviation is adopted as the standard deviation for that pixel.

[0170] The standard deviation thus obtained is desirably normalized, and a composite image is created based on the result. Note that, when using variance or half-width as a statistical variation value, the calculation can be performed in the same manner.

[0171] In this embodiment, a predetermined number of images are excluded from the calculation in order from among the multiple images, and the statistical variation value of each pixel is calculated, so that the optimal sampling candidate can be easily selected. Moreover, the maximum of the calculated variation values ​​is adopted as the variation value for that pixel, so that a composite image with a higher S / N ratio can be created.

[0172] In this embodiment, a case has been described in which a plurality of images are acquired in one period of the light and dark pattern by the lighting unit 6 while the workpiece 1 is moved relatively to the lighting unit 6 and the camera at a predetermined speed.

[0173] However, it is also possible to obtain multiple images for one period of the lighting pattern by moving only the lighting unit 6 relative to the workpiece 1 and the camera 8, and then create a composite image based on these multiple images by calculating variations such as standard deviation. [Explanation of symbols]

[0174] 1 Work 2 Moving mechanism 3 Lighting Frame 4 Support stand 6 Lighting Unit 7 Camera Frame 8. Camera 21 Master PC 22 Defect detection PC 30 Temporary defect candidates 40 Estimated coordinates 221 Image Acquisition Unit 222 Temporary defect candidate extraction unit 223 Coordinate Estimation Unit 224 Defect candidate determination unit 225 Image Group Creation Department 226 Image Synthesis Unit 227 Defect Detection Section

Claims

1. an image acquisition means for acquiring a plurality of images of a measurement portion of a workpiece, the measurement portion being subjected to surface defect detection, while moving a light and dark pattern generated by an illumination device relative to the workpiece; A dividing means for dividing each image acquired by the image acquiring means into a plurality of regions; a position shift amount setting means for setting a position shift amount corresponding to a movement amount of the workpiece in consideration of a three-dimensional shape of the workpiece in each area divided by the dividing means; an image synthesis means for synthesizing images of corresponding areas in a state in which the corresponding areas of the subsequent image are shifted in sequence with respect to the areas of the previous image by the position shift amount set for each area by the position shift amount setting means, and for performing this synthesis for each area; a detection means for detecting defects based on the composite image for each region created by the image synthesis means; A workpiece surface defect detection device comprising:

2. An image acquisition means for acquiring a plurality of images of a measurement portion of a workpiece to be detected for surface defects while moving a light and dark pattern generated by an illumination device relative to the workpiece; A dividing means for dividing each image acquired by the image acquiring means into a plurality of regions; a position shift amount setting means for setting a position shift amount corresponding to a movement amount of the work for each area divided by the dividing means; an image synthesis means for synthesizing images of corresponding areas in a state in which the corresponding areas of the subsequent image are shifted in sequence with respect to the areas of the previous image by the position shift amount set for each area by the position shift amount setting means, and for performing this synthesis for each area; a detection means for detecting defects based on the composite image for each region created by the image synthesis means; Equipped with The position shift amount setting means sets a plurality of different position shift amount candidates for each of the divided regions, and when the image synthesis means applies each of the set position shift amount candidates to synthesize the image, it sets the position shift amount candidate with the maximum evaluation value as the position shift amount for that region.

3. The position shift amount setting means of the workpiece surface defect detection device described in claim 1 images the same portion of the workpiece or a support member that moves in the same way as the workpiece using multiple position-specific cameras lined up in the direction of movement of the workpiece, determines position information of the workpiece from the images, and calculates an average position shift amount for each region from the obtained position information of the workpiece, information on the arrangement of the cameras and the workpiece, and optical information from the cameras, and sets this as the position shift amount for each region.

4. The position shift amount setting means measures the same part of the workpiece or a support member that moves in the same way as the workpiece using a measurement system including a distance sensor, a speed sensor, or a vibration sensor, either alone or in combination with a plurality of these, to obtain position information of the workpiece, and calculates an average position shift amount for each area from the obtained workpiece position information, information on the arrangement of the camera and the workpiece, and optical information from the camera, to set this as the position shift amount for each area.The workpiece surface defect detection device described in claim 1.

5. 5. The workpiece surface defect detection device according to claim 1, wherein the composite image is at least one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

6. 6. The workpiece surface defect detection device according to claim 1, wherein the detection means performs defect discrimination based on discrimination criteria set according to the characteristics of the workpiece and the defect.

7. An illumination device that irradiates illumination light with a light and dark pattern onto a workpiece that is to be detected for surface defects; an imaging means for imaging the reflected light of the illumination light from the illumination device at the measurement target portion of the workpiece; A moving means for moving the light and dark pattern by the lighting device relative to the workpiece; a control means for controlling the imaging means so as to image a measurement portion of the workpiece while moving a light and dark pattern of the illumination device relative to the workpiece by the moving means; A workpiece surface defect detection device according to any one of claims 1 to 6, Equipped with A work surface inspection system, characterized in that the image acquisition means acquires a plurality of images of the measurement portion of the work from the imaging means.

8. an image acquiring step of acquiring a plurality of images of a measurement portion of a workpiece, the measurement portion being inspected for surface defects, while moving a light and dark pattern generated by an illumination device relative to the workpiece; a division step of dividing each image acquired by the image acquisition step into a plurality of regions; a position shift amount setting step of setting a position shift amount corresponding to a movement amount of the workpiece in consideration of a three-dimensional shape of the workpiece in each area divided by the dividing step; an image synthesis step of synthesizing images of corresponding regions in a state in which the corresponding regions of the subsequent image are sequentially shifted with respect to the regions of the previous image by the position shift amount set for each region in the position shift amount setting step, and performing this synthesis for each region; a detection step of detecting defects based on the composite image created by the image synthesis step; A method for detecting surface defects on a workpiece, comprising the steps of:

9. An image acquisition step of acquiring a plurality of images of a measurement portion of a workpiece to be detected for surface defects while moving a light and dark pattern generated by an illumination device relative to the workpiece; a division step of dividing each image acquired by the image acquisition step into a plurality of regions; a position shift amount setting step of setting a position shift amount corresponding to a movement amount of the work for each area divided by the dividing step; an image synthesis step of synthesizing images of corresponding regions in a state in which the corresponding regions of the subsequent image are sequentially shifted with respect to the regions of the previous image by the position shift amount set for each region in the position shift amount setting step, and performing this synthesis for each region; a detection step of detecting defects based on the composite image created by the image synthesis step; The surface defect detection device for the workpiece performs the above. This method for detecting surface defects on a workpiece is characterized in that, in the position shift amount setting step, a plurality of different position shift amount candidates are set for each of the divided regions, and in the image synthesis step, when the set position shift amount candidates are applied and the images are synthesized, the position shift amount candidate which has the maximum evaluation value is set as the position shift amount for that region.

10. The method for detecting surface defects of a workpiece as described in claim 8, wherein, in the position shift amount setting step, images of the same portion of the workpiece or a support member which moves in the same way as the workpiece are captured using multiple position-specific cameras arranged in the direction of movement of the workpiece, and position information of the workpiece is obtained from the images, and an average position shift amount for each region is calculated from the obtained position information of the workpiece, information on the arrangement of the cameras and the workpiece, and optical information from the cameras, and this is used as the position shift amount for each region.

11. The method for detecting surface defects of a workpiece as described in claim 8, wherein in the position shift amount setting step, the same portion of the workpiece or a support member which moves in the same manner as the workpiece is measured using a measurement system including a distance sensor, a speed sensor, or a vibration sensor, either alone or in combination with a plurality of these, to obtain position information of the workpiece, and an average position shift amount for each region is calculated from the obtained position information of the workpiece, position information of the camera and the workpiece, and optical information of the camera, to obtain the position shift amount for each region.

12. The method for detecting surface defects of a workpiece according to any one of claims 8 to 11, wherein the composite image is at least one of a standard deviation image, a phase image, a phase difference image, a maximum value image, a minimum value image, and an average value image.

13. 13. A method for detecting surface defects of a workpiece according to claim 8, wherein in the detection step, defect discrimination is performed based on discrimination criteria set according to characteristics of the workpiece and the defect.

14. A program for causing a computer to execute the method for detecting surface defects of a workpiece according to any one of claims 8 to 13.

Images