Image processing apparatus and image processing method

Abstract

To provide an image processing device and image processing method which can execute positioning of an image of a product applied with painting having an irregular pattern.SOLUTION: A light and dark pattern in which a light portion L and a dark portion D appear alternately and periodically in a plane vertical to the irradiation direction is irradiated to a painted inspection object P, the light and dark pattern is periodically varied on a surface S of the inspection object P, a plurality of images of the surface S of the inspection object P irradiated with the light and dark pattern are acquired at a predetermined time interval, a prescribed pattern is extracted from an irregular pattern of a coating film formed on the surface S of the inspection object P in the plurality of images, alignment of the plurality of images is performed by using the extracted prescribed pattern, and a composite image is generated by superimposing the plurality of aligned images.SELECTED DRAWING: Figure 10B

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and an image processing method for aligning images.

Background Art

[0002] For a product having a periodic pattern or texture on its surface, when inspecting for defects on the painted surface by comparing the image of the product with the image of a good sample, the first frequency characteristic obtained by frequency-converting the image of the product to be inspected and the second frequency characteristic obtained by frequency-converting the image of the good sample are calculated. From the difference between the first frequency characteristic and the second frequency characteristic, a correction value for removing the influence (noise) of brightness changes caused by scratches and foreign matters is determined, and the first frequency characteristic corrected by the correction value is inverse-converted to generate an image of the product with noise removed. It is known to perform alignment between the image of the product with noise removed and the image of the good sample using the image of the product and the image of the good sample (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, when an irregular (random) pattern is applied to the surface of the product, the frequency characteristics obtained from the image of the product to be inspected and the frequency characteristics obtained from the image of the good sample become irregular. Therefore, it is impossible to determine a correction value for noise removal from the difference between the two frequency characteristics, and there is a problem that alignment of the image of the product to be inspected cannot be performed.

[0005] The problem to be solved by the present invention is to provide an image processing apparatus and an image processing method capable of performing alignment of an image of a product having a painted irregular pattern.

Means for Solving the Problem

[0006] The present invention irradiates a painted product to be inspected with a light and dark pattern in which light and dark portions alternately and periodically appear in a plane perpendicular to the irradiation direction, and periodically changes the light and dark pattern on the surface of the product to be inspected. A plurality of images of the surface of the product to be inspected irradiated with the light and dark pattern are acquired at predetermined time intervals, and a predetermined pattern is extracted from the irregular pattern of the coating film formed on the surface of the product to be inspected from the plurality of images. Using the extracted predetermined pattern, alignment of the plurality of images is performed, and the aligned plurality of images are superimposed to generate a composite image, thereby solving the above problem.

Effect of the Invention

[0007] According to the present invention, alignment of an image of a product having a painted irregular pattern can be performed.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 9C

Figure 10A

Figure 10B

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 14

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of an image processing apparatus and an image processing method according to the present invention will be described with reference to the drawings.

[0010] [Configuration of Image Processing System] FIG. 1 is a block diagram showing an image processing system 1 according to an embodiment of the present invention. The image processing system 1 of the present embodiment is a device for highlighting and detecting defects on the surface of an inspection object. The inspection object is not particularly limited, and examples include vehicle parts such as bonnets, bumpers, fenders, doors, door mirrors, roof panels, windshields, rear glass, and front grilles, touch panels for smartphones and tablet terminals, exterior parts of home appliances such as refrigerators and washing machines, containers for cosmetics, and lenses.

[0011] In addition, the defects on the surface of the inspection object refer to scratches, foreign substances, etc. present on the surface of the inspection object. Examples of scratches include scratches generated when the inspection object comes into contact with other articles during transportation. On the other hand, an example of a foreign substance is the adhesive layer of the masking tape remaining on the surface of the inspection object when the masking tape attached to the inspection object is peeled off. Since the appearance of the inspection object is impaired by such scratches and foreign substances, it is preferable to use the image processing system 1 in the inspection process when painting the inspection object and the final quality inspection process of the inspection object to remove the inspection object with defects on the surface.

[0012] As shown in FIG. 1, the image processing system 1 includes an irradiation device 11, an imaging device 12, a conveyance device 13, and an image processing device 14. The devices constituting the image processing system 1 are connected to each other in a state where data can be exchanged with each other by known means such as wired or wireless LAN.

[0013] The irradiation device 11 is a device for irradiating the inspection object with a light and dark pattern for highlighting defects on the surface of the inspection object, and is, for example, a flat panel display such as a liquid crystal display and an organic EL display. The size of the display is, for example, 15 to 25 inches, and the aspect ratio of the screen is not particularly limited. In addition, the irradiation device 11 includes a control device 11a for controlling the irradiation of the irradiation device 11 as necessary.

[0014] In the light and dark pattern of this embodiment, in a plane perpendicular to the irradiation direction of the irradiation device 11, bright portions and dark portions appear alternately and periodically. In this embodiment, the bright portion refers to a portion with a higher brightness than the dark portion in the light and dark pattern, and the dark portion refers to a portion with a lower brightness than the bright portion in the light and dark pattern. Hereinafter, an example of the light and dark pattern will be shown with reference to FIGS. 2A to 2C.

[0015] FIGS. 2A to 2C are front views of the irradiation device 11, and a light and dark pattern is displayed on the screen 11b of the irradiation device 11. In the light and dark pattern shown in FIG. 2A, in the xy plane perpendicular to the z-axis direction which is the irradiation direction of the irradiation device 11, a rectangular bright portion L extending in the x-axis direction and a rectangular dark portion D extending in the same x-axis direction form a pair (that is, one cycle Cyc), and this is a continuous horizontal stripe pattern. In the light and dark pattern shown in FIG. 2B, in the xy plane perpendicular to the z-axis direction which is the irradiation direction of the irradiation device 11, a rectangular bright portion L extending in the y-axis direction and a rectangular dark portion D extending in the same y-axis direction form a pair (that is, one cycle Cyc), and this is a continuous vertical stripe pattern. The light and dark pattern irradiated by the irradiation device 11 may be any of these vertical stripe patterns, horizontal stripe patterns, and lattice patterns combining the vertical stripe pattern and the horizontal stripe pattern.

[0016] The brightness difference between the bright portion L and the dark portion D can be set to an appropriate value within a range that can emphasize the defects on the surface of the product to be inspected. That is, as long as the necessary brightness difference is ensured, it is not necessarily the case that the bright portion L is white and the dark portion D is black, and other colors may be used. Here, white is, for example, a color with a brightness of 90 to 100, and black is, for example, a color with a brightness of 0 to 10. Also, the shapes of the bright portion L and the dark portion D can be appropriately set within a range where a plurality of cycles of the light and dark pattern appear on the surface of the product to be inspected and the defects can be sufficiently emphasized. In order to reduce the variation in the brightness difference on the surface of the product to be inspected and improve the detection accuracy of the defects, it is preferable that the bright portion L and the dark portion D have the same shape.

[0017] Examples of other light and dark patterns include, for example, the striped pattern shown in FIG. 2C. FIG. 2C shows a rectangular bright portion L extending in a direction inclined by approximately 45° counterclockwise with respect to the x-axis and a rectangular dark portion D extending in a direction inclined by approximately 45° counterclockwise with respect to the x-axis in the xy plane perpendicular to the z-axis direction, which is the irradiation direction of the irradiation device 11. These form a pair (i.e., one cycle Cyc), and this is a continuous striped pattern. In addition, in the vertical stripe light and dark pattern shown in FIG. 2B, a pattern in which the shapes of the bright portion L and the dark portion D are changed from rectangular shapes to zigzag shapes, and in the light and dark patterns shown in FIGS. 2A and 2B, a pattern in which the shapes of the bright portion L and the dark portion D are changed from rectangular shapes to waveform shapes can be used.

[0018] Returning to FIG. 1, the imaging device 12 is a device for acquiring image data of the surface of the inspection object irradiated with the light and dark pattern by the irradiation device 11, and is, for example, a camera equipped with an imaging element such as a CCD. The number of pixels of the camera is, for example, 2 to 5 million pixels, the field of view is, for example, 60×60 to 200×200 mm, and the focal length is, for example, 10 to 30 mm. Also, the image captured by the imaging device 12 may be full-color or monochrome. Also, the frequency at which the imaging device 12 acquires image data can be set to an appropriate value according to the defect detection speed in the image processing system 1, and for example, one image is captured every 0.01 to 0.1 seconds. Note that the image processing system 1 shown in FIG. 1 includes one imaging device 12, but a plurality of imaging devices 12 may be provided. The acquired image data is acquired by the image processing device 14 at a predetermined time interval.

[0019] The conveying device 13 is a device for conveying the irradiation device 11 and the imaging device 12 so as to be in a predetermined posture with respect to the product to be inspected. The conveying device 13 is, for example, a robot arm as shown in FIG. 1, but does not necessarily need to have a gripping portion, and may be attached to the back side of the irradiation device 11 or may be integrated with the irradiation device 11. The conveying device 13 operates the movable part of the arm to move the irradiation device 11 and the imaging device 12 so that the imaging device 12 can acquire image data for the entire surface of the product to be inspected. At this time, the distance between the imaging device 12 and the product to be inspected is controlled so as to be within the range of the focal length of the imaging device 12. Further, the conveying device 13 includes a control device 13a for controlling the conveyance of the irradiation device 11 and the imaging device 12 as necessary.

[0020] In addition, in the present embodiment, the conveying device 13 conveys the irradiation device 11 and the imaging device 12. However, when the product to be inspected is relatively small, such as a vehicle emblem, and is easier to move compared to the irradiation device 11 and the imaging device 12, the conveying device 13 may convey the product to be inspected. In this case, the conveying device 13 is, for example, a table provided with a device for fixing the product to be inspected, and is configured to move the table using a rail or an endless track.

[0021] Next, the positional relationship among the irradiation device 11, the imaging device 12, and the product to be inspected will be described. FIG. 3 is a side view of the image processing system 1 showing the irradiation device 11, the imaging device 12, and a part of the conveying device 13. In the image processing system 1 shown in FIG. 3, the imaging device 12 is attached to the irradiation device 11 via a bracket B. Further, the conveying device 13 is attached to the back side of the irradiation device 11 so that the irradiation device 11 and the imaging device 12 can be moved.

[0022] In the image processing system 1, as shown in FIG. 3, a light and dark pattern is irradiated from the irradiation device 11 toward the article to be inspected P. The imaging device 12 detects the light reflected from the surface S of the article to be inspected P, and acquires image data of the light and dark pattern on the surface S of the article to be inspected P. Since the amount of reflected light is different between the defective part and the non-defective part, defects can be detected from the difference in the amount of reflection in the acquired image data. In the image processing system 1, in order to accurately calculate the difference in the amount of reflection, the mounting position of the imaging device 12 is set to a position where the reflected light of the light and dark pattern irradiated from the irradiation device 11 enters at a predetermined angle. For example, as shown in FIG. 3, the imaging device 12 is mounted at a position where the incident angle θa at which the light and dark pattern irradiated from the irradiation device 11 enters the surface S of the article to be inspected P and the reflection angle θb of the reflected light reflected from the surface S of the article to be inspected P toward the imaging device 12 are the same angle. At the mounting position shown in FIG. 3, the imaging device 12 can mainly detect the light that is specularly reflected from the surface S of the article to be inspected P among the light and dark patterns irradiated from the irradiation device 11.

[0023] Returning to FIG. 1, the image processing device 14 is a device for causing the devices included in the image processing system 1 to cooperate and control, and is, for example, a computer. The image processing device 14 has functions of irradiating the article to be inspected with a light and dark pattern, periodically changing the light and dark pattern, acquiring an image of the surface of the article to be inspected at predetermined time intervals, extracting a predetermined pattern from the irregular pattern of the coating film in the acquired image, performing image alignment using the extracted predetermined pattern, and generating a composite image by overlapping a plurality of images that have been aligned.

[0024] To implement these functions, as shown in FIG. 4, the image processing apparatus 14 includes a CPU (Central Processing Unit) 141 as a processor, a ROM (Read Only Memory) 142 storing programs, and a RAM (Random Access Memory) 143 functioning as an accessible storage device. The CPU 141 is an operation circuit for functioning as the image processing apparatus 14 by executing the programs stored in the ROM 142. Note that the image processing apparatus 14 does not necessarily need to be provided together with the irradiation device 11, the imaging device 12, and the conveyance device 13, and it may be provided in a server at a remote location away from these devices.

[0025] [Functions of the processing unit] The program used in the image processing apparatus 14 includes a processing unit 2 which is a functional block for implementing the functions of the image processing apparatus 14 described above. The processing unit 2 has a function of controlling the irradiation device 11, the imaging device 12, and the conveyance device 13 to acquire image data, and processing the acquired image data to detect defects. As shown in FIG. 4, the processing unit 2 includes an irradiation unit 21, an acquisition unit 22, an extraction unit 23, an alignment unit 24, and a generation unit 25. In FIG. 4, each unit is shown by being extracted for convenience.

[0026] The defects detected by the functions of the processing unit 2 are mainly coating defects on the surface of the painted inspection object. The color of the coating applied to the inspection object P is not particularly limited, but the coating includes a layer containing a brightening material. The brightening material is a material for imparting gloss such as metallic luster to the coating and making the coating have a shiny finish. For example, it includes metal particles or flakes such as aluminum and copper alloys, and fine particles of minerals such as silica. The average particle size of the particles is, for example, 0.1 to 30 μm, and in the case of flakes, the thickness is, for example, 0.01 to 1 μm. If the thickness of the flakes is less than 0.01 μm, the base layer can be seen through, and if it exceeds 1 μm, the flakes do not orient smoothly and particles can be seen floating on the surface. Also, if the average particle size is less than 0.1, the sense of gloss decreases, and if it exceeds 30 μm, the sense of gloss increases too much and the color shading difference decreases.

[0027] Specific examples of painting defects are shown in Fig. 5. Fig. 5 is a diagram showing the main painting defects occurring in the topcoat painting of an automobile body. The main defects occurring in the topcoat painting of an automobile body can be classified into defects with unevenness on the surface of the paint film and defects without unevenness on the surface of the paint film. Examples of defects with unevenness on the surface of the paint film include defects called "foreign matter (batsu)", defects called "pinholes", and defects called "rejection". Examples of defects without unevenness on the surface of the paint film include defects called "thinner stain" and defects called "water stain".

[0028] "Foreign matter (batsu)" is a defect in which dust attached to the body before applying the topcoat paint or dust attached to the body during the application of the topcoat paint remains on the undried paint film and appears convex. For example, as shown in the cross-sectional view of Fig. 5, when dust Z1 adheres to the surface S of the inspection item P, the painted surface around the area where the dust Z1 adheres swells convexly, resulting in the defect of "foreign matter (batsu)". "Pinhole" is a defect in which the solvent of the topcoat paint remaining on the bottom side of the paint film penetrates through the semi-cured paint film and evaporates, appearing as small holes. For example, as shown in the cross-sectional view of Fig. 5, when the solvent penetrates through the semi-cured paint film and evaporates, small holes Z2 are generated, resulting in "pinholes". "Rejection" is a defect in which an oil-repellent or water-repellent substance scatters on the undried paint film and appears in a crater shape. For example, as shown in the cross-sectional view of Fig. 5, when the solvent scatters on the surface of the inspection item P, crater-shaped recesses Z3 are generated, resulting in "rejection". Examples of images of each defect are shown in Fig. 5. The portions X1 to X3 surrounded by the dashed lines are the areas where each defect occurred.

[0029] "Thinner stain" refers to a defect where thinner (such as the solvent of the topcoat paint) remains locally inside the undried paint film when applying the topcoat paint, appearing as a stain. "Water stain" refers to a defect where water remains locally inside the undried paint film when applying the topcoat paint, appearing as a stain. "Thinner stain" and "water stain" appear as circles as shown in the plan view of Fig. 5, for example. Images of the surface S of the actual parts where these defects occurred are shown as X4 and X5 in Fig. 5. In addition, in a metallic topcoat paint mixed with fine aluminum particles called aluminum flakes, "metal spots" where the bright pigments are locally concentrated and the orientation of the bright pigments becomes non-uniform, and "penetration defects" where the paint film is scraped due to scratches or the like and the metal surface of the body is exposed are also cited as defects without unevenness on the surface of the paint film.

[0030] The defects shown in Fig. 5 may or may not be visible depending on the lighting conditions, and some may not be visible under white lighting. Therefore, when detecting using a device such as the image processing system 1, it is necessary to preset a lighting method that can detect the defects in advance, but this setting requires a considerable amount of trial and error. Also, when there are unevenness on the surface S of the inspection object P, such as a vehicle, it is difficult to determine whether the detected defect is due to the uneven shape.

[0031] To address these issues, in a device such as the image processing system 1, an illumination pattern that emphasizes the defects due to the brightness difference is irradiated, a plurality of images of the illumination pattern are acquired by the imaging device 12, and the acquired plurality of image data are superimposed to detect the defects. In this case, as shown in Fig. 3, when shooting while intermittently moving the irradiation device 11 and the imaging device 12 with the transport device 13, the transport device 13 that should be stationary during the shooting of the imaging device 12 may not be completely stationary, and the positions of the acquired images may shift. If the images with shifted positions are superimposed and synthesized, the resulting synthesized image will be blurred, and the defects cannot be accurately detected. At this time, it is conceivable to mark the inspection object P for alignment and align the positions of the images, but in the case of exterior parts, marking cannot be done.

[0032] Therefore, in the processing unit 2 of the present embodiment, a predetermined pattern is extracted from the irregular pattern of the coating film formed on the surface S of the inspection object P, and alignment is performed using the extracted pattern, so that alignment of a plurality of images can be performed without applying alignment markings to the product surface. Hereinafter, the functions performed by each functional block of the image processing apparatus 14 shown in FIG. 4 will be described.

[0033] The irradiation unit 21 has a function of irradiating the painted inspection object P with a light and dark pattern by the irradiation device 11 and periodically changing the light and dark pattern on the surface S of the inspection object P. Periodically changing the light and dark pattern means that at a certain point on the surface S of the inspection object P, the light and dark pattern irradiated by the irradiation device 11 changes from the bright part L to the dark part D and then to the bright part L again, or from the dark part D to the bright part L and then to the dark part D again. That is, at a certain point on the surface S of the inspection object P, a pattern of one cycle consisting of a pair of bright part L and dark part D is irradiated for at least one cycle. Preferably, the irradiation unit 21 irradiates the light and dark pattern for two or more cycles. By acquiring images of the light and dark pattern for a plurality of cycles, the boundary between the bright part L and the dark part D appears sharp, improving the defect detection accuracy.

[0034] A specific example of the periodic change of the light and dark pattern will be described with reference to FIGS. 6A to 6C, which are front views of the irradiation device 11. In FIG. 6A, on the screen 11b of the irradiation device 11, in the xy plane perpendicular to the z-axis direction, which is the irradiation direction of the irradiation device 11, a rectangular bright part extending in the x-axis direction and a rectangular dark part extending in the same x-axis direction form a pair (that is, one cycle Cyc), and a horizontal stripe pattern is irradiated. Here, it is assumed that the bright part and the dark part of the light and dark pattern shown in FIG. 6A are moved in the negative y-axis direction. Focusing on point A in FIG. 6A, in the state of FIG. 6A, the bright part L1 is irradiated at point A. When the bright part and the dark part are moved in the negative y-axis direction to the state of FIG. 6B, the dark part D1 is irradiated at point A. When the bright part and the dark part are further moved in the negative y-axis direction to the state of FIG. 6C, the bright part L2 is irradiated at point A. At this stage, for point A, a light and dark pattern of one cycle consisting of the bright part L1 and the dark part D1 has been irradiated.

[0035] In this way, the irradiation unit 21 moves the light and dark pattern by one cycle or more of the pair of light parts L and dark parts D in the direction in which the light parts L and the dark parts D are aligned. The direction in which the light parts L and the dark parts D are aligned is the y-axis direction for the light and dark patterns in FIGS. 2A and 6A, and the x-axis direction for the light and dark patterns in FIGS. 2B and 2D. Further, since the light and dark pattern in FIG. 2C is inclined by about 45° with respect to the x-axis, the axis for moving the light and dark pattern is a direction inclined by about 135° with respect to the x-axis. Note that the direction in which the light and dark pattern is moved may be either the positive direction or the negative direction of the axis, and it is not necessarily moved in one direction. That is, after moving the light and dark pattern in the positive direction of the axis, the light and dark pattern may be moved in the negative direction.

[0036] The speed at which the light and dark pattern is moved can be set to an appropriate value according to the frequency at which the imaging device 12 can acquire image data. As an example, when the imaging device 12 can capture one image every 0.01 to 0.1 seconds, if, for example, 10 images are captured in one cycle, the speed at which the light and dark pattern is moved is set to the speed at which the light and dark pattern moves one cycle in 0.1 to 1.0 seconds. That is, when the shooting frequency of the imaging device 12 is determined, the more images are captured in one cycle, the slower the speed at which the light and dark pattern is moved is set, and the fewer images are captured in one cycle, the faster the speed at which the light and dark pattern is moved is set. Such a setting is performed, for example, using the image processing device 14 and the control device 11a by the function of the irradiation unit 21. Since the light and dark pattern irradiated from the irradiation device 11 is, that is, the video of the light and dark pattern displayed on the display, the CPU 141, which is the processor of the image processing device 14, outputs an instruction to the control device 11a, and the control device 11a controls the reproduction of the video of the light and dark pattern. Specifically, the control device 11a adjusts the speed of the video to be reproduced, or reproduces or reversely reproduces the video.

[0037] Returning to FIG. 4, the acquisition unit 22 has a function of acquiring, at predetermined time intervals, an image of the surface S of the inspection object P irradiated with a light and dark pattern by the imaging device 12. The predetermined time interval is set to an appropriate value according to the frequency at which the imaging device 12 can acquire image data. For example, when the imaging device 12 can capture one image every 0.01 to 0.1 seconds, 0.01 to 0.1 seconds is set as the predetermined time interval. Further, when acquiring an image of the surface S of the inspection object P, the transport device 13 is operated by the function of the acquisition unit 22, and the irradiation device 11 and the imaging device 12 are moved so that an image of the entire surface S of the inspection object P can be acquired.

[0038] Regarding the conveyance of the irradiation device 11 and the imaging device 12 by the conveyance device 13, description will be made with reference to FIG. 7 which is a front view of the inspection object P. FIG. 7 shows the irradiation device 11, the imaging device 12 fixed to the irradiation device 11 by the bracket B, and the surface S of the inspection object P. A conveyance device 13 (not shown) is attached to the irradiation device 11. It is assumed that the irradiation device 11 and the imaging device 12 are in the position of the point Y1 in the upper right of FIG. 7 in the initial state and are capturing the range R1 of the surface S. In this case, in order to capture the entire surface S of the inspection object P, the irradiation device 11 and the imaging device 12 are moved along, for example, the path T using the conveyance device 13. Specifically, when the capture of the range R1 of the surface S is completed, it moves from the point Y1 to the point Y2 and captures the range R2 of the surface S. Next, when the capture of the range R2 of the surface S is completed, it moves from the point Y2 to the point Y3 and captures the range R3 of the surface S. Similarly, along the path T, the irradiation device 11 and the imaging device 12 are moved as Y3 → Y4 → Y5 → Y6 → Y7 → Y8, and images of the ranges R3 → R4 → R5 → R6 → R7 → R8 are sequentially acquired. Thereby, image data of the entire surface S of the inspection object P can be acquired by the imaging device 12.

[0039] Path T varies according to the size of the range within which the imaging device 12 can acquire image data, that is, the sizes of ranges R1 to R8. For example, when the imaging device 12 can capture the ranges of R1 and R2, path T becomes a path that moves the surface S of the inspection object P only in the vertical direction of the drawing. Such conveyance by the conveyance device 13 is performed using, for example, the image processing device 14 and the control device 13a. The CPU 141, which is a processor of the image processing device 14, outputs an instruction to the control device 13a, and the control device 13a controls the operation of the movable part of the conveyance device 13. For the control of the movable part of the conveyance device 13 by the control device 13a, for example, teaching is used. As an example, when moving the irradiation device 11 and the imaging device 12 along path T, a teach pendant is used to store in the control device 13a the operation of the movable part during the movement along path T and reproduce the stored movement.

[0040] An example of the image data acquired by the imaging device 12 by the function of the acquisition unit 22 is shown in FIG. 8. The image data shown in FIG. 8 is of a roof panel coated with a bright white paint. As the irradiation device 11, a flat panel display was used, and the light and dark pattern was a vertical stripe pattern as shown in FIG. 2B, which was changed at about 0.1 to 0.5 seconds per cycle. As the imaging device 12, a monochrome CCD camera with 4 million pixels (2048×2048) was used. The field of view of the camera is 200 mm×200 mm, but the image shown in FIG. 8 is a cut-out of the portion around the foreign object (object) indicated by the dashed black circle from the captured image data. The ten images from "1 / 10" to "10 / 10" shown in FIG. 8 are ten pieces of image data captured while the light and dark pattern changes by one cycle. Focusing on the portion of the foreign object (object) indicated by the dashed black circle, it can be seen that the irradiated light and dark pattern changes by one cycle as dark part D ("1 / 10" to "2 / 10") → bright part L ("3 / 10" to "7 / 10") → dark part D ("8 / 10" to "10 / 10"). Thus, by the function of the acquisition unit 22, at a predetermined time interval, for a certain point on the surface S of the inspection object P irradiated with the light and dark pattern, a plurality of images are acquired for at least one cycle of the light and dark pattern.

[0041] Returning to FIG. 4, the extraction unit 23 has a function of extracting a predetermined pattern from the irregular pattern of the coating film formed on the surface S of the inspection object P in a plurality of images acquired by the function of the acquisition unit 22. The irregular pattern of the coating film is, for example, a pattern formed by a brightening material contained in the coating. The brightening material is fine particles of a metal such as particles or scales, or minerals such as silica, and these particles and scales are oriented along the base material in the coating film. When the brightening material oriented along this base material is viewed from the surface S of the inspection object P, a pattern formed by irregularly arranged particles and scales can be seen. The extraction unit 23 extracts a pattern from the irregular pattern particularly in the portion irradiated with the bright part L of the light and dark pattern. This is because in the dark part D of the light and dark pattern, the irregular pattern cannot be detected well and pattern extraction may not be possible in some cases.

[0042] When extracting a predetermined pattern, edge extraction processing is performed on the acquired image. Specifically, by the function of the extraction unit 23, using the image processing software stored in the image processing device 14, filters such as a differential filter, a Prewitt filter, a Sobel filter, a second-order differential filter, a Laplacian filter, a LoG (Laplacian of Gaussian) filter, and a DoG (Difference of Gaussian) filter are used to extract the edge portion where the brightness changes suddenly in the image. The setting of the filter in the edge extraction process can be appropriately set within the range where the edge as the pattern used for alignment is appropriately extracted.

[0043] A specific example of pattern extraction will be described with reference to FIGS. 9A to 9C. FIG. 9A is an enlarged and cut-out portion of the "1 / 10" image in FIG. 8. First, by the function of the extraction unit 23, using the image processing apparatus 14, the brightness of the image is detected in the image shown in FIG. 9A, and the portion determined to be irradiated with the bright part L is extracted. The extracted portion is shown in FIG. 9B. Next, in the portion shown in FIG. 9B, a process of extracting edges is performed using, for example, a differential filter. The extracted edges are shown in FIG. 9C. The portion indicated by the fine line in FIG. 9C is the extracted edge and is the pattern used for alignment. In this way, the extraction unit 23 performs pattern extraction using a part (for example, the central part) of the image acquired by the function of the acquisition unit 22 for a plurality of images.

[0044] Returning to FIG. 4, the alignment unit 24 has a function of aligning a plurality of images using a predetermined pattern extracted by the extraction unit 23. For the alignment of a plurality of images, a matching process using the extracted pattern is performed. As an example of alignment, those using the pattern (extracted edge) shown in FIG. 9C are shown in FIGS. 10A to 10B. The image shown in FIG. 10A is an enlarged and cut-out portion of the "2 / 10" image in FIG. 8. In the image shown in FIG. 10A, similar to FIGS. 9B to 9C, the portion determined to be irradiated with the bright part L is extracted, and an edge extraction process is performed using a differential filter, whereby the pattern (extracted edge) shown in FIG. 10B is obtained. Since the images of FIG. 9A and FIG. 10A are the same except that the positions of the bright part L and the dark part D are slightly shifted, the pattern shown in FIG. 10B will be the same as or very similar to that shown in FIG. 9C.

[0045] Next, the alignment unit 24 performs pattern matching processing on the image shown in FIG. 10B using the pattern shown in FIG. 9C. Through pattern matching, it can be seen that the pattern shown in FIG. 9C matches the pattern shown in FIG. 10B, and at the same time, the difference (shift amount) between the position of the pattern shown in FIG. 9C and the position of the pattern shown in FIG. 10B can be calculated. This difference in position (shift amount) is also referred to as the movement amount of the drawing. As factors causing the movement amount (shift amount), in addition to the case where an image is taken by the imaging device 12 while moving a small inspection object P such as an emblem, it is also conceivable that another object touches the stationary transport device 13 while the imaging device 12 is taking an image. Then, the alignment unit 24 shifts the image in FIG. 10A by the calculated movement amount and overlaps it with the image in FIG. 9A. At this time, the pattern shown in FIG. 9C and the pattern shown in FIG. 10B, which are the reference for alignment, overlap each other.

[0046] The combination of images for alignment can be appropriately combined within the range where pattern matching can be used to perform image alignment appropriately, considering factors such as the frequency of taking images by the imaging device 12 and the speed of changing the light and dark pattern by the irradiation device 11. However, as shown in FIGS. 9C and 10B, when extracting a pattern from an irregular pattern in the portion irradiated with the bright part L of the light and dark pattern, it is preferable that a plurality of images for alignment are taken at close timings. This is because for images taken at close timings, the position of the portion irradiated with the bright part L where pattern extraction is performed does not change significantly, making it easier to perform matching.

[0047] For example, as shown in FIG. 11, when five time-series images are acquired while shifting the light and dark pattern one period in the negative x-axis direction, a predetermined pattern is extracted from each of the first to fifth images. In the case of FIG. 11, it is assumed that patterns N1 to N4 are extracted. Next, the pattern N1 of the first image and the pattern N1 of the second image are superimposed to calculate the displacement amount between the first image and the second image. The displacement amount is calculated, for example, in the x-axis direction and the y-axis direction. Then, alignment between the first image and the second image is performed using the calculated displacement amount (movement amount). Similarly, alignment between the second image and the third image is performed using pattern N2, alignment between the third image and the fourth image is performed using pattern N3, and alignment between the fourth image and the fifth image is performed using pattern N4. The alignment is repeated until alignment is completed for all images. When alignment is completed for all of the first to fifth images, a composite image in which the defect X is emphasized is generated by the function of the generation unit 25 described later.

[0048] In this way, alignment is performed between the image acquired at the nth (n is a natural number) time series and the image acquired at the (n + 1)th time series for the images acquired in time series. Alternatively, instead of this, alignment may be performed between the image acquired at the nth (n is a natural number) time series and the image acquired at the (n + 2)th time series for the images acquired in time series, or alignment may be performed between the image acquired at the nth (n is a natural number) time series, the image acquired at the (n + 1)th time series, and the image acquired at the (n + 2)th time series. Further, for example, when detecting a defect while transporting a painted part on a belt conveyor on an assembly line in an assembly factory and taking images while moving the inspection object P, when three or more images are acquired by the acquisition unit 22, the movement amount for moving the images so that the positions of the extracted patterns match is calculated by a plurality of different paths, and alignment of the plurality of images may be performed using the median value of the calculated plurality of movement amounts. As an example, FIG. 12 shows a method for calculating the movement amount when four images are acquired.

[0049] In FIG. 12, four time-series images are acquired, and it is assumed that the amount of movement is calculated by pattern matching between each image by the function of the alignment unit 24. The calculated amount of movement is M between the first image and the second image 12 , M between the first image and the third image 13 , M between the first image and the fourth image 14 , M between the second image and the third image 23 , M between the second image and the fourth image 24 , and M between the third image and the fourth image 34 . Here, the amount of movement between the first image and the second image is M directly calculated by pattern matching 12 In addition to, the difference between M 13 and M 23 , and the difference between M 14 and M 24 , can be calculated by three different paths. These M 12 , the difference between M 13 and M 23 , and the difference between M 14 and M 24 will be the same value, but if any pattern matching is not performed properly, the three values may not be the same. In this case, in this embodiment, alignment is performed using the median value of the calculated multiple amounts of movement. M 12 , and the difference between M 13 and M 23 , and the difference between M 14 and M 24 , in this case, the second highest value will be used.

[0050] The reason for using the median value instead of the average value is that considering the amount of movement that was not calculated properly will reduce the accuracy of image alignment. By using the median value, alignment can be performed excluding the amount of movement that was not calculated properly. Also, by calculating the amount of movement for alignment in a superimposed manner through multiple paths like this, even if an error occurs in some pattern matching, all the images can be properly superimposed.

[0051] In addition to the amount of movement between the first image and the second image, the amount of movement between the second image and the third image, and the amount of movement between the third image and the fourth image can also be calculated superimposedly. For the amount of movement between the second image and the third image, M 23 and M 13 and the difference between M 12 and M 24 and the difference between M 34 and M are compared. For the amount of movement between the third image and the fourth image, M 34 and M 14 and the difference between M 13 and M 24 and the difference between M 24 and M are compared.

[0052] Returning to FIG. 4, the generation unit 25 has a function of generating a composite image by superimposing a plurality of images that have been aligned by the function of the alignment unit 24. The composite image is an image in which the data of a plurality of image data are superimposed pixel by pixel and the defective portions are emphasized and displayed. As a method of obtaining such a composite image, for example, each image is processed to generate a "maximum brightness image" that takes the maximum brightness for each pixel and a "minimum brightness image" that takes the maximum brightness for each pixel, and a "difference image" is generated from the difference between the "maximum brightness image" and the "minimum brightness image" to generate a composite image in which the defects are emphasized.

[0053] In a portion where defects such as foreign objects (buts) or pinholes occur, the irradiated light and dark pattern is diffusely reflected, so the specularly reflected light is not detected. Therefore, when photographing with a monochrome camera, when the bright part L is irradiated, the brightness becomes lower than other parts, and when the dark part D is irradiated, the brightness becomes higher than other parts. Therefore, in the portion where the defect occurs, the difference between the maximum brightness and the minimum brightness becomes small, and in the "difference image", it is displayed in black as a portion with low brightness. On the other hand, in a portion where no defect occurs, diffuse reflection does not occur, so the brightness difference in the "difference image" is the same as the brightness difference between the bright part L and the dark part D, and in the "difference image", it is displayed in white as a portion with high brightness. When detecting a defect from the "difference image", by scanning the "difference image" and detecting the portion displayed in black, the portion where the defect occurs can be specified.

[0054] By the said method, the composite image generated by synthesizing the 10 images shown in FIG. 8 is shown in FIG. 13A. In the composite image shown in FIG. 13A, the boundary part is clear as a whole. In particular, in the part indicated by the black circle with a broken line in the center of FIG. 13A, the foreign object (but) displayed in black clearly stands out. The processing unit 2 measures the brightness of the composite image generated by the generation unit 25 using the image processing apparatus 14, and detects the portion displayed in black as a surface defect. On the other hand, FIG. 13B shows a composite image generated by synthesizing the 10 images shown in FIG. 8 by the image processing system according to the comparative example of the present invention. In the composite image shown in FIG. 13B, the boundary part is unclear as a whole. In particular, the foreign object (but) indicated by the black circle with a broken line in the center of FIG. 13B does not clearly stand out. In this image, it cannot be used for detecting surface defects using the image processing apparatus 14.

[0055] So far, the functions of the functional blocks have been described. Note that the image processing apparatus 14 shown in FIG. 4 includes all of the above functional blocks, but it is not necessary for a single image processing apparatus 14 to include all of the functional blocks. Some of the above functional blocks may be provided in other devices included in the image processing system 1 or in another information processing apparatus (not shown). For example, the irradiation unit 21 shown in FIG. 4 may be provided in the irradiation apparatus 11 shown in FIG. 1. In this case, the functions of the irradiation unit 21 are executed using the CPU, ROM, and RAM of the irradiation apparatus 11.

[0056] Also, it is not necessary to execute all of the processes of each functional block in a single device. The functions of each functional block may be realized across a plurality of devices connected in a state where data can be exchanged. For example, among the processes executed by the processing unit 2, some of the processes may be executed by the imaging apparatus 12 in FIG. 1, and the remaining processes may be executed by the image processing apparatus 14. In this case, some of the processes for realizing the functions of the processing unit 2 are performed using the CPU, ROM, and RAM of the imaging apparatus 12.

[0057] [Processing in the Image Processing System] With reference to FIG. 14, the procedure when the image processing apparatus 14 processes information will be described. FIG. 14 is an example of a flowchart showing the processing of information in the image processing system 1 of the present embodiment. The following-described processing is executed in the image processing apparatus 14 when a user of the image processing system 1 inputs an instruction to detect a defect on the surface S of the inspection object P.

[0058] First, in step S1, the movement of the transfer device 13 is set by the function of the acquisition unit 22. At this time, for example, using the image processing device 14 and the teach pendant, the movement along the path T is stored in the control device 13a. In the subsequent step S2, the surface S of the inspection object P is irradiated with a light and dark pattern by the function of the irradiation unit 21. In the subsequent step S3, by the function of the acquisition unit 22, using the imaging device 12, a plurality of images of the light and dark pattern on the surface S of the inspection object P are taken, and time-series image data is acquired. In the subsequent step S4, pattern extraction is performed on each of the acquired images by the function of the extraction unit 23. Then, in step S5, it is determined whether appropriate pattern extraction can be performed on each image. If appropriate pattern extraction cannot be performed, the process returns to step S3 to acquire images again. On the other hand, if appropriate pattern extraction can be performed, the process proceeds to step S6.

[0059] In step S6, by the function of the alignment unit 24, for the images acquired in time series, the movement amount between the nth (n is a natural number) acquired image and the (n + 1)th acquired image is calculated. In the subsequent step S7, the images are shifted by the calculated movement amount and superimposed. In step S8, it is determined whether the alignment of all the images is completed. If the alignment of all the images is not completed, the process returns to step S5 to repeat the alignment. On the other hand, if the alignment of all the images is completed, the process proceeds to step S9.

[0060] In step S9, by the function of the generation unit 25, the aligned multiple images are synthesized to generate a synthesized image. In the subsequent step S10, it is determined whether a synthesized image capable of detecting defects can be generated. If a synthesized image capable of detecting defects cannot be generated, the detection result is displayed on the display part of the image processing device 14. Then, the execution of the routine is stopped and the process ends. On the other hand, if a synthesized image capable of detecting defects can be generated, the process proceeds to step S11. In step S11, the defective part is detected from the brightness difference of the synthesized image by the function of the processing unit 2. In step S12, the detection result is displayed on the display part of the image processing device 14. Then, the execution of the routine is stopped and the process ends.

[0061] [Embodiments of the Present Invention] As described above, according to the present embodiment, the painted inspection object P is irradiated with a light and dark pattern in which bright portions L and dark portions D alternately and periodically appear in a plane perpendicular to the irradiation direction, and on the surface S of the inspection object P, an irradiation unit 21 that periodically changes the light and dark pattern, an acquisition unit 22 that acquires a plurality of images of the surface S of the inspection object P irradiated with the light and dark pattern at predetermined time intervals, an extraction unit 23 that extracts a predetermined pattern from the irregular pattern of the coating film formed on the surface S of the inspection object P from the plurality of images, an alignment unit 24 that aligns the plurality of images using the predetermined pattern, and a generation unit 25 that generates a composite image by overlapping the plurality of aligned images are provided. Thereby, alignment of images of products coated with an irregular pattern can be performed.

[0062] Further, according to the image processing apparatus 14 of the present embodiment, the irradiation unit 21 moves the light and dark pattern by one cycle or more of the pair of the bright portion L and the dark portion D in a direction in which the bright portion L and the dark portion D are aligned. Thereby, more accurate alignment can be performed.

[0063] Further, according to the image processing apparatus 14 of the present embodiment, the extraction unit 23 extracts the pattern from the portion irradiated with the bright portion L among the plurality of images. Thereby, pattern extraction can be surely performed.

[0064] Further, according to the image processing apparatus 14 of the present embodiment, when three or more images are acquired by the acquisition unit 22, the alignment unit 24 calculates the amount of movement when moving the plurality of images so that the positions of the patterns match, by a plurality of different paths, and aligns the plurality of images using the median value of the calculated amounts of movement. Thereby, even when an error occurs in some pattern matching, all the images can be appropriately overlapped.

[0065] Further, according to the image processing apparatus 14 of the present embodiment, the light and dark pattern is a vertical stripe pattern or a horizontal stripe pattern. Thereby, defects can be emphasized more.

[0066] Also, according to the image processing apparatus 14 of the present embodiment, the coating film includes a layer containing a brightening material. Thereby, pattern extraction can be surely performed.

[0067] Further, according to the present embodiment, in an image processing method of performing processing using a processor, the processor irradiates a coated inspection object P with a light and dark pattern in which bright portions L and dark portions D alternately and periodically appear in a plane perpendicular to the irradiation direction, and on the surface S of the inspection object P, the light and dark pattern is periodically changed, and a plurality of images of the surface of the inspection object irradiated with the light and dark pattern are acquired at predetermined time intervals. From the irregular patterns of the coating film formed on the surface S of the inspection object P in the plurality of images, a predetermined pattern is extracted, and using the predetermined pattern, the plurality of images are aligned, and the aligned plurality of images are overlapped to generate a composite image. An image processing method is provided. Thereby, alignment of images of products coated with an irregular pattern can be performed.

Explanation of Reference Numerals

[0068] 1... Image processing system 11... Irradiation device 11a... Control device 11b... Screen 12... Imaging device 13... Conveying device 13a... Control device 14... Image processing device 141... CPU (Processor) 142... ROM 143... RAM 2... Processing unit 21... Irradiation unit 22... Acquisition unit 23... Extraction unit 24... Alignment unit 25... Generation unit A... Point B… Bracket Cyc… 1 cycle D… Dark part L… Light part M 12 、M 13 、M 14 、M 23 、M 24 、M 34 … Movement amount N1, N2, N3, N4, N5… Pattern P… Product under inspection R1, R2, R3, R4, R5, R6, R7, R8… Range S… Surface of the product under inspection T… Path X, X1, X2, X3, X4, X5… Defect Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8… Point Z1… Dust Z2… Hole Z3… Crater-like concave part θa… Incident angle θb… Reflection angle

Claims

1. Irradiate a painted inspection object with a light and dark pattern in which light and dark parts alternately and periodically appear in a plane perpendicular to the irradiation direction, and on the surface of the inspection object, an irradiation unit that periodically changes the light and dark pattern, An acquisition unit that acquires a plurality of images of the surface of the inspection object irradiated with the light and dark pattern at predetermined time intervals, An extraction unit that extracts a predetermined pattern from an irregular pattern of a coating film formed on the surface of the inspection object in the plurality of images, An alignment unit that aligns the plurality of images using the predetermined pattern, An image processing apparatus comprising: a generation unit that generates a composite image by overlapping the plurality of aligned images.

2. The irradiation unit periodically changes the light and dark pattern by moving the light and dark pattern by one cycle or more of a pair of the light and dark parts in a direction in which the light and dark parts are aligned. The image processing apparatus according to claim 1.

3. The extraction unit extracts the pattern from a portion irradiated with the light part in the plurality of images. The image processing apparatus according to claim 1 or 2.

4. When three or more images are acquired by the acquisition unit, the alignment unit Calculates the amount of movement when moving the plurality of images so that the positions of the patterns match, by a plurality of different paths, The image processing apparatus according to any one of claims 1 to 3, wherein the plurality of images are aligned using the median value of the calculated amounts of movement.

5. The light and dark pattern is a vertical stripe pattern or a horizontal stripe pattern. The image processing apparatus according to any one of claims 1 to 4.

6. The coating film includes a layer containing a brightening material. The image processing apparatus according to any one of claims 1 to 5.

7. In an image processing method for performing processing using a processor, The processor Irradiates a painted inspection object with a light and dark pattern in which light and dark parts alternately and periodically appear in a plane perpendicular to the irradiation direction, and on the surface of the inspection object, periodically changes the light and dark pattern, Acquires a plurality of images of the surface of the inspection object irradiated with the light and dark pattern at predetermined time intervals, Extracts a predetermined pattern from an irregular pattern of a coating film formed on the surface of the inspection object in the plurality of images, Performs alignment of the plurality of images using the predetermined pattern, An image processing method for generating a composite image by overlapping a plurality of the images that have been aligned.

Images