Image processing system and program

Abstract

To reduce the time required for inspection based on captured image acquired by imaging objects.SOLUTION: A control device disclosed herein is configured to acquire a first signal indicating lighting timings of light sources synchronized with imaging, and output a second signal to lighting means, based on the first signal, for turning on multiple light sources at a first timing of the lighting timings of the light sources and turning on light sources different from the multiple light sources at a second timing.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technology for inspecting an object. Background Art

[0002] As an appearance inspection technology for industrial products, a technology for detecting irregularities on an inspection surface is known. Patent Document 1 discloses a technology for combining a plurality of images obtained by imaging an inspection object using the photometric stereo method to determine whether the inspection object is good or defective. Prior Art Documents Patent Documents

[0003] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2015-232480 Summary of the Invention Problem to be Solved by the Invention

[0004] Depending on the type of industrial product, the number of units produced per unit time is large, and the time available for a single inspection may be short. When a plurality of light sources are turned on one by one, imaging takes time corresponding to the number of light sources, and the inspection processing time also increases in accordance with the number of captured images.

[0005] Accordingly, an object of the present invention is to shorten the time required for inspection in inspection based on a captured image obtained by imaging an object. Means for Solving the Problem

[0006] In order to solve the above problem, according to the present invention,[REDACTED] Image processing system comprises: An image processing system, wherein the image processing system controls an imaging device an imaging means for imaging an object; and a plurality of light sources dispersedly arranged above the object have ed illumination Lighting means for controlling the illumination of the device and The system comprises control means for controlling the imaging means and the illumination means, and image processing means for processing the image captured by the imaging device, wherein the control means is said imaging DeviceAn acquisition means for acquiring a first signal indicating the timing of lighting up the plurality of light sources in synchronization with imaging by the camera, and an output means for outputting a second signal to the illumination means, based on the first signal, for lighting up the plurality of light sources at a first timing and lighting up one light source different from the plurality of light sources at a second timing different from the first timing. Having , The aforementioned image processing means is The device is characterized by comprising: a gloss inspection means for inspecting the gloss of an object from an image obtained by imaging an object that has been illuminated with light from the plurality of light sources that have been lit at the first timing, and a color and shape inspection means for inspecting at least one of the color and shape of an object from an image obtained by imaging an object that has been illuminated with light from one of the light sources that has been lit at the second timing, using the imaging means. [Effects of the Invention]

[0007] According to the present invention, the time required for inspection based on an image obtained by imaging an object can be reduced. [Brief explanation of the drawing]

[0008] [Figure 1] Diagram showing the appearance and hardware configuration of the visual inspection system. [Figure 2] Diagram showing the functional configuration of the visual inspection system. [Figure 3] Diagram showing the arrangement of light sources in a lighting device. [Figure 4] A flowchart showing the process for controlling the illumination of a light source. [Figure 5] A diagram showing the correspondence between the synchronization signal and the timing of the light source illumination. [Figure 6] Diagram showing an example of a user interface [Figure 7] Flowchart illustrating the processing in an image processing system [Figure 8] Diagram showing the hardware configuration of the visual inspection system. [Figure 9] Diagram showing the functional configuration of the visual inspection system. [Figure 10] A diagram showing the correspondence between the synchronization signal and the timing of the light source illumination. [Modes for carrying out the invention]

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all combinations of features described in each embodiment are essential to the solution of the present invention.

[0010] [First Embodiment] <Appearance and Hardware Configuration of the Visual Inspection System> Figure 1(a) is a diagram showing an example of the hardware configuration of the visual inspection system in this embodiment. Figure 1(b) is a front view of the entire visual inspection system, and Figure 1(c) is a top view of the entire visual inspection system. The visual inspection system in this embodiment includes an image processing system 1, a start signal output interface 101, a transport control device 111, and a transport device 112.

[0011] The image processing system 1 includes an imaging control device 102, an imaging device 103, an image processing device 104, a display 105, a mouse 106, a keyboard 107, and a lighting device 108. The image processing system 1 is connected to a transport control device 111 that controls a transport device 112. The transport control device 111 uses the transport device 112 to transport the object to be inspected 113 to the image processing system 1 and sends an inspection start signal to the image processing system 1 via a start signal output interface 101.

[0012] The imaging control apparatus 102 includes a control unit 114, and controls the imaging apparatus 103 and the illumination apparatus 108 to capture an image of the object 113 in synchronization with the lighting of a light source. Specifically, when receiving an inspection start signal from a start signal input interface 115, the imaging control apparatus 102 transmits an imaging instruction to the imaging apparatus 103 via a release signal output interface 116. The imaging control apparatus 102 also receives, via a sync signal input interface 117, a sync signal transmitted from the imaging apparatus 103 for notifying an external strobe light source of lighting timing in synchronization with imaging. Further, according to the received sync signal, the imaging control apparatus 102 turns on the light sources of the illumination apparatus 108 in a predetermined order and combination via a lighting signal output interface 119. Through the above operation, the object 113 irradiated with light by predetermined illumination can be imaged. The imaging control apparatus 102 is connected to the image processing apparatus 104 via a USB interface 118, receives commands from the image processing apparatus 104, and provides information indicating the state of the imaging control apparatus 102 to the image processing apparatus 104.

[0013] The imaging apparatus 103 includes a control unit 125 and an imaging optical system 121 including a lens, an image sensor, and the like. The imaging apparatus 103 generates a captured image by quantizing an optical image obtained through imaging based on an imaging instruction received via a release signal input interface 120 by an image processing engine 123. The imaging apparatus 103 transfers the generated captured image to the image processing apparatus 104 via a USB interface 124. In the present embodiment, an example is described in which a still image captured using a digital camera is acquired and used, but a predetermined frame may be extracted as a still image from a moving image captured using a video camera and used. The imaging apparatus 103 transmits a sync signal to the imaging control apparatus 102 via a sync signal output interface 122.

[0014] The image processing apparatus 104 includes a RAM 126, a ROM 127, a CPU 128, a GPU 129, and a USB interface 130. Each component is connected via an internal bus. The processing shown in the flowchart described later is stored as program code in the ROM 127. This program code is expanded in the RAM 126 and executed by the CPU 128 and the GPU 129.

[0015] The illumination device 108 includes a plurality of light sources 109. In the present embodiment, the light sources 109 are LEDs, but other light sources such as xenon lamps may also be used. FIG. 3 shows an example of the arrangement of the plurality of light sources 109. FIG. 3(a) is a front view of the illumination device 108, and FIG. 3(b) is a top view of the illumination device 108. The light sources 109 shown as squares are dispersedly arranged in a hemispherical shape above the object 113, and differ from each other in at least one of a zenith angle and an azimuth angle. Note that the number, arrangement, and the like of the light sources are not limited to the above example. In inspection using a captured image, it is necessary to change the light irradiation method according to the visual inspection items. For example, when inspecting the gloss of an object, it is necessary to irradiate light from a direction in which the specularly reflected light from the inspection surface can be imaged. In addition, when inspecting the color or surface shape of an object, it is necessary to irradiate light from a direction that does not capture specularly reflected light from the inspection surface. Therefore, some of the light sources 109 are arranged in a direction with a large incident angle with respect to the installation surface of the object 113, which enables imaging under geometric conditions where the imaging device 103 easily receives diffusely reflected light. In addition, some of the light sources 109 are arranged in a direction with a small incident angle with respect to the installation surface of the object 113, which enables imaging under geometric conditions where the imaging device 103 easily receives specularly reflected light. Note that the light emitting surfaces and spectral characteristics of the light sources 109 may be different from each other. For example, a spot irradiation type illumination (a light source shown by a white square in FIG. 3) may be installed in a direction with a large incident angle with respect to the installation surface of the object 113, and an illumination in which light emitting elements are arranged in an annular shape (a light source shown by a gray square in FIG. 3) may be installed in a direction with a small incident angle. Note that an annular ring illumination may also be used in a direction with a small incident angle. The illumination device 108 turns on a predetermined light source among the light sources 109 for a preset time in accordance with an instruction from the control unit 114.

[0016] In this embodiment of the image processing system 1, the imaging control device 102, imaging device 103, image processing device 104, and illumination device 108 are separate devices, but multiple devices may be integrated into one unit.

[0017] <Processing performed by the image processing system> Figure 2 shows an example of the functional configuration of the visual inspection system in this embodiment. The control unit 114 of the imaging control device 102 has an imaging control unit 202. The imaging control unit 202 has a release signal output unit 207, a sync signal input unit 208, a sync signal count unit 209, and a lighting signal output unit 210. The control unit 125 of the imaging device 103 has an imaging unit 203. The imaging unit 203 has a release signal input unit 211, a control unit 212, a sync signal output unit 213, and an image acquisition unit 214. The image processing device 104 has an image processing unit 204. The image processing unit 204 has an inspection image acquisition unit 215, a color and shape inspection unit 216, a gloss inspection unit 217, and an output unit 218. The transport control device 111 has a start signal output unit 201. The transport device 112 has a transport unit 206. The illumination device 108 has an illumination unit 205.

[0018] Figure 7 is a flowchart of the process performed by the image processing system 1 in this embodiment. When the object 113 is transported to a predetermined position by the transport device 112, the start signal output unit 201 of the transport control device 111 sends an inspection start signal to the imaging control unit 202. The process in Figure 7 begins when the imaging control unit 202 receives the inspection start signal. Hereafter, each step (process) is represented by adding an S before the reference numeral.

[0019] In S701, the imaging control unit 202 and the imaging unit 203 acquire multiple images obtained by illuminating the object 113 to be inspected with a predetermined light source and transfer them to the image processing unit 204. Specifically, first, the release signal output unit 207, which receives the inspection start signal, sends a release signal to the release signal input unit 211. When the control unit 212 detects that the release signal input unit 211 has received the release signal, it executes the imaging operation. In this embodiment, high-speed continuous imaging is performed using a known continuous imaging function. The continuous imaging function is a function that repeats imaging at a predetermined speed as long as the release signal continues to be sent, and in this embodiment, imaging is performed at 30 frames per second. When imaging is performed, the control unit 212 causes the sync signal output unit 213 to output a sync signal in order to synchronize the timing of illumination of the external strobe light source with the timing of shutter curtain opening. When a sync signal is input by the sync signal input unit 208, the sync signal count unit 209 counts the number of sync signal inputs, i.e., the number of images captured. The lighting signal output unit 210 switches the light sources to be lit sequentially according to the number of captured images and outputs a lighting signal to the illumination unit 205. Details of the process by which the lighting signal output unit 210 controls the lighting of the light sources will be described later. The image acquisition unit 214 repeatedly transfers the captured images obtained by litting predetermined light sources and imaging the object 113 to be inspected to the inspection image acquisition unit 215 as needed. Through the above process, in S701, the imaging control unit 202 and the imaging unit 203 can transfer multiple captured images to the image processing unit 204.

[0020] In S702, the color and shape inspection unit 216 and the gloss inspection unit 217 perform inspection processing based on multiple captured images acquired by the inspection image acquisition unit 215. The color and shape inspection unit 216 detects defects by performing spatial filtering on an inspection image containing normal information and color information obtained by synthesizing captured images using the photometric stereo method. The abnormality score is calculated by integrating the reaction values ​​to the spatial filter, and the pass / fail status of the inspection is determined by comparing the abnormality score with a threshold for judgment. The calculated abnormality score is displayed in the abnormality score display area 609 of the inspection screen user interface (UI) shown in Figure 6. The threshold for judgment can be set in the threshold setting area 603 of the inspection screen UI shown in Figure 6. The gloss inspection unit 217 detects defects by performing spatial filtering in the same manner as above, using an captured image obtained by receiving specularly reflected light on the inspection surface of an object as an inspection image containing gloss information. The photometric stereo method is a technique that can acquire normal information representing the surface shape of an object and color information representing the color of an object by synthesizing captured images corresponding to multiple illumination directions. Please note that the above inspection method is just one example, and other inspection methods may be used.

[0021] In this embodiment, the visual inspection items are of three types: color, surface shape, and gloss. However, the visual inspection items are not limited to the above example; any item that represents the appearance and can be captured and identified is acceptable. For example, material or pattern may also be used. The inspection image acquisition unit 215 displays the inspection screen UI shown in Figure 6 on the display 105 and accepts instructions from the user. One or more visual inspection items are set in the inspection screen UI shown in Figure 6. The user can input information into the inspection screen UI displayed on the display 105 using the mouse 106 or keyboard 107. When a product to be inspected is selected from a dropdown menu in the product selection area 601 from among several pre-registered products, the visual inspection items corresponding to the selected product to be inspected are set in the item setting area 602. The user can also change the visual inspection items by inputting into the radio buttons in the item setting area 602.

[0022] The inspection screen UI shown in Figure 6 has an inspection start button 604 for instructing the start of the inspection and an inspection stop button 605 for instructing the stop of the inspection. The inspection screen UI also has an inspection date area 606 for displaying the inspection date and an inspection time area 607 for displaying the inspection time.

[0023] In step S703, the output unit 218 displays the results of the inspection process on the display 105. For example, in the inspection screen UI shown in Figure 6, the judgment result area 608 displays "OK" if the inspection is passed and "NG" if the inspection is failed. The abnormal event display area 610 displays that an abnormal event has occurred, such as an imaging failure. Furthermore, the output unit 218 outputs instructions to the transport unit 206 for transporting the passed and failed products to the next process.

[0024] <Process to control the illumination of the light source> Figure 4 is a flowchart of the process by which the lighting signal output unit 210 controls the lighting of the light sources. In S401, the lighting signal output unit 210 sets the light source lighting counter to zero. This light source lighting counter counts the number of times the light source is lit, and it corresponds to the number of times the imaging device 103 images a single object. In S402, the lighting signal output unit 210 sets the light source ID, which is the identification number of each light source 109, to zero. The light source ID corresponds to the number assigned to each light source in Figure 3.

[0025] In S403, the lighting signal output unit 210 acquires the synchronization signal input by the synchronization signal input unit 208. In S404, the lighting signal output unit 210 determines whether the current light source ID is a light source ID that has been pre-set to be lit simultaneously. If it is a light source ID that is lit simultaneously, the process proceeds to S405; otherwise, the process proceeds to S408. In this embodiment, it is assumed that light sources with light source IDs from 0 to 7 are pre-set as light sources that are lit simultaneously, but the IDs of the light sources that are lit simultaneously are not limited to these. The IDs of the light sources that are lit simultaneously may be set by the user via the UI displayed on the display 105, or fixed IDs may be set.

[0026] In S405, the lighting signal output unit 210 determines whether or not it has performed a determination for all light source IDs set for simultaneously lit light sources. If it has not performed a determination for all simultaneously lit light source IDs, it increments the light source ID in S410 and proceeds to S404. Steps S404, S405, and S410 are repeated, and in S405, if the lighting signal output unit 210 determines that it has performed a determination for all simultaneously lit light source IDs, it proceeds to S406. In S406, the lighting signal output unit 210 outputs a lighting signal to the lighting unit 205 to light up the light sources set for simultaneously lit light sources. Upon receiving the lighting signal, the lighting unit 205 lights up the simultaneously lit light sources. In this embodiment, the lighting unit 205 lights up the light sources with light source IDs from 0 to 7.

[0027] In S404, if it is determined that the current light source ID is not a simultaneously lit light source ID, in S408, the lighting signal output unit 210 outputs a lighting signal to the illumination unit 205 to light up the light source with the ID that matches the current light source ID. Upon receiving the lighting signal, the illumination unit 205 lights up the light source with the ID that matches the current light source ID. In S409, the lighting signal output unit 210 increments the light source ID. In S407, the lighting signal output unit 210 increments the light source lighting counter because one of the light sources was lit in S406 or S409. In S411, the lighting signal output unit 210 determines whether all light sources have been lit. If not, it returns to S403; if all have been lit, it terminates the process shown in Figure 4.

[0028] As described above, the light source is turned on for each object 113 under inspection for a predetermined number of images. Figure 5 shows the correspondence between the synchronization signal acquired by the lighting signal output unit 210 and the lighting signal generated based on the predetermined lighting time. The lighting signal for each light source is generated by the flowchart shown in Figure 4, and it can be seen that 8 of the 24 light sources are turned on simultaneously, resulting in a total of 17 turns on, i.e., 17 images, to complete the imaging of one object under inspection. Note that the method of simultaneously turning on the light sources is not limited to the example described above. For example, if there are no plans to change the simultaneously lit light sources, the lighting device 108 may have light sources with light source IDs 0 to 7 electrically connected, and the lighting device 108 may have one signal terminal that can input a signal to this group of light sources. Also, if a light source with high power consumption is connected, the overall electrical capacity of the lighting device 108 will need to be increased. In addition, the brightness of simultaneously lit light sources will be high, which may cause the pixel values ​​of the captured image to saturate. For this reason, light sources with relatively low power consumption should be used for simultaneously lit light sources.

[0029] In this embodiment, if the visual inspection item is color, the color information obtained by combining 16 captured images taken in synchronization with the illumination of light sources 8 to 23, which have a relatively large zenith angle, becomes the inspection image. If the visual inspection item is color only, a single image taken in synchronization with the illumination of any one of the light sources 8 to 23, which has little effect of specular reflection, may be used as the inspection image. Alternatively, for example, the pixel values ​​of four images taken in synchronization with the illumination of light sources 8 to 11 may be averaged, and the resulting average image may be used as the inspection image. Furthermore, similar to the example of light sources 0 to 7 described above, it is also possible to use a single image obtained by simultaneously illuminating light sources 8 to 11 and taking images as the inspection image.

[0030] In this embodiment, when the visual inspection item is surface shape, the normal information obtained by combining 16 captured images taken in synchronization with the illumination of light sources 8 to 23, which have relatively large zenith angles, becomes the inspection image. In this embodiment, when the visual inspection item is gloss, the captured image obtained by capturing the light specularly reflected on the inspection surface becomes the inspection image. Specifically, the single captured image taken in synchronization with the illumination of light sources 0 to 7, which have relatively small zenith angles, is the inspection image for gloss inspection.

[0031] Furthermore, the color and shape inspection unit 216 and the gloss inspection unit 217 are capable of extracting and inspecting a predetermined area on the inspection surface of the object 113. For example, an area pre-set by the user can be extracted as the inspection target area. Depending on the three-dimensional shape of the industrial product, in the image captured corresponding to light irradiation from a light source with a relatively large zenith angle, a shadowed area may occur in the inspection target area. In this case, by using an image captured corresponding to light irradiation from a light source with a relatively small zenith angle, the inspection target area can be extracted under conditions where the occurrence of shadowed areas is suppressed, thereby improving inspection accuracy.

[0032] Furthermore, the inspection screen UI shown in Figure 6 may include an area where the user can select the light source ID to be illuminated. In this case, a warning may be displayed if the light source ID selected by the user is not suitable for the color, surface shape, and gloss selected in the item setting area 602. For inspection objects with relatively matte surfaces, it is also possible to capture images for gloss inspection using a light source with a large zenith angle, and the user can select to simultaneously illuminate light sources with light source IDs 8 to 11 for gloss inspection.

[0033] [Second Embodiment] In the first embodiment, simultaneous illumination and individual sequential illumination of light sources were controlled based on a sync signal output from the imaging device 103. In this embodiment, synchronized imaging is performed by simultaneously issuing an imaging instruction to the imaging device 103 and an illumination instruction to the lighting device 108 using a pulse signal output from the control unit 114. The following will mainly describe the differences between this embodiment and the first embodiment. Components identical to those in the first embodiment will be denoted by the same reference numerals.

[0034] Figure 8 shows an example of the hardware configuration of the visual inspection system in this embodiment. In the image processing system 1 in this embodiment, the imaging device 103 does not have a sync signal output interface 122, and the imaging control device 102 does not have a sync signal input interface 117. The other hardware configurations are the same as in the first embodiment.

[0035] <Processing performed by the image processing system> Figure 9 shows an example of the functional configuration of the visual inspection system in this embodiment. In the image processing system 1 in this embodiment, the imaging unit 203 does not have a synchronization signal output unit 213, and the imaging control unit 202 does not have a synchronization signal input unit 208 and a synchronization signal counting unit 209. The other functional configurations are the same as in the first embodiment.

[0036] In this embodiment, the processing performed by the image processing system 1 will also be explained using the flowchart in Figure 7. In S701, the imaging control unit 202 and the imaging unit 203 acquire multiple images obtained by illuminating the object 113 to be inspected with a predetermined light source and transfer them to the image processing unit 204. Specifically, first, the control unit 114, upon receiving the inspection start signal, transmits a pulse signal not only to the release signal output unit 207 but also to the illumination signal output unit 210. Figure 10 shows an example of a pulse signal transmitted by the control unit 114. The pulse signal enclosed by the thick line in Figure 10 is a preset signal waveform. At the timing when the start signal output unit 201 notifies the start of the inspection, the imaging signal enclosed by this thick line is transmitted from the release signal output unit 207 to the release signal input unit 211 of the imaging unit 203, and imaging is performed by the imaging unit 203. At the same time, the light source activation signal for the area enclosed by the thick line is transmitted from the activation signal output unit 210 to the illumination unit 205, and each of the light sources 109 starts to light up. This method enables imaging synchronized with the activation of each light source, and as shown in Figure 10, simultaneous activation can be controlled by sending the same waveform signal to the light sources of the IDs to be activated simultaneously.

[0037] As described above, the visual inspection system of the embodiment described above allows for simultaneous inspection of the color, surface shape, and gloss of the object being inspected. Furthermore, by illuminating multiple light sources of the lighting device 108 at approximately the same timing during a single imaging cycle, the number of images can be reduced, thereby shortening the imaging time and the time required for inspection processing.

[0038] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of Symbols]

[0039] 102 Imaging control device 202 Imaging Control Unit 210 Lighting signal output section

Claims

1. An image processing system, The aforementioned image processing system is An imaging means that controls an imaging device to image an object, A lighting means for controlling the illumination of a lighting device having multiple light sources distributed above the object, Control means for controlling the imaging means and the illumination means, Image processing means for processing the captured image obtained by the aforementioned imaging device It has, The control means is An acquisition means for acquiring a first signal indicating the timing of lighting up the plurality of light sources synchronized with imaging by the imaging device, Based on the first signal, an output means outputs a second signal to the lighting means, which causes multiple light sources to be lit at a first timing among the lighting timings of the multiple light sources, and one light source different from the multiple light sources to be lit at a second timing different from the first timing. It has, The aforementioned image processing means is A gloss inspection means for inspecting the gloss of an object from an image obtained by imaging the object, which has been illuminated by the plurality of light sources that were turned on at the first timing, using the imaging means, A color and shape inspection means for inspecting at least one of the color and shape of an object from an image obtained by imaging the object, which is illuminated by one of the light sources that was turned on at the second timing, using the imaging means. An image processing system characterized by having the following features.

2. The light source of the aforementioned lighting device is installed in a hemispherical arrangement above the object. The image processing system according to claim 1, characterized in that the plurality of light sources that are lit at the first timing have a smaller zenith angle than one of the light sources that is lit at the second timing.

3. The image processing system according to claim 1, characterized in that the plurality of light sources of the lighting device, which are lit at the first timing, are selected based on user instructions.

4. The multiple light sources of the lighting device, which light up at the first timing, are electrically connected. The image processing system according to claim 1, characterized in that the lighting device has one signal terminal capable of inputting signals to the plurality of light sources.

5. The image processing system according to claim 1, characterized in that the plurality of light sources of the lighting device that are lit at the first timing are light sources that consume less power than one of the light sources that are lit at the second timing.

6. A program for causing a computer to function as an image processing system according to any one of claims 1 to 5.

Images