Control device, control method, and program

Abstract

To reduce the time required for inspection based on captured image acquired by imaging objects.SOLUTION: A control device disclosed herein is configured to acquire a first signal indicating lighting timings of light sources synchronized with imaging, and output a second signal to lighting means, based on the first signal, for turning on multiple light sources at a first timing of the lighting timings of the light sources and turning on light sources different from the multiple light sources at a second timing.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to techniques for inspecting objects. [Background technology]

[0002] A technique for detecting unevenness on an inspection surface is known as an appearance inspection technique for industrial products. Patent Document 1 discloses a technique for determining whether an inspection object is good or bad by combining multiple images obtained by capturing images of the inspection object using a photometric stereo method. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-232480 Summary of the Invention [Problem to be solved by the invention]

[0004] Depending on the type of industrial product, there may be a large number of products produced per unit time, meaning that the time available for each inspection is short. If multiple light sources are turned on one by one, the image capturing time increases in proportion to the number of light sources, and the inspection processing time also increases according to the number of captured images.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to reduce the time required for inspection based on captured images obtained by capturing an image of an object. [Means for solving the problem]

[0006] In order to solve the above problem, the control device according to the present invention is characterized by having an acquisition means for acquiring a first signal indicating the lighting timing of a light source synchronized with imaging, and an output means for outputting a second signal to a lighting means based on the first signal, the second signal being used to light a plurality of light sources at a first timing and to light a light source different from the plurality of light sources at a second timing among the lighting timings of the light sources. [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the time required for inspection based on an image obtained by capturing an image of an object. [Brief explanation of the drawings]

[0008] [Figure 1] A diagram showing the appearance and hardware configuration of the visual inspection system. [Figure 2] Diagram showing the functional configuration of the visual inspection system [Figure 3] A diagram showing the arrangement of light sources in the lighting device. [Figure 4] 10 is a flowchart showing a process for controlling the lighting of a light source. [Figure 5] A diagram showing the correspondence between the synchronization signal and the timing of lighting the light source. [Figure 6] A diagram showing an example of a user interface [Figure 7] 1 is a flowchart showing processing in an image processing system; [Figure 8] Hardware configuration of the visual inspection system [Figure 9] Diagram showing the functional configuration of the visual inspection system [Figure 10] A diagram showing the correspondence between the synchronization signal and the timing of lighting the light source. DETAILED DESCRIPTION OF THE INVENTION

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all of the combinations of features described in each embodiment are necessarily essential to the solution of the present invention.

[0010] [First embodiment] <Appearance and hardware configuration of the visual inspection system> Fig. 1(a) is a diagram showing an example of the hardware configuration of a visual inspection system according to this embodiment. Fig. 1(b) is a front view of the overall appearance of the visual inspection system, and Fig. 1(c) is a top view of the overall appearance of the visual inspection system. The visual inspection system according to this embodiment includes an image processing system 1, a start signal output interface 101, a transport control device 111, and a transport device 112.

[0011] The image processing system 1 has an imaging control device 102, an imaging device 103, an image processing device 104, a display 105, a mouse 106, a keyboard 107, and a lighting device 108. The image processing system 1 is connected to a transport control device 111 that controls a transport device 112. The transport control device 111 transports an object 113 to be inspected to the image processing system 1 using the transport device 112, and sends an inspection start signal to the image processing system 1 via the start signal output interface 101.

[0012] The imaging control device 102 has a control unit 114 and controls the imaging device 103 and the lighting device 108 to capture an image of the object 113 in synchronization with the lighting of the light source. Specifically, upon receiving an inspection start signal from a start signal input interface 115, the imaging control device 102 sends an imaging instruction to the imaging device 103 via a release signal output interface 116. The imaging control device 102 also receives a synchronization signal from the imaging device 103 via a synchronization signal input interface 117. The synchronization signal is sent from the imaging device 103 in synchronization with the imaging to notify the external strobe light source of the lighting timing. Furthermore, in response to the received synchronization signal, the imaging control device 102 lights up the light sources of the lighting device 108 in a predetermined order and combination via a lighting signal output interface 119. Through the above operations, an image of the object 113 illuminated by light from a predetermined illumination can be captured. The imaging control device 102 is connected to the image processing device 104 via a USB interface 118, and receives commands from the image processing device 104 and provides information indicating the status of the imaging control device 102 to the image processing device 104.

[0013] The imaging device 103 has a control unit 125 and an imaging optical system 121 including a lens, an imaging element, etc. The imaging device 103 generates a captured image by quantizing an optical image obtained by capturing an image based on an imaging command received via a release signal input interface 120 using an image processing engine 123. The imaging device 103 transfers the generated captured image to the image processing device 104 via a USB interface 124. Note that, in this embodiment, an example is described in which a still image captured using a digital camera is acquired and used, but a predetermined frame may be extracted as a still image from a moving image captured using a video camera and used. The imaging device 103 sends a synchronization signal to the imaging control device 102 via a synchronization signal output interface 122.

[0014] The image processing device 104 has a RAM 126, a ROM 127, a CPU 128, a GPU 129, and a USB interface 130. Each component is connected via an internal bus. The processing shown in the flowcharts described below is stored in the ROM 127 as program code. This program code is loaded into the RAM 126 and executed by the CPU 128 and GPU 129.

[0015] The illumination device 108 has multiple light sources 109. In this embodiment, the light sources 109 are LEDs, but other light sources such as xenon lamps may also be used. FIG. 3 shows an example of the arrangement of the multiple light sources 109. FIG. 3(a) is a front view of the illumination device 108, and FIG. 3(b) is a top view of the illumination device 108. The light sources 109, indicated by squares, are installed in a hemispherical pattern above the object 113, with at least one of the zenith angle and the azimuth angle differing from each other. Note that the number and arrangement of the light sources are not limited to the above example. In inspections using captured images, the light irradiation method must be changed depending on the appearance inspection item. For example, when inspecting the gloss of an object, it is necessary to irradiate light from a direction that allows light specularly reflected from the inspection surface to be captured. Furthermore, when inspecting the color or surface shape of an object, it is necessary to irradiate light from a direction that does not allow light specularly reflected from the inspection surface to be captured. Therefore, some of the light sources 109 are installed in a direction with a large incident angle with respect to the installation surface of the object 113, enabling imaging under geometric conditions where the imaging device 103 is likely to receive diffusely reflected light. Furthermore, some of the light sources 109 are installed in a direction with a small incident angle with respect to the installation surface of the object 113, enabling imaging under geometric conditions where the imaging device 103 is likely to receive specularly reflected light. The light sources 109 may each have a different light-emitting surface or spectral characteristics. For example, a spot-illumination type light source (illuminant indicated by a white square in FIG. 3 ) may be installed in a direction with a large incident angle with respect to the installation surface of the object 113, and a light source with light-emitting elements arranged in a circular ring shape (illuminant indicated by a gray square in FIG. 3 ) may be installed in a direction with a small incident angle. An annular ring light may also be used in a direction with a small incident angle. The illumination device 108 turns on a predetermined light source among the light sources 109 for a preset time in response to a command from the control unit 114.

[0016] In the image processing system 1 of this embodiment, the imaging control device 102, the imaging device 103, the image processing device 104, and the lighting device 108 are separate devices, but a plurality of devices may be integrated.

[0017] <Processing performed by the image processing system> FIG. 2 is a diagram showing an example of the functional configuration of the appearance inspection system according to this embodiment. The control unit 114 of the imaging control device 102 includes an imaging control unit 202. The imaging control unit 202 includes a release signal output unit 207, a synchronization signal input unit 208, a synchronization signal count unit 209, and an illumination signal output unit 210. The control unit 125 of the imaging device 103 includes an imaging unit 203. The imaging unit 203 includes a release signal input unit 211, a control unit 212, a synchronization signal output unit 213, and an image acquisition unit 214. The image processing device 104 includes an image processing unit 204. The image processing unit 204 includes an inspection image acquisition unit 215, a color and shape inspection unit 216, a gloss inspection unit 217, and an output unit 218. The transport control device 111 includes a start signal output unit 201. The transport device 112 includes a transport unit 206. The lighting device 108 includes an illumination unit 205.

[0018] Fig. 7 is a flowchart of processing executed by the image processing system 1 in this embodiment. When the object 113 is transported to a predetermined position by the transport device 112, the start signal output unit 201 of the transport control device 111 sends an inspection start signal to the imaging control unit 202. The processing in Fig. 7 starts when the imaging control unit 202 receives the inspection start signal. Hereinafter, each step (process) will be represented by adding an S before the reference number.

[0019] In S701, the imaging control unit 202 and the imaging unit 203 acquire multiple captured images by turning on a predetermined light source and capturing images of the object 113 to be inspected, and transfer the captured images to the image processing unit 204. Specifically, the release signal output unit 207 first receives an inspection start signal and sends a release signal to the release signal input unit 211. When the control unit 212 detects that the release signal input unit 211 has received the release signal, it executes an imaging operation. In this embodiment, high-speed continuous imaging is performed using a known continuous imaging function. The continuous imaging function is a function that repeatedly captures images at a predetermined speed while the release signal is continuously sent. In this embodiment, imaging is performed at 30 frames per second. During imaging, the control unit 212 causes the synchronization signal output unit 213 to output a synchronization signal to synchronize the timing of the external strobe light source with the timing of the shutter curtain opening. When the synchronization signal is input by the synchronization signal input unit 208, the synchronization signal count unit 209 counts the number of synchronization signal inputs, i.e., the number of captured images. The lighting signal output unit 210 switches the light sources to be turned on sequentially according to the number of captured images, and outputs a lighting signal to the illumination unit 205. Details of the process of controlling the lighting of the light sources by the lighting signal output unit 210 will be described later. The image acquisition unit 214 repeatedly transfers captured images obtained by turning on a predetermined light source and capturing an image of the object 113 to be inspected to the inspection image acquisition unit 215 as needed. Through the above process, in S701, the imaging control unit 202 and the imaging unit 203 can transfer a plurality of captured images to the image processing unit 204.

[0020] In S702, the color and shape inspection unit 216 and the gloss inspection unit 217 perform inspection processing based on the multiple captured images acquired by the inspection image acquisition unit 215. The color and shape inspection unit 216 detects defects by performing spatial filtering on the inspection image, which includes normal information and color information obtained by combining captured images using the photometric stereo method. The response value to the spatial filter is integrated and quantified to determine the degree of abnormality, and the pass / fail of the inspection is determined by comparing the degree of abnormality with a judgment threshold. The calculated degree of abnormality is displayed in the abnormality display area 609 of the inspection screen user interface (UI) shown in FIG. 6. The judgment threshold can be set in the threshold setting area 603 of the inspection screen UI shown in FIG. 6. The gloss inspection unit 217 uses the captured image obtained by receiving specularly reflected light from the inspection surface of the object as an inspection image containing gloss information, and performs spatial filtering in the same manner as above to detect defects. The photometric stereo method is a technique that can acquire normal information representing the surface shape of an object and color information representing the color of the object by combining captured images corresponding to multiple lighting directions. The above inspection process method is an example, and other inspection methods may be used.

[0021] In this embodiment, the appearance inspection items are three types: color, surface shape, and gloss. The appearance inspection items are not limited to the above examples, and may be anything that represents the appearance and can be identified by capturing an image. For example, material or pattern may be used. The inspection image acquisition unit 215 displays the inspection screen UI shown in FIG. 6 on the display 105 and accepts instructions from the user. One or more appearance inspection items are set in the inspection screen UI shown in FIG. 6. The user can input information into the inspection screen UI displayed on the display 105 using the mouse 106 or keyboard 107. When a product to be inspected is selected from a plurality of pre-registered products in the product selection area 601 using a drop-down menu, the appearance inspection item corresponding to the selected product to be inspected is set in the item setting area 602. The user can also change the appearance inspection item by inputting information into the radio buttons in the item setting area 602.

[0022] 6 has an examination start button 604 for instructing the start of an examination, and an examination stop button 605 for instructing the end of an examination. The examination screen UI also has an examination date area 606 for displaying the examination date, and an examination time area 607 for displaying the examination time.

[0023] In S703, the output unit 218 displays the results of the inspection process on the display 105. For example, in the inspection screen UI shown in FIG. 6, the judgment result area 608 displays "OK" if the inspection is passed, and displays "NG" if the inspection is failed. In addition, the abnormal event display area 610 displays that an abnormal event such as an imaging failure has occurred. Furthermore, the output unit 218 outputs to the transport unit 206 a transport instruction to a subsequent process for each of the products that passed the inspection and the products that failed the inspection.

[0024] <Processing to control the lighting of light sources> 4 is a flowchart of a process for controlling the lighting of a light source by the lighting signal output unit 210. In S401, the lighting signal output unit 210 sets a light source lighting counter to zero. This light source lighting counter counts the number of times a light source is turned on, and corresponds to the number of times the imaging device 103 captures an image of one object. In S402, the lighting signal output unit 210 sets a light source ID, which is an identification number for each light source 109, to zero. The light source ID corresponds to the number assigned to each light source in FIG. 3.

[0025] In S403, the lighting signal output unit 210 acquires the synchronization signal input by the synchronization signal input unit 208. In S404, the lighting signal output unit 210 determines whether the current light source ID is a light source ID that has been preset to be turned on simultaneously. If it is a light source ID for simultaneous lighting, the process proceeds to S405, and if it is not a light source ID for simultaneous lighting, the process proceeds to S408. Note that in this embodiment, light sources with light source IDs of 0 to 7 are preset as simultaneously turned on light sources, but the IDs of simultaneously turned on light sources are not limited to this. The simultaneously turned on light source ID may be set by the user via a UI displayed on the display 105, or a fixed ID may be set.

[0026] In S405, the lighting signal output unit 210 determines whether or not determination has been made for all light source IDs set as simultaneously turned on light sources. If determination has not been made for all simultaneously turned on light source IDs, the light source ID is incremented in S410 and the process proceeds to S404. S404, S405, and S410 are repeated, and if the lighting signal output unit 210 determines in S405 that determination has been made for all simultaneously turned on light source IDs, the process proceeds to S406. In S406, the lighting signal output unit 210 outputs a lighting signal to the illumination unit 205 to turn on the light sources set as simultaneously turned on light sources. Upon receiving the lighting signal, the illumination unit 205 turns on the simultaneously turned on light sources. In this embodiment, the illumination unit 205 turns on light sources with light source IDs 0 to 7.

[0027] If it is determined in S404 that the current light source ID is not a simultaneously-lit light source ID, in S408 the lighting signal output unit 210 outputs a lighting signal to the lighting unit 205 to light up the light source having an ID that matches the current light source ID. The lighting unit 205, having received the lighting signal, turns on the light source having an ID that matches the current light source ID. In S409, the lighting signal output unit 210 increments the light source ID. In S407, the lighting signal output unit 210 increments the light source lighting counter because any light source was turned on in S406 or S409. In S411, the lighting signal output unit 210 determines whether lighting of all light sources has been completed, and if not, the process returns to S403, and if completed, the process shown in FIG. 4 ends.

[0028] Through the above-described process, the light sources are turned on for each of the inspection target objects 113 for a predetermined number of images. FIG. 5 shows the correspondence between the synchronization signal acquired by the lighting signal output unit 210 and the lighting signal generated based on a predetermined lighting time. The process of the flowchart shown in FIG. 4 generates lighting signals for each light source, and eight of the 24 light sources are simultaneously turned on, resulting in a total of 17 lighting cycles, i.e., 17 images, completing imaging of one inspection target object. The method of simultaneously lighting the light sources is not limited to the above-described example. For example, if there are no plans to change the simultaneously lit light sources, the lighting device 108 may electrically connect light sources with light source IDs 0 to 7 and have one signal terminal capable of inputting a signal to this light source group. Furthermore, connecting light sources with high power consumption requires increasing the electrical capacity of the entire lighting device 108. Furthermore, the increased brightness caused by simultaneously lighting light sources may saturate the pixel values ​​of the captured image. Therefore, light sources with relatively low power consumption are used for the light sources to be simultaneously lit.

[0029] In this embodiment, when the appearance inspection item is color, the inspection image is color information obtained by combining 16 images captured in synchronization with the lighting of light sources 8 to 23, which have relatively large zenith angles. If the appearance inspection item is color only, a single image captured in synchronization with the lighting of any one of light sources 8 to 23, which is less affected by specular reflection, may be used as the inspection image. Alternatively, for example, pixel values ​​of four images captured in synchronization with the lighting of light sources 8 to 11 may be averaged, and the resulting average image may be used as the inspection image. Similarly to light sources 0 to 7 in the above example, a single image captured by simultaneously lighting light sources 8 to 11 may also be used as the inspection image.

[0030] In this embodiment, when the appearance inspection item is surface shape, the inspection image is normal information obtained by combining 16 captured images taken in synchronization with the lighting of light sources 8 to 23, which have relatively large zenith angles. In this embodiment, when the appearance inspection item is gloss, the inspection image is an image obtained by capturing light specularly reflected from the inspection surface. Specifically, the inspection image for gloss inspection is one captured image taken in synchronization with the lighting of light sources 0 to 7, which have relatively small zenith angles.

[0031] The color and shape inspection unit 216 and gloss inspection unit 217 can extract and inspect a predetermined area on the inspection surface of the object 113, for example, extracting an area preset by a user as the inspection target area. Depending on the three-dimensional shape of the industrial product, a captured image corresponding to illumination by a light source with a relatively large zenith angle may produce a shadow area in the inspection target area. In this case, by using a captured image corresponding to illumination by a light source with a relatively small zenith angle, the inspection target area can be extracted under conditions in which the generation of a shadow area is suppressed, thereby improving inspection accuracy.

[0032] 6 may have an area where the user can select the light source ID to be turned on. In this case, a warning may be displayed if the light source ID selected by the user is not suitable for the color, surface shape, and gloss selected in the item setting area 602. For inspection objects with relatively matte surfaces, it is also possible to capture images for gloss inspection using light sources with a large zenith angle, and the user can select light sources with light source IDs 8 to 11 to be turned on simultaneously as light sources for gloss inspection.

[0033] [Second embodiment] In the first embodiment, simultaneous lighting and individual sequential lighting of the light sources were controlled based on a synchronization signal output from the imaging device 103. In this embodiment, synchronized imaging is performed by simultaneously issuing an imaging instruction to the imaging device 103 and an illumination instruction to the lighting device 108 using a pulse signal output from the control unit 114. The following mainly describes the differences between this embodiment and the first embodiment. Note that the same components as in the first embodiment will be described using the same reference numerals.

[0034] 8 is a diagram showing an example of the hardware configuration of the appearance inspection system according to this embodiment. In the image processing system 1 according to this embodiment, the imaging device 103 does not have a synchronization signal output interface 122, and the imaging control device 102 does not have a synchronization signal input interface 117. The other hardware configurations are the same as those in the first embodiment.

[0035] <Processing performed by the image processing system> 9 is a diagram showing an example of the functional configuration of the appearance inspection system according to this embodiment. In the image processing system 1 according to this embodiment, the imaging unit 203 does not have the synchronization signal output unit 213, and the imaging control unit 202 does not have the synchronization signal input unit 208 or the synchronization signal count unit 209. The other functional configurations are the same as those in the first embodiment.

[0036] The processing executed by the image processing system 1 in this embodiment will also be described using the flowchart in FIG. 7. In S701, the imaging control unit 202 and the imaging unit 203 acquire multiple captured images obtained by turning on a predetermined light source and capturing an image of the object 113 to be inspected, and transfer the captured images to the image processing unit 204. Specifically, upon receiving an inspection start signal, the control unit 114 first transmits a pulse signal not only to the release signal output unit 207 but also to the lighting signal output unit 210. FIG. 10 shows an example of the pulse signal transmitted by the control unit 114. The pulse signal in the portion enclosed by the thick line in FIG. 10 has a preset signal waveform. When the start signal output unit 201 notifies the start of inspection, the release signal output unit 207 transmits the captured image signal of the portion enclosed by the thick line to the release signal input unit 211 of the imaging unit 203, and the imaging unit 203 performs imaging. At the same time, a light source lighting signal for the part surrounded by the thick line is sent from lighting signal output unit 210 to lighting unit 205, starting lighting of each light source 109. This method makes it possible to capture images synchronized with the lighting of each light source, and as shown in Fig. 10, simultaneous lighting can be controlled by sending a signal with the same waveform to light sources with IDs that are to be turned on simultaneously.

[0037] As explained above, by using the appearance inspection system of the above-mentioned embodiment, it is possible to simultaneously inspect the color, surface shape, and gloss of the object to be inspected. Furthermore, by turning on multiple light sources of the lighting device 108 at approximately the same timing during one image capture, the number of images captured can be reduced, thereby shortening the image capture time and the time required for inspection processing.

[0038] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0039] 102 Imaging control device 202 Imaging control unit 210 Lighting signal output unit

Claims

1. an acquisition means for acquiring a first signal indicating a lighting timing of the light source synchronized with the image capture; an output means for outputting a second signal to a lighting means, based on the first signal, for lighting a plurality of light sources at a first timing and for lighting a light source different from the plurality of light sources at a second timing among the lighting timings of the light sources; A control device comprising:

2. 2. The control device according to claim 1, wherein the output means outputs the second signal to the lighting means to turn on the plurality of light sources at the first timing and to turn on one light source different from the plurality of light sources at the second timing.

3. 2. The control device according to claim 1, wherein the captured image obtained by capturing an object illuminated with light from the plurality of light sources turned on at the first timing is used to inspect the gloss of the object.

4. 2. The control device according to claim 1, wherein an image obtained by capturing an object illuminated by a light source different from the plurality of light sources that is turned on at the second timing is used to inspect the color of the object.

5. 2. The control device according to claim 1, wherein an image obtained by capturing an object illuminated with light from a light source different from the plurality of light sources that is turned on at the second timing is used to inspect a surface shape of the object.

6. The light sources of the illumination means are arranged in a hemispherical distribution above the object to be inspected, 2. The control device according to claim 1, wherein the plurality of light sources that are turned on at the first timing have a smaller zenith angle than a light source that is different from the plurality of light sources that are turned on at the second timing.

7. The control device according to claim 1 , wherein the plurality of light sources that are turned on at the first timing are selected based on an instruction from a user.

8. the plurality of light sources that are turned on at the first timing are electrically connected to each other; 2. The control device according to claim 1, wherein the lighting means has one signal terminal capable of inputting signals to the plurality of light sources.

9. 2. The control device according to claim 1, wherein the plurality of light sources that are turned on at the first timing consume less power than a light source that is different from the plurality of light sources that are turned on at the second timing.

10. A program for causing a computer to function as the control device according to any one of claims 1 to 9.

11. an acquisition step of acquiring a first signal indicating a lighting timing of the light source synchronized with the image capture; an output step of outputting, to a lighting means, a second signal for lighting a plurality of light sources at a first timing and a light source different from the plurality of light sources at a second timing, based on the first signal; A control method comprising:

Images