Information processing device, information processing method, and program

Abstract

The objective is to provide a process for easily determining appropriate lighting conditions in order to suppress synchronization discrepancies between light source activation and image capture. [Solution] The information processing device acquires information regarding the synchronization discrepancy between image capture and light source illumination, displays the illumination conditions based on the acquired information, and performs an inspection of the object based on the image data obtained by capturing the object under the illumination conditions specified by the user.

Description

Technical Field

[0001] The present invention relates to a technique for inspecting objects.

Background Art

[0002] Conventionally, inspection systems for detecting defects on the surface of objects have been known for appearance inspection of industrial products. A general inspection system irradiates an object to be inspected with light, captures an image of the reflected light reflected by the object to be inspected with an imaging device, and detects defects based on the captured image. In recent years, inspection systems that perform various processes using a large number of light sources have been proposed. In this inspection system, it is necessary to select light sources to be turned on from among a large number of light sources, but depending on the number of light sources to be turned on, the appearance of defects changes, and the inspection accuracy also changes. Patent Document 1 discloses turning on a plurality of light sources in accordance with a lighting pattern corresponding to a control signal among a plurality of lighting patterns of light sources stored in a storage unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] When the lighting time of the light source is short, the difficulty of synchronization between light source lighting and imaging increases, and the possibility of synchronization misalignment increases. There has been a problem that it is difficult to determine appropriate illumination conditions to suppress this synchronization misalignment.

[0005] Accordingly, an object of the present invention is to provide a process for easily determining appropriate illumination conditions in order to suppress synchronization misalignment between light source lighting and imaging.

Means for Solving the Problems

[0006] To solve the above problems, the information processing device according to the present invention is characterized by comprising: an acquisition means for acquiring information on the synchronization mismatch between imaging and the illumination of a light source; a display control means for displaying lighting conditions based on the information; and an inspection means for inspecting an object based on image data obtained by imaging the object under lighting conditions specified by the user. [Effects of the Invention]

[0007] According to the present invention, appropriate lighting conditions can be easily determined to suppress synchronization discrepancies between light source illumination and image capture. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing the hardware configuration of the information processing device. [Figure 2] Diagram showing the appearance of the image inspection system. [Figure 3] Block diagram showing the functional configuration of the information processing device. [Figure 4] A flowchart illustrating the processes performed by an information processing device. [Figure 5] Diagram showing an example of a user interface [Figure 6] Diagram showing an example of a user interface [Figure 7] Diagram showing an example of a user interface [Modes for carrying out the invention]

[0009] Each embodiment will be described below with reference to the drawings. Note that the following embodiments do not necessarily limit the present invention. Furthermore, not all combinations of features described in each embodiment are essential to the solution of the present invention.

[0010] [First Embodiment] <Configuration of the imaging inspection system> Figure 1 is a block diagram showing the hardware configuration of the information processing device 1. The information processing device 1 includes a CPU 101, ROM 102, and RAM 103. It also includes a VC (video card) 104, a general-purpose I / F (interface) 105, a SATA (serial ATA) I / F 106, and a NIC (network interface card) 107. The CPU 101 uses RAM 103 as work memory to execute the OS (operating system) and various programs stored in storage devices 113 such as ROM 102 and HDD (hard disk drive). The CPU 101 also controls each component via the system bus 108. The processing described later using a flowchart involves program code stored in ROM 102 and storage devices 113 being loaded into RAM 103 and executed by the CPU 101. A display device 115 is connected to VC 104. Input devices 110 such as a mouse and keyboard, an imaging device 111, and a lighting device 116 are connected to the general-purpose I / F 105 via the serial bus 109. The imaging device 111 is a device provided for imaging the object to be inspected, and may be an imaging device that captures still images or an imaging device that captures moving images. The illumination device 116 is a device provided for irradiating the object to be inspected with light, and has multiple light sources. A general-purpose drive 114 that reads and writes to the storage device 113 and various recording media is connected to the SATAI / F 106 via the serial bus 112. The NIC 107 performs information input and output with external devices. The CPU 101 uses the storage device 113 and various recording media mounted on the general-purpose drive 114 as storage locations for various data. The CPU 101 displays the UI (user interface) provided by the program on the display device 115 and receives input such as user instructions via the input device 110. The CPU 101 outputs the processing results from the CPU 101 to the display device 115 via the VC 104, thereby displaying the processing results on the display device 115 as images or characters. Furthermore, the display device 115 may be a touch panel display that has the function of detecting the position of a touch made by an indicator such as a finger. Alternatively, the display device 115 may be an image projection device such as a projector.The information processing device 1 can be a computer device such as a PC (personal computer), smartphone, or tablet terminal. Note that the configuration of the information processing device 1 shown in Figure 1 is just one example; for example, a device combining two or more of the devices shown in Figure 1 may also be used as the information processing device 1.

[0011] Figure 2 shows the external appearance of the image inspection system in this embodiment. Figure 2(a) is a front view of the illumination device 116, and Figure 2(b) is a top view of the illumination device 116. The illumination device 116 has a plurality of light sources 203. In this embodiment, the light sources 203 are LEDs, but other light sources such as xenon lamps may also be used. The light sources 203, indicated by squares, are installed in a hemispherical distribution above the object 202, and at least one of their zenith angle and azimuth angle is different from each other. The imaging device 201 is an imaging device that images the object 202 illuminated by the light sources 203. Note that the number and arrangement of the light sources 203 are not limited to the above example, and it is sufficient to have a plurality of light sources with different zenith angles and azimuth angles. The illumination device 116 lights up a predetermined light source among the light sources 203 for a preset time in response to a command from the information processing device 1.

[0012] <Functional Configuration of Information Processing Devices> Figure 3 is a block diagram showing the functional configuration of the information processing device 1. The CPU 101 functions as shown in Figure 3 by reading and executing program code stored in the ROM 102 or storage device 113, using the RAM 103 as work memory. Note that not all of the following processes must be executed by the CPU 101; the information processing device 1 may be configured so that some or all of the processes are performed by one or more processing circuits other than the CPU 101.

[0013] The information processing device 1 includes an illumination control unit 301, an imaging control unit 302, an output unit 303, a display control unit 304, and an inspection unit 305. The illumination control unit 301 controls the lighting of multiple light sources in the illumination device 116 to illuminate the object to be inspected. The imaging control unit 302 controls the imaging device 111 to image the illuminated object. The output unit 303 derives information regarding the synchronization difference between imaging and light source illumination based on the illumination time and the number of light sources to be illuminated by the illumination device 116. The display control unit 304 controls the display device 115 based on the information regarding the synchronization difference and displays the illumination time and the number of light sources to be illuminated as recommended lighting conditions. The inspection unit 305 inspects the object to be inspected based on image data obtained by imaging under lighting conditions specified by the user.

[0014] <Processing performed by the information processing device> In this embodiment, the processing flow executed by the information processing device 1 will be explained using the flowchart in Figure 4. The processing shown in the flowchart in Figure 4 begins when an instruction is input by the user via the input device 110 and the CPU 101 receives the input instruction. Hereafter, each step (process) will be represented by adding an S before the symbol.

[0015] In S401, the derivation unit 303 derives information regarding the synchronization mismatch between imaging and light source illumination based on the light irradiation time by the illumination device 116 and the number of light sources to be illuminated. First, the derivation unit 303 obtains multiple combinations of light irradiation time and the number of light sources to be illuminated. For example, multiple combinations can be obtained by the user specifying candidates for each of the light irradiation time and the number of light sources to be illuminated via the UI. Note that multiple combinations of light irradiation time and the number of light sources to be illuminated may be stored in advance in the ROM 102 or storage device 113. Next, the derivation unit 303 derives the synchronization failure rate between the imaging timing and the light source illumination timing for each of the multiple combinations of light irradiation time and the number of light sources to be illuminated. The derivation unit 303 derives the synchronization failure rate such that the shorter the light irradiation time, the higher the synchronization failure rate. This is because when the light illumination time is short, the synchronization timing must be very precise, and even a slight mismatch is likely to lead to synchronization failure. Furthermore, the derivation unit 303 derives the synchronization failure rate so that it increases in proportion to the number of light sources lit, because the control processing becomes more complex as the number of light sources lit increases, and the possibility of synchronization failure increases. An example of a specific method for deriving the synchronization failure rate is shown below. The ROM 102 or storage device 113 stores the synchronization failure rate for each light irradiation time per lamp. The derivation unit 303 obtains the synchronization failure rate corresponding to the acquired light irradiation time from the ROM 102 or storage device 113. For example, suppose the synchronization failure rate when the light irradiation time per lamp is 10 ms is 0.1% (=0.001). If the number of light sources lit is 10, the probability that all 10 lamps will successfully synchronize is (1-0.001). 10 ≈ 0.99. Therefore, the derivation unit 303 can derive the probability that at least one light will fail to synchronize in a single inspection as 1 - 0.99 = 0.01 (= 1%). The derivation unit 303 derives the synchronization failure rate for each of several combinations of light irradiation time and the number of light sources to be lit using the method described above.

[0016] In S402, the display control unit 304 presents, to the user, the light irradiation time and the number of light sources to be turned on as recommended illumination conditions, based on the derived synchronization failure rate. FIG. 5 shows an example of a UI for presenting the recommended illumination conditions. An area 501 is an area for the user to input a threshold value of the synchronization failure rate. When the threshold value is input into the area 501, among a plurality of combinations of the light irradiation time and the number of light sources to be turned on, the combinations whose synchronization failure rate is less than the threshold value are displayed in a list in an area 503. In the area 503, after the user selects one combination from the combinations displayed in the list, a setting button 502 is pressed, whereby the combination of the light irradiation time used for inspection and the number of light sources to be turned on can be set. By scrolling the display range using a scroll bar 504, all combinations with the synchronization failure rate less than the threshold value can be checked. Note that, by selecting a displayed combination, the synchronization failure rate corresponding to the selected combination may be displayed. Further, the synchronization failure rate may be displayed by being replaced with a synchronization failure degree such as "high", "medium", or "low". Further, before the user makes a selection, the synchronization failure rate may be displayed by, for example, hovering a cursor or the like over a certain combination. Further, the respective synchronization failure rates may be always displayed for all the combinations whose synchronization failure rate is less than the threshold value. Further, the display method is not limited to a table format, and for example, a pull-down format may be adopted so that only those satisfying the conditions can be selected.

[0017] In S403, the illumination control unit 301 and the imaging control unit 302 synchronize imaging and light source lighting to capture an image of the object to be inspected based on the illumination conditions set in S402. Specifically, the illumination control unit 301 sequentially turns on the light sources included in the illumination device 116 to irradiate the object to be inspected with light. The imaging control unit 302 causes the imaging device 111 to capture an image of the object in accordance with the lighting timing of each light source. Note that in the present embodiment, imaging is performed using all light sources, but the number of imaging operations is not limited to this. For example, in order to speed up the inspection, the number of imaging operations may be reduced by setting effective light sources in advance according to the object to be inspected. The effective light sources are, for example, light sources that reduce the shadowed area according to the height of the object, or light sources that easily receive reflected light near specular reflection. Further, for example, a plurality of imaging operations may be performed while one light source is turned on in order to reduce the influence of noise. Further, a plurality of light sources may be turned on simultaneously to perform imaging.

[0018] In S404, the inspection unit 305 inspects the object to be inspected based on the plurality of images obtained by the imaging in S403. First, the inspection unit 305 combines the plurality of images obtained by the imaging in S403 using a known photometric stereo method, and generates a shape image representing the shape of the object surface and a color image representing the color of the object surface. The shape image is, for example, an image having normal information for each pixel, and the color image is, for example, an image having color information such as RGB values for each pixel. Next, the inspection unit 305 detects defects by applying spatial filter processing to each of the shape image and the color image. In the present embodiment, the inspection unit 305 calculates, as an anomaly score, a value obtained by integrating and quantifying the response value to the spatial filter processing on the inspection image. The display control unit 304 issues an error notification or the like to the user via the display device 115 in accordance with the magnitude of the calculated anomaly score.

[0019] As described above, the information processing device 1 of this embodiment presents the user with recommended lighting conditions, including the light irradiation time and the number of light sources to be lit, based on the synchronization failure rate between the timing of image capture and the timing of light source illumination. This makes it easy to determine appropriate lighting conditions to suppress synchronization discrepancies between light source illumination and image capture.

[0020] <Variation> In this embodiment, the shape and color were inspected using the photometric stereo method, but the inspection method is not limited to the example above. For example, the glossiness of the object's surface may be inspected.

[0021] In this embodiment, the derivation unit 303 derives the synchronization failure rate, but if the synchronization failure rate is known in advance, the user may input it. Alternatively, the synchronization failure rate may be stored in advance in the ROM 102 or the storage device 113.

[0022] Furthermore, in this embodiment, while the synchronization failure rate and degree of synchronization failure were described as information regarding synchronization misalignment, it could also be, for example, the number of checks until a predetermined number of synchronization failures occur.

[0023] [Second Embodiment] In the first embodiment, a list was displayed of multiple combinations of light irradiation time and the number of light sources to be lit, in which the synchronization failure rate was below a threshold. In this embodiment, the synchronization failure rate corresponding to the lighting conditions is displayed, and recommended lighting conditions are presented to the user. The hardware configuration and functional configuration of the information processing device 1 in this embodiment are the same as those of the first embodiment, so a description is omitted. The following mainly describes the differences between this embodiment and the first embodiment.

[0024] <Presentation of lighting conditions (S402)> In S402, the display control unit 304 presents the user with recommended lighting conditions, including light irradiation time and the number of light sources to be lit, based on the derived synchronization failure rate. Figure 6 shows an example of a UI for presenting recommended lighting conditions. The user can display the recommended range by checking the checkbox 601, which is used to set whether or not to display the recommended range. For items for which the recommended range has been set to be displayed using checkbox 601, the inspection conditions can be adjusted using the slider 602. For example, if the user is willing to accept a certain amount of inspection time but wants to increase inspection accuracy, they can set the inspection time slider 602 to the middle and the inspection accuracy slider 602 to a higher value, as shown in Figure 6. In this embodiment, the inspection conditions are adjusted using sliders, but the method of setting the inspection conditions is not limited to this. For example, numerical values ​​can be entered directly, or the settings can be made in a pull-down format.

[0025] The user specifies the items for which they want the recommended range to be displayed using checkbox 601 and sets the inspection conditions using slider 602. The correspondence display 603 between lighting conditions and synchronization failure rate then displays the recommended range of lighting conditions that meet the set inspection conditions. To distinguish between recommended ranges when multiple items are specified in checkbox 601, the type of line representing the recommended range differs for each item. Note that the method of distinguishing between recommended ranges for each item is not limited to the type of line; it could also be color, blinking pattern, etc. Area 604 represents the lighting conditions specified by the user. The user can set the lighting conditions by selecting one lighting condition from the recommended range and pressing the setting button 605. Figure 6 only displays 42 combinations of light irradiation time and the number of light sources to be lit, but all combinations can be viewed by scrolling using the scroll bar 606.

[0026] As described above, the information processing device 1 of this embodiment presents the user with a recommended range of illumination conditions according to the inspection conditions specified by the user. This makes it easy for the user to determine appropriate illumination conditions in order to suppress synchronization discrepancies between light source illumination and image capture while satisfying the inspection conditions they desire.

[0027] [Other embodiments] In the second embodiment, the synchronization failure rates corresponding to the lighting conditions are displayed in a list, and the recommended range that satisfies the desired inspection conditions is displayed overlaid. In another embodiment, instead of displaying the synchronization failure rate, "OK" may be displayed for lighting conditions corresponding to a synchronization failure rate below a threshold, and "NG" may be displayed for lighting conditions corresponding to a synchronization failure rate above the threshold. Figure 7 shows an example of the UI. Area 701 is an area for the user to input a threshold for the synchronization failure rate. The threshold entered in area 701 is compared with the derived synchronization failure rate, and "OK" is displayed for lighting conditions corresponding to a synchronization failure rate below the threshold, and "NG" is displayed for lighting conditions corresponding to a synchronization failure rate above the threshold. According to this embodiment, it is possible to see at a glance whether the lighting conditions are OK or OK from the perspective of the synchronization failure rate, and the lighting conditions can be set smoothly.

[0028] The embodiments described above may be combined as appropriate. For example, as in the first embodiment, lighting conditions may be filtered by a synchronization failure rate threshold, and as in the second embodiment, a recommended range of lighting conditions according to the inspection conditions may be displayed.

[0029] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of symbols]

[0030] 1. Information Processing Device 303 Derivation part 304 Display Control Unit 305 Inspection Department

Claims

1. An acquisition means for acquiring information regarding the synchronization discrepancy between image capture and light source illumination, Based on the aforementioned information, a display control means for displaying lighting conditions, An inspection means for inspecting an object based on image data obtained by imaging the object under lighting conditions specified by the user, An information processing device characterized by having the following features.

2. A lighting control means that illuminates the object by turning on the light source of the lighting means based on lighting conditions specified by the user, An imaging control means for controlling an imaging means that images the object in synchronization with the illumination of the light source, The information processing apparatus according to claim 1, further comprising the features.

3. The information processing apparatus according to claim 1, characterized in that the acquisition means derives the information based on a combination of light irradiation time and the number of light sources to be lit.

4. The information processing apparatus according to claim 3, characterized in that the aforementioned information is the synchronization failure rate between imaging and light source illumination.

5. The information processing apparatus according to claim 4, characterized in that the display control means displays lighting conditions corresponding to the synchronization failure rate below a threshold.

6. The information processing apparatus according to claim 5, characterized in that the display control means sets the threshold based on the user's instructions.

7. The information processing apparatus according to claim 4, characterized in that the display control means displays the lighting conditions and the synchronization failure rate in correspondence.

8. The information processing apparatus according to claim 7, characterized in that the display control means presents the user with a range of recommended lighting conditions.

9. The information processing apparatus according to claim 1, characterized in that the inspection means inspects the shape and color of the object by photometric stereo.

10. A program for causing a computer to function as one of the means of an information processing device described in any one of claims 1 to 9.

11. An acquisition process to acquire information regarding the synchronization discrepancy between image capture and light source illumination, Based on the aforementioned information, a display control process is performed to display the lighting conditions, An inspection step in which an object is inspected based on image data obtained by imaging the object under lighting conditions specified by the user, An information processing method characterized by having the following features.

Images