Information processing device, information processing method, and program

Abstract

To detect, at low cost, whether the synchronization between light source illumination and image capture was properly performed. [Solution] The imaging control unit performs synchronous imaging by synchronously controlling the imaging unit and the illumination unit to continuously image the object to be inspected at the imaging position. The sync signal input unit reads the state of the sync signal output from the imaging unit, i.e., whether it is ON or OFF, and records the time when the sync signal was read. The imaging control unit calculates the elapsed time from the time when the sync signal ON was last detected to the time when the reading of the sync signal was resumed, and determines whether that elapsed time is longer than the period + pulse width of the sync signal. If the elapsed time is longer than the period + pulse width of the sync signal, the control unit outputs a message to the output unit indicating that the sync signal ON may have been missed.

Description

Technical Field

[0001] The present disclosure relates to an information processing technology for controlling an imaging system that performs imaging in synchronization with lighting.

Background Art

[0002] Currently, in appearance inspection, an imaging system that controls the lighting timing of a light source in synchronization with the imaging timing of an imaging device is used for imaging.

[0003] <……>Patent Document 1 discloses a high-speed imaging system that controls the lighting state of a light source based on a synchronization signal synchronized with the exposure timing of an imaging frame output from a high-speed camera. Such synchronous imaging control, which is real-time control, is preferably performed using a dedicated information processing device from the perspective of stability. However, it is also required to operate at low cost using a general-purpose information processing device that uses a general-purpose OS such as a PC.

[0004] However, when using a general-purpose information processing device, since a plurality of processes other than synchronous imaging control are executed in parallel, a shortage of calculation resources occurs intermittently, and synchronous imaging control cannot be stably and appropriately executed. When performing an inspection using an imaging image for which a certain degree of failure in synchronous imaging is assumed, in order to ensure the reliability of the inspection result, information indicating whether the imaging image of the inspection target was appropriately synchronously imaged is required.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, information indicating whether or not the captured images were properly synchronized can be obtained, for example, through image analysis. Obtaining analysis results in this way requires high processing power, making it difficult to operate at a low cost. [Means for solving the problem]

[0007] This disclosure is characterized by comprising: reading means for reading changes in a synchronization signal for generating a control signal to synchronize the turning on of an illumination device for illuminating an object to be inspected with imaging of the object to be inspected by an imaging device; and output means for outputting a signal indicating that the synchronization between imaging by the imaging device and the turning on of the illumination device has failed if the time from the detection that the synchronization signal has been turned ON until the next reading of the change in the synchronization signal is longer than a predetermined time. [Effects of the Invention]

[0008] According to this disclosure, it is possible to detect at low cost whether the synchronization between the illumination of the light source and the imaging has been properly performed. [Brief explanation of the drawing]

[0009] [Figure 1] This figure illustrates an example of the hardware configuration and appearance of an image processing system according to Embodiment 1. [Figure 2] This is a block diagram showing an example of the functional configuration of an image processing system according to Embodiment 1. [Figure 3] This is a flowchart illustrating the transport process of the object to be inspected, the imaging process of the object to be inspected, and the inspection process according to Embodiment 1. [Figure 4] This is an explanatory diagram showing an example of a timing chart for signals used in the synchronous imaging process according to Embodiment 1. [Figure 5] This figure shows an example of a user interface representing the results of the inspection process according to Embodiment 1. [Figure 6] This figure illustrates a method for detecting a failure in synchronized imaging according to Embodiment 1. [Figure 7]This figure illustrates a method for detecting a failure in synchronized imaging according to Embodiment 1. [Figure 8] This is a diagram illustrating the hardware configuration of the image processing system according to Embodiment 2. [Figure 9] This is a diagram illustrating the hardware configuration of the image processing system according to Embodiment 3. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described below with reference to the drawings. Note that the embodiments described below are not limiting to this disclosure, and not all combinations of features described in the embodiments are necessarily essential as solutions of this disclosure. Hereinafter, identical components will be described using the same reference numerals. Furthermore, each step in the flowchart will be described using a reference numeral preceded by "S".

[0011] [Embodiment 1] <Hardware Configuration> Figure 1(a) shows an example of the hardware configuration of the image processing system 1, including the information processing device 102 according to this embodiment. Figure 1(b) shows the side view of the image processing system 1. Figure 1(c) shows the top view of the image processing system 1.

[0012] The image processing system 1 according to this embodiment includes an information processing device 102, an imaging device 103, an image processing device 104, a display 105, a mouse 106, a keyboard 107, and a lighting device 108. The image processing system 1 is communicatively connected to a start signal output interface (start output I / F) 101 and a transport control device 111.

[0013] The information processing device 102 outputs an imaging instruction signal (hereinafter referred to as a release signal) for instructing imaging to the imaging device 103, and acquires an imaging timing signal (hereinafter referred to as a sync signal) indicating the timing of imaging from the imaging device 103. Further, the information processing device 102 performs lighting control of at least one of the light sources 109 in response to acquiring the sync signal.

[0014] The information processing device 102 controls the lighting device 108 to light a plurality of light sources 109 having different irradiation angles with respect to the installation surface of the inspection object 113-2, individually or simultaneously, in a predetermined order, for a predetermined time. The information processing device 102 can cause the imaging device 103 to continuously image the inspection object 113-2 simultaneously with the control of the lighting device 108. Thereby, the information processing device 102 can acquire a plurality of imaging images with different lighting directions for the inspection object 113.

[0015] <Hardware Configuration of Image Processing Device> The image processing device 104 is, for example, a personal computer, and performs image processing on the image acquired from the imaging device 103. The image processing device 104 includes a RAM 126, a ROM 127, a CPU 128, a GPU 129, and a USB interface (USBI / F) 130. These functional units are communicably connected to each other via an internal bus. Data for executing the processing shown in the flowchart described in FIG. 3 to be described later is stored in advance in the ROM 127 as program code. This program code is developed in the RAM 126 and executed by the CPU 128 or the GPU 129.

[0016] The mouse 106 and keyboard 107 are each connected to the image processing device 104 via USBI / F130 so as to be able to communicate with it and accept input from the user. In this embodiment, the image processing device 104 is described as being connected to the mouse 106 and keyboard 107 as input units that accept input from the user, but it may have different input units such as a touch panel or mechanical switches. The display 105 is connected to the GPU 129 and presents information to the user. The information processing device 102, imaging device 103, and transport control device 111 are connected to the image processing device 104 via USBI / F130 so as to be able to communicate with it.

[0017] <Hardware configuration around the conveying device> The transport device 112 is a belt conveyor that transports the object to be inspected 113. The transport control device 111 is a single-board microcontroller equipped with GPIO (General-purpose input / output) and USBI / F. The image processing device 104 outputs transport control commands to the transport control device 111 via the USBI / F 130. The transport control device 111 controls the transport device 112 via GPIO based on the input transport control commands. A PLC (Programmable Logic Controller) may be used as the transport control device 111.

[0018] The start output interface is a GPIO provided by the single-board microcontroller that constitutes the transport control device 111. The transport control device 111 outputs an imaging start signal to the information processing device 102 via the start output interface.

[0019] <Hardware configuration of the information processing device> The information processing device 102 according to this embodiment is described as a personal computer (PC) equipped with a device control board having GPIO, and is assumed to have a general-purpose OS installed that is not for real-time control purposes. However, the information processing device 102 may be a device with a different configuration as long as it has an OS that is not for real-time control purposes and is capable of executing the processing performed by the information processing device 102 according to this embodiment. For example, the information processing device 102 may be a single-board microcomputer equipped with GPIO, a server that communicates with the imaging device 103 and the illumination device 108, or a device built into either the imaging device 103 or the illumination device 108.

[0020] The information processing device 102 controls the imaging device 103 and the illumination device 108. This allows the information processing device 102 to control the imaging device 103 to image the object to be inspected 113 while individually or simultaneously illuminating multiple light sources 109 that constitute the multi-lamp illumination system, in a predetermined order and for a predetermined time, thereby acquiring multiple images with different illumination directions. The information processing device 102 includes a control unit 114, a start signal input interface (start input I / F) 115, and a release signal output interface (release output I / F) 116. Furthermore, the information processing device 102 includes a sync signal input interface (sync input I / F), a USB I / F 118, and a lighting signal output interface (lighting output I / F) 119. These functional units are connected to each other via an internal bus for communication.

[0021] The control unit 114 is a microcontroller equipped with RAM, ROM, and a CPU. The data for executing the processes shown in the flowchart in Figure 3, which will be described later, is pre-stored as program code in the ROM of the control unit 114. This program code is loaded into the RAM of the control unit 114 and executed by the CPU of the control unit 114.

[0022] The USBI / F118 connects the control unit 114 to the image processing unit 104 via the USBI / F130. The image processing unit 104 stores the program code executed by the control unit 114 in the RAM provided by the control unit 114 via the USBI / F130 and USBI / F118.

[0023] The start input interface 115 is a GPIO pin provided by the single-board microcontroller that constitutes the information processing device 102. In addition, the release output interface 116, the sync input interface 117, and the illumination output interface 119 are connected to GPIO pins provided by the single-board microcontroller that constitutes the information processing device 102.

[0024] The start input interface 115 acquires the imaging start signal output from the start output interface 101. When the imaging start signal is input, the control unit 114 starts executing the program code pre-stored in the ROM of the control unit 114. Alternatively, the image processing device 104 may output commands related to the execution or cessation of the program code in the control unit 114 to the control unit 114 via the USB interface 130 and USB interface 118.

[0025] The release output interface 116 outputs a release signal to the release signal input interface (release input interface) 120 provided by the imaging device 103, instructing the imaging of the object to be inspected 113. In this embodiment, the release signal is an electrical signal consisting of two states: ON and OFF. However, the release signal is not limited to an electrical signal, but may be a wireless signal or an optical signal, as long as it is a signal that can similarly instruct imaging. The imaging device 103 starts continuous imaging when the input release signal turns ON. The imaging device 103 then continues continuous imaging as long as the release signal is ON, and stops continuous imaging when the release signal turns OFF. The imaging device 103 may receive both a signal instructing continuous imaging ON and a signal instructing continuous imaging OFF, or it may turn OFF after a predetermined period of time following the reception of the signal instructing continuous imaging ON.

[0026] The sync input interface 117 acquires the imaging timing signal output from the imaging device 103 in accordance with the timing of capturing each still image during continuous imaging, from the sync signal output interface (sync output interface) 122 provided by the imaging device 103. In this embodiment, the sync signal is an electrical signal consisting of two states: ON and OFF. However, the sync signal is not limited to an electrical signal, as long as it indicates the timing of imaging, it may also be a wireless signal or an optical signal.

[0027] The lighting output interface 119 outputs a lighting signal to the lighting device 108 that controls the lighting of each light source 109 provided in the lighting device 108. Whenever the input synchronization signal is turned ON, the control unit 114 outputs a lighting signal to the lighting device 108 via the lighting output interface 119, and lights up each light source 109 individually or in groups simultaneously, in a predetermined order, for a predetermined time.

[0028] As described above, the information processing device 102 controls the imaging device 103 and the illumination device 108 to continuously image the object to be inspected 113 while illuminating multiple light sources constituting the multi-lamp illumination individually or simultaneously, in a predetermined order and for a predetermined time. This allows the information processing device 102 to acquire multiple images with different illumination directions.

[0029] <Hardware configuration of the imaging device> The imaging device 103 is a digital camera that images the object to be inspected 113. In this embodiment, a consumer interchangeable-lens single-lens reflex camera is used as the imaging device 103, but the imaging device 103 may be an industrial camera as well as a consumer camera.

[0030] The imaging device 103 includes a release input I / F 120, an imaging optical system 121, a sync output I / F 122, an image processing engine 123, a USB I / F 124, and a control unit 125. Each of these functional units is connected to each other via an internal bus so as to be able to communicate with one another.

[0031] The control unit 125 is a microcontroller equipped with RAM, ROM, and a CPU. The data for executing the processes shown in the flowchart in Figure 3, which will be described later, is pre-stored as program code in the ROM of the control unit 125. This program code is loaded into the RAM of the control unit 125 and executed by the CPU of the control unit 125.

[0032] The release input interface 120 acquires the release signal output from the release output interface 116 of the information processing device 102. When the ON release signal is input, the control unit 125 controls the imaging optical system 121, which consists of a lens and an image sensor, to image the object to be inspected 113. Specifically, the control unit 125 starts continuous imaging when the release signal is turned ON. The control unit 125 then continues continuous imaging as long as the release signal is ON, and stops continuous imaging when the release signal is turned OFF. The function of continuing continuous imaging while the release signal is ON is a function that is generally provided in consumer interchangeable-lens single-lens reflex cameras.

[0033] The control unit 125 outputs a sync signal during continuous imaging, in accordance with the timing of capturing each still image. Specifically, the control unit 125 outputs a sync signal to the sync input I / F 117 of the information processing device 102 via the sync output I / F 122. The function of outputting a sync signal during continuous imaging, in accordance with the timing of capturing each still image, is a function commonly found in consumer interchangeable-lens single-lens reflex cameras. The sync signal in this embodiment is a signal commonly used to illuminate an external strobe light source in synchronization with imaging.

[0034] The image processing engine 123 generates digital image data based on the optical image on the imaging sensor and stores it as an captured image in the RAM of the control unit 125. The USBI / F 124 connects the control unit 125 to the image processing device 104 via the USBI / F 130.

[0035] The image processing device 104 acquires the captured image stored in the RAM of the control unit 125 of the imaging device 103 via USBI / F124 and USBI / F130. The image processing device 104 stores the acquired captured image in its own RAM 126. The image processing device 104 also sets the imaging mode of the imaging device 103 (ISO sensitivity, shutter speed, aperture, continuous imaging mode, imaging area, or image format, etc.) via USBI / F130 and USBI / F124.

[0036] Furthermore, some or all of the processes described below as being performed by the image processing device 104 may be performed by the information processing device 102. Also, some of the processes performed by the imaging device 103 may be performed by the information processing device 102. In addition, if the same processes can be performed by the image processing system 1, some of the processes performed by the information processing device 102 may be performed by the imaging device 103 or the image processing device 104.

[0037] <Hardware configuration of the lighting system> The illumination device 108 is a dome-shaped multi-lamp illumination system equipped with multiple light sources. In the image processing system 1 according to this embodiment, the imaging device 103 is positioned at the apex of the dome, as shown in Figure 1(b). The illumination device 108 can irradiate the object to be inspected 113-2 located directly below it with light from multiple directions.

[0038] Figure 1(d) shows an example of the arrangement of light sources 109 in a cross-section passing through the vertex of the lighting device 108. Figure 1(e) shows an example of the arrangement of light sources 109 as seen from the top surface of the lighting device 108. In Figures 1(d) and (e), each light source 109 is indicated by a black square. The lighting device 108 according to this embodiment is equipped with 40 light sources as a plurality of light sources 109, as shown in Figure 1(e). These 40 light sources 109 are arranged at five different zenith angles: 10°, 30°, 45°, 60°, and 80°, as shown in Figure 1(d). Furthermore, these 40 light sources 109 are arranged at 24 different azimuth angles from 0° to 345° at intervals of 15°, as shown in Figure 1(e). The information processing device 102 controls the imaging device 103 and the illumination device 108 to continuously image the object to be inspected 113 while illuminating multiple light sources 109 that constitute the multi-lamp illumination individually or simultaneously, in a predetermined order and for a predetermined time. As a result, the information processing device 102 can acquire multiple images with different illumination directions.

[0039] As will be described later, in this embodiment, eight light sources 109 with a zenith angle of 10° are lit simultaneously, while light sources 109 with zenith angles from 30° to 80° are lit individually. In the case where the power consumption of each light source 109 is the same, the total power consumption is greater when they are lit simultaneously than when they are lit individually. An increase in the total power consumption makes the power supply unit more expensive. Also, the amount of light emitted by a light source 109 is almost proportional to the power consumption. Therefore, if the number of light sources 109 lit simultaneously increases, they may become too bright compared to individually lit light sources, causing the pixel value to saturate. For this reason, light-emitting devices with lower power consumption are used for the light sources 109 that are lit simultaneously than for the light sources lit individually. That is, since the light source 109 with a zenith angle of 10° is intended to be lit simultaneously, a light-emitting device with lower power consumption is used for it than for the light sources 109 with other zenith angles. For this reason, in Figures 1(d) and (e), when each light source is represented by a black square, the light source with a zenith angle of 10° is represented by a smaller rectangle than the other light sources.

[0040] <Functional Configuration and Processing Flow> Figure 2 shows an example of the functional configuration of the image processing system 1 according to this embodiment. The image processing system 1 according to this embodiment consists of a start signal output unit 201, an imaging control unit 202, an imaging unit 203, an image processing unit 204, an illumination unit 205, and a transport unit 206. The imaging control unit 202 includes a release signal output unit 207, a sync signal input unit 208, a control unit 209, and a lighting signal output unit 210. The imaging unit 203 includes a release signal input unit 211, a control unit 212, a sync signal output unit 213, and an image acquisition unit 214. The image processing unit 204 includes an inspection image acquisition unit 215, a color / shape inspection unit 216, a gloss inspection unit 217, and an output unit 218.

[0041] The functional units shown in Figure 2 are realized by the hardware shown in Figure 1. Specifically, the function of the start signal output unit 201 is realized by the start output I / F 101 and the transport control device 111. The function of the imaging control unit 202 is realized by the information processing device 102. The function of the imaging unit 203 is realized by the imaging device 103. The function of the image processing unit 204 is realized by the image processing device 104. The function of the illumination unit 205 is realized by the illumination device 108. The function of the transport unit 206 is realized by the transport control device 111, the transport device 112, and the image processing device 104.

[0042] Figure 3 shows a flowchart illustrating an example of information processing performed in the image processing system 1 according to this embodiment. Figure 4 shows the parameter settings and timing charts for each signal used in the information processing performed in the image processing system 1.

[0043] <Overall processing> Figure 3(a) shows a flowchart illustrating the transport process of the object to be inspected according to this embodiment. The process illustrated in Figure 3(a) is a process in which the transport device 112 is operated intermittently. That is, the transport device 112 alternates between operating and stopping. The object to be inspected 113 is transported to the imaging position by the operation of the transport device 112, and is imaged when the transport device 112 is stopped. Once imaging is complete, the object to be inspected 113 is transported from the imaging position to another location by the operation of the transport device 112.

[0044] The process shown in Figure 3(a) is initiated when an instruction to start the operation of the transport device 112 is given (for example, by user operation on the transport device 112).

[0045] In S301, the transport unit 206 operates the transport device 112 to begin transporting the object to be inspected 113.

[0046] In S302, the transport unit 206 determines whether the object to be inspected 113 has been transported to the imaging position (the position of object to be inspected 113-2). For example, the transport unit 206 may use an optical sensor installed near the imaging position to determine whether or not the object to be inspected 113 has been transported to the imaging position. If it is determined that the object to be inspected 113 has been transported to the imaging position, the process proceeds to S303; otherwise, S302 is repeated.

[0047] In S303, the transport unit 206 stops the transport device 112 and stops the transport of the object to be inspected 113.

[0048] In S304, the start signal output unit 201 transmits an imaging start command when the transport unit 206 stops the transport device 112.

[0049] In S305, the transport unit 206 determines whether the inspection of all inspection targets 113 has been completed. For example, the transport unit 206 may determine that the inspection of all inspection targets 113 has been completed when a specified number of inspection targets 113 have been transported. Alternatively, in S302, the transport unit may determine that the inspection of all inspection targets 113 has been completed if no inspection targets 113 appear at the transport position for a predetermined time. If imaging of all inspection targets 113 has been completed, the process shown in Figure 3(a) ends; otherwise, the process returns to S301. Here, it is assumed that the inspection targets 113 whose imaging has been completed are transported to the position of inspection target 113-3 at the time the transport device 112 operates, and then transported outside by the robot arm.

[0050] Figure 3(b) shows a flowchart illustrating the imaging and inspection process of the object to be inspected according to this embodiment. The process shown in Figure 3(b) is executed when the release signal output unit 207 receives an imaging start command. During S306, the transport device 112 is stopped. During this time, the object to be inspected 113 is transported from the outside by a robot arm and placed at the position of the object to be inspected 113-1 on the transport device 112. In this embodiment, the robot arm is controlled by the image processing device 104 via the transport control device 111.

[0051] In S306, the imaging control unit 202 performs synchronous imaging processing by synchronously controlling the imaging unit 203 and the illumination unit 205 to continuously image the object to be inspected 113 at the imaging position. Specifically, the imaging control unit 202 continuously images the surface of the object to be inspected 113 while illuminating multiple light sources constituting the multi-lamp illumination individually or simultaneously in a predetermined order for a predetermined time. As a result, the imaging control unit 202 acquires multiple images with different illumination directions. In addition, in S306, the synchronization signal is monitored by polling as part of the synchronous imaging processing, and the details thereof will be described later with reference to Figures 6 and 7.

[0052] In S307-S308, the image processing unit 204 performs a visual inspection of the object to be inspected 113 based on the image captured in S306.

[0053] In S307, the color / shape inspection unit 216 performs a color / shape inspection of the object to be inspected 113 based on the image captured in S306.

[0054] In S308, the gloss inspection unit 217 performs a gloss inspection on the object to be inspected 113 based on the image captured in S306.

[0055] Here, once the imaging process in S306 is completed, multiple captured images stored in the image acquisition unit 214 are sent to the inspection image acquisition unit 215. The image processing unit 204 in the image processing device 104 performs image processing based on the captured images acquired by the inspection image acquisition unit 215. In this embodiment, the object to be inspected 113 is assumed to be an industrial product such as a home appliance or cosmetics. Here, the image processing unit 204 performs a visual inspection of the object to be inspected 113 based on the multiple captured images acquired. Next, the output unit 218 controls the display to show the inspection result on the display 105. The output unit 218 also notifies the transport unit 206 of the inspection result.

[0056] Here, with reference to Figure 4, the details of the visual inspection process described above will be explained. Figure 4(a) shows an example of a table that defines the parameter settings for controlling multiple light sources and the type of visual inspection when imaging to perform a visual inspection on a single object 113. Figure 4(b) shows a timing chart of each signal used when imaging based on the parameter settings shown in Figure 4(a).

[0057] The parameter settings shown in Figure 4(a) are set by the user via the mouse 106 and keyboard 107, and each count from 0 to 9 corresponds to the waiting time until the light source is switched, the light source to be illuminated, and the purpose of the captured image. In the case shown in Figure 4(a), nine captured images are acquired corresponding to counts from 1 to 9.

[0058] In Figure 4(a), the waiting time for switching the light source represents the time from when the sync signal output from the imaging unit 203 turns ON until the light source is switched. When the sync signal turns ON and the count becomes 1, the system waits for the waiting time for switching the light source, then turns the light signal LS1 OFF and the light signal LS2 ON. When the sync signal turns ON again and the count becomes 2, the system waits in the same manner, then turns the light signal LS2 OFF and the light signal LS3 ON. Furthermore, when the sync signal turns ON 9 more times and the count becomes 9, the system waits in the same manner, then turns the light signal LS9 OFF.

[0059] Here, as shown in Figure 4(a), when the count is between 1 and 8, the eight light sources L1 through L8 light up individually and sequentially. On the other hand, when the count is 9, all eight light sources L33 through L40 light up simultaneously.

[0060] The information processing device 102 according to this embodiment can perform an appearance inspection process of the object to be inspected 113 based on a plurality of captured images, including inspection of the color or shape of the object to be inspected 113, or inspection of its gloss. When performing an appearance inspection process of the surface of the object to be inspected 113, the inspection image acquisition unit 215 first distributes the plurality of captured images to the color / shape inspection unit 216 and the gloss inspection unit 217. The eight light sources L1 to L8, which are lit when the count is between 1 and 8, have a zenith angle of 80° and, as shown in Figure 1(d), can irradiate the object to be inspected 113 from a relatively low angle. The captured images acquired at this time contain almost no specular reflection component and are therefore considered suitable for color / shape inspection of the surface of the object to be inspected 113. In addition, irradiating light from a low angle emphasizes the contrast of shadows caused by the shape of the surface of the object to be inspected 113, so the captured images acquired at this time are considered suitable for shape inspection. Therefore, in the table shown in Figure 4(a), the use (type of visual inspection) of the eight captured images acquired between counts 1 and 8 is set to color / shape inspection. Based on this setting, the inspection image acquisition unit 215 outputs the eight captured images acquired between counts 1 and 8 to the color / shape inspection unit 216.

[0061] On the other hand, the eight light sources L33 to L40, which are lit simultaneously when the count is 9, have a zenith angle of 10°, and as shown in Figure 1(d), they can irradiate the object to be inspected 113 from a relatively high angle. The image acquired at this time contains a large specular reflection component, making it suitable for gloss inspection of the surface of the object to be inspected 113. Therefore, in the table shown in Figure 4(a), gloss inspection is set as the purpose (type of visual inspection) of the single image acquired when the count is 9. Based on this setting, the inspection image acquisition unit 215 outputs the single image acquired when the count is 9 to the gloss inspection unit 217.

[0062] Then, in S307, the color / shape inspection unit 216 performs a color / shape inspection of the surface of the object to be inspected 113 based on the captured images sorted by the inspection image acquisition unit 215, and outputs the result to the output unit 218. Similarly, in S308, the gloss inspection unit 217 performs a gloss inspection of the surface of the object to be inspected 113 based on the captured images sorted by the inspection image acquisition unit 215, and outputs the result to the output unit 218.

[0063] Specifically, the color / shape inspection unit 216 takes the captured image as input and obtains the normal distribution of the surface of the object to be inspected 113 using the illuminance difference stereo method. Next, it visualizes the normal distribution. For example, it visualizes the normal vector at each position by associating it with the RGB value of one pixel. Next, it extracts the defective areas on the surface of the object to be inspected 113 by applying a predetermined spatial filter to the image representing the normal distribution. For example, a DoG (Difference of two Gaussian) filter corresponding to the spatial scale to be extracted is used as the spatial filter. Next, statistical values ​​are obtained from the image to which the spatial filter has been applied. An example of a statistical value is the area of ​​the defective area (number of pixels in the defective area). The statistical values ​​are compared with a predetermined threshold, and if the statistical values ​​do not exceed the predetermined threshold, the inspection is considered passed; otherwise, the inspection is considered failed. Here, the above statistical values ​​are called the abnormality score, and the value used to determine whether the inspection is passed or failed by comparing it with the abnormality score is called the judgment threshold.

[0064] The gloss inspection unit 217 applies a predetermined spatial filter process to the captured image, similar to the inspection performed by the color / shape inspection unit 216, to extract defective areas on the surface of the object, and obtains an abnormality score from the image to which the spatial filter has been applied. The method for determining the abnormality score may differ from that of the color / shape inspection unit 216. The abnormality score is then compared with a judgment threshold, and if the abnormality score does not exceed the judgment threshold, the inspection is considered passed; otherwise, the inspection is considered failed.

[0065] In this manner, the color / shape inspection unit 216 and the gloss inspection unit 217 each perform a pass / fail judgment. If both inspections pass, the final result is output as "inspection passed". On the other hand, if either inspection fails, the final result is output as "inspection failed".

[0066] Furthermore, the inspection process performed here is not limited to this, and any known inspection process using captured images may be employed. For example, the process described in Patent Document 1 may be performed as an inspection method.

[0067] Furthermore, in this embodiment, the robot arm is controlled by the image processing device 104 via the transport control device 111. Specifically, if the inspection result is satisfactory, the image processing device 104 transports the object to be inspected 113-3 to a tray for storing satisfactory items by controlling the robot arm. On the other hand, if the inspection result is unsatisfactory, the image processing device 104 transports the object to be inspected 113-3 to a tray for storing unsatisfactory items by controlling the robot arm.

[0068] Figure 5 shows an example of a UI that displays the results of the inspection process. Figure 5 shows an example of a UI when two inspection targets 113 are inspected in a single inspection. The judgment result 506 and abnormality score 507, described later, represent the inspection result of one inspection target 113(1), and the judgment result 508 and abnormality score 509 represent the inspection result of the other inspection target 113(2).

[0069] Figure 5(a) shows the UI before clicking the reception start button 504, which will be described later according to this embodiment. Figure 5(b) shows the UI after clicking the reception start button 504 according to this embodiment.

[0070] 501 is an input field for the judgment threshold. The value entered in this input field is used as the judgment threshold in the inspection process by the color / shape inspection unit 216 and the gloss inspection unit 217.

[0071] 502 is the date the test started. It displays the date the registration start button 504 was clicked. Nothing is displayed before the registration start button 504 is clicked.

[0072] 503 is the time the test started. It displays the time when the registration start button 504 was clicked. Nothing is displayed before the registration start button 504 is clicked.

[0073] 504 is the reception start button. When the reception start button 504 is clicked, the imaging control unit 202 enters a state where it can receive an imaging start command from the start signal output unit 201. In other words, steps S306 to S308 in Figure 3(b) are executed only when the reception start button 504 is clicked and the system is in a state where it can receive an imaging start command; otherwise, steps S306 to S308 are not executed.

[0074] Button 505 is the acceptance termination button. When the acceptance termination button 505 is clicked, the imaging control unit 202 enters a state where it will not accept an imaging start command from the start signal output unit 201. The user can control whether or not the imaging control unit 202 accepts an imaging start command from the start signal output unit 201 using the acceptance start button 504 and the acceptance termination button 505. This allows the user to control whether or not to perform inspection independently of the state of the transport unit 206.

[0075] 506 is the result of the inspection of object 113(1). It displays the pass / fail judgment result of the inspection process for object 113(1).

[0076] 507 is the degree of abnormality of the object being inspected 113(1). This displays the degree of abnormality obtained from the inspection process of the object being inspected 113(1). In Figure 3(b), the color / shape inspection unit 216 and the gloss inspection unit 217 each determined the degree of abnormality. The degree of abnormality displayed here is the larger of these two degrees. Alternatively, both may be displayed.

[0077] 508 is the result of the inspection of object 113(2). It displays the pass / fail judgment result of the inspection process for object 113(2).

[0078] 509 represents the degree of abnormality of the object being inspected 113(2). This displays the degree of abnormality obtained from the inspection process for the object being inspected 113(2).

[0079] Area 510 is a region that displays messages such as errors that occurred during the inspection process.

[0080] Figure 6 shows a diagram illustrating a method for detecting a synchronous imaging failure that occurs when the polling interval in this embodiment becomes longer than expected. In the example shown in Figure 6, the time during which the sync signal is ON, i.e., the pulse width, is 8 milliseconds. Also, the time from when the sync signal is ON until it is ON again, i.e., the period of the sync signal, is 33 milliseconds.

[0081] In the synchronous imaging process performed in S306, the sync signal is monitored by polling. Figure 6(a) shows how the sync signal is read by polling and how the change from OFF to ON in the sync signal is detected, with triangles indicating the timing of reading the sync signal.

[0082] Figure 6(b) illustrates how polling cannot be performed at predetermined intervals in some sections, resulting in missed changes in the synchronization signal. After detecting the first synchronization signal ON, the synchronization signal reading is interrupted, and the second synchronization signal ON is not detected. As shown in the example in Figure 6(b), if the synchronization signal is read immediately after it turns ON, it is possible to determine whether the synchronization signal ON was missed by checking whether the next synchronization signal reading was performed more than 41 (=8+33) milliseconds later.

[0083] If the sync signal is not turned ON, a phenomenon occurs in S306 where continuous imaging occurs without the light source switching. When synchronized imaging is performed as intended, the images obtained during synchronized imaging are captured with different light sources lit. On the other hand, if the sync signal is not turned ON, the light source does not switch during continuous imaging, and the resulting images will contain multiple images with the same lit light source. This state is referred to here as a failure of synchronized imaging.

[0084] However, in the above method for detecting missed synchronization signal ON, as shown in Figure 6(c), if the synchronization signal reading is resumed immediately before the synchronization signal turns OFF, a missed synchronization signal ON will occur even if the interval until the reading is resumed is 41 milliseconds or less. In this case, the missed synchronization signal ON can be detected by checking whether the interval until polling is resumed is longer than 33 milliseconds. However, if the synchronization signal is read immediately after it turns ON, as shown in Figure 6(d), the synchronization signal ON can be read even though the next synchronization signal reading is performed more than 33 milliseconds later. Therefore, if the criterion is whether the next synchronization signal reading is performed more than 33 milliseconds later, a missed synchronization signal ON will be falsely detected even though the synchronization signal ON was not actually missed.

[0085] Thus, regardless of whether the synchronization signal reading interval used as the criterion for determining whether the synchronization signal is missed is set to 41 milliseconds or 33 milliseconds, a missed synchronization signal ON or a false detection of a missed synchronization signal ON may occur. However, the case in which the synchronization signal ON is read just before it turns OFF, as shown in Figure 6(c), is considered to occur less frequently than the case in which the synchronization signal ON is read between 33 milliseconds and 41 milliseconds, as shown in Figure 6(d). Therefore, when the polling interval is sufficiently short, it is preferable to use 41 milliseconds (synchronization signal period + synchronization signal pulse width) as the synchronization signal reading interval for determining whether the synchronization signal ON has been missed.

[0086] Figure 7 shows a diagram illustrating a method for detecting synchronous imaging failures that occur when the waiting time for light source switching becomes longer than expected. In Figure 7, the time the sync signal is ON, i.e., the pulse width of the sync signal, is 8 milliseconds. The time from when the sync signal is ON until it is ON again, i.e., the period of the sync signal, is 33 milliseconds. The waiting time from when the sync signal is ON until the light source is switched, i.e., the time from when the sync signal is ON until the control signal for the currently lit light source is turned OFF and the control signal for the next light source to be lit is turned ON, is 21 milliseconds. The concept of light source switching is shown in Figure 3(d), and the waiting process during light source switching will be described later in S323 of Figure 3(c). Note that polling of the sync signal is stopped while waiting to switch light sources.

[0087] Figure 7(a) shows how the light source is switched in sync with imaging. If the sync signal is read immediately after it is turned ON, it is possible to detect whether the light source has switched by the time of the next image by checking whether the waiting time until the light source is switched is longer than 33 milliseconds. In Figure 7(a), after the detection of the second sync signal ON, the system started waiting to switch the light source, but because the waiting time was longer than the expected 33 milliseconds, the light source had not switched by the time of the third sync signal ON, i.e., the start of the third image.

[0088] However, in the above method for detecting the failure of switching the spot light source, when the sync signal ON is read immediately before the sync signal becomes OFF, even if the reading interval is 33 milliseconds or less, as shown in Fig. 7(b), a delay in switching the spot light source occurs. In this case, by checking whether the waiting time until switching the spot light source is longer than 25 milliseconds, it is possible to detect that the switching of the spot light source has failed. However, as shown in Fig. 7(c), when the sync signal is read immediately after the sync signal becomes ON, even though the waiting time until switching the spot light source is 25 milliseconds or more, the switching of the light source is in time. Therefore, if it is determined based on whether the waiting time until switching the spot light source is longer than 25 milliseconds, although the switching of the spot light source is actually in time, a failure in switching the spot light source will be erroneously detected.

[0089] Thus, whether the waiting time of the control signal used as the criterion for determining the failure of switching the spot light source is set to 33 milliseconds or 25 milliseconds, it is possible to miss detecting the failure of switching the spot light source or erroneously detect the failure of switching the spot light source. However, the case where the sync signal ON is read immediately before the sync signal becomes OFF as shown in Fig. 7(b) is considered to occur less frequently than the case where the spot light source is switched between 25 milliseconds and 33 milliseconds later as shown in Fig. 7(c). Therefore, in the method for detecting the failure of switching the spot light source based on the waiting time of the control signal, when the polling interval is sufficiently short, it is preferable to use 33 milliseconds (the period of the sync signal) as the waiting time for determining the failure of switching the spot light source.

[0090] <Details of the processing in S306> Next, referring to Fig. 3(c), the details of the processing in S306 will be described. S306 is a procedure for synchronous imaging control that synchronizes the timings of imaging and lighting the light source. S306 includes a procedure for detecting that a discrepancy has occurred between the timings of imaging and lighting the light source, that is, for detecting the missed detection of the sync signal ON.

[0091] In S306, the imaging control unit 202 controls the imaging unit 203 and the illumination unit 205 by performing the processing shown in Figure 3(c) to image the object at the imaging position. The processing S309 to S328 shown in Figure 3(c) is performed by the program code read by the information processing device 102.

[0092] After stopping the transport of the object in S303, the start signal output unit 201 sends an imaging start command to the release signal output unit 207 of the imaging control unit 202 in S304. When this imaging start command is input to the release signal output unit 207, the process shown in Figure 3(c) begins.

[0093] In S309, the control unit 209 of the imaging control unit 202 sets the sync signal count to 0. The sync signal count corresponds to the count of still images obtained by continuous imaging. Hereafter, when simply referred to as "count," it refers to this sync signal count. Figure 4(b) shows the count being set to 0 at time t0, and the count changing from 0 to 9.

[0094] In S310, the release signal output unit 207 sends a release signal to the release signal input unit 211. In Figure 4(b), time t R The process of the release signal switching from OFF to ON is shown. And at time t R The release signal remains ON during the period from [time] to time t9. During this period, the imaging unit 203 performs continuous imaging and acquires nine images.

[0095] In this embodiment, the ON / OFF status of the release signal is controlled based on the number of times the sync signal output from the imaging unit 203 is turned ON. Here, the imaging control unit 202 counts the number of times the sync signal output from the imaging unit 203 is turned ON, and when the count reaches a predetermined number, it can turn the release signal OFF. For example, in the case of Figure 4(b), the imaging control unit 202 turns the release signal OFF when the count reaches 9. With this process, it is possible to stop continuous imaging when the number of still images acquired reaches a predetermined number.

[0096] Here, with reference to Figure 3(d), the imaging process in response to the synchronization signal by the imaging unit 203 will be explained. Figure 3(d) is a flowchart showing an example of the imaging process in the imaging unit 203. Each of the processes S329 to S332 shown in Figure 3(d) is executed by the program code read out by the imaging device 103.

[0097] First, time t shown in Figure 4(b) R In this configuration, the release signal input unit 211 acquires a release signal from the release signal output unit 207. When the release signal is turned ON, the control unit 212 controls the image acquisition unit 214 and the sync signal output unit 213 to start continuous imaging. Continuous imaging continues as long as the release signal is ON, and stops when the release signal turns OFF.

[0098] time t R When the release signal is turned ON, the image acquisition unit 214 starts capturing images at time t1. R The period from to time t1 is the release time lag, and when a consumer camera is used as the imaging device 103, time t R The period from [start time] to time t1 varies depending on the circumstances.

[0099] In S329, the control unit 212 controls the sync signal output unit 213 to turn on the sync signal at the same time as the imaging that starts at time t1. That is, the sync signal output unit 213 outputs a sync signal to the sync signal input unit 208 of the imaging control unit 202 at time t1. Note that the timing for starting imaging here is the timing for starting exposure.

[0100] In S330, the control unit 212 controls the image acquisition unit 214 to expose the object to be inspected 113 for a period corresponding to the set value of the shutter speed.

[0101] In S331, the control unit 212 controls the sync signal output unit 213 to turn off the sync signal at the timing when imaging is finished. In other words, in the process shown in Figure 3(d), the sync signal is ON during the exposure period and OFF during the non-exposure period.

[0102] In S332, the control unit 212 determines whether the release signal is ON or OFF. If the release signal is ON, the process returns to S329 and continuous imaging continues. Otherwise (when the release signal turns OFF), the process in Figure 3(d) ends and continuous imaging stops.

[0103] Figure 4(b) shows the timing chart of the release signal, sync signal, and illumination signal during continuous imaging. In the example shown in Figure 4(b), the period during which the release signal is ON is at time t R From time t9. During this time, continuous imaging is performed and nine still images are captured. The lighting signal LS1 is at time t RThe lights switch to ON. When the sync signal is turned ON, the lighting signal LS1 switches to OFF after a predetermined waiting time, and at the same time, the lighting signal LS2 switches to ON. Subsequently, the ON / OFF states of the lighting signals switch sequentially in the same manner, and on the 9th sync signal ON, the lighting signal LS9 switches to OFF, and the series of synchronized imaging ends. In the case of Figure 4(a), the predetermined waiting time from the sync signal ON to the switching of the lighting source is set to 21 milliseconds.

[0104] Now, let's return to the explanation of the processing flow by the imaging control unit 202 (Figure 3(c)). In the processing flow explained in Figure 3(c), the imaging control unit 202 images the surface of the object to be inspected 113 while lighting up multiple light sources constituting the multi-lamp illumination individually or simultaneously, in a predetermined order and for a predetermined time. As a result, multiple images with different illumination directions are acquired.

[0105] In S311, the control unit 209 controls the lighting signal output unit 210 to turn on the lighting signal LS1.

[0106] In S312, the control unit 209 sets 0 as the time when the synchronization signal was last turned ON.

[0107] S313 to S328 is a loop process that is repeated. In S313, a determination is made as to whether to continue the loop process. The control unit 209 determines whether the count is less than a predetermined count. In this embodiment, as shown in Figure 4(a), nine images are acquired corresponding to counts from 1 to 9. Therefore, the predetermined count used in S313 is 9. If the determination in S313 is YES, the process proceeds to S314; otherwise, the process terminates.

[0108] In S314, the synchronization signal input unit 208 reads the state of the synchronization signal, i.e., whether it is ON or OFF. At this time, the synchronization signal input unit 208 records the time when the synchronization signal was read.

[0109] In S315, the imaging control unit 202 determines that the count is not 0. If the determination is YES, the process proceeds to S316; otherwise, the process proceeds to S318. The reason this branch is necessary is that the determination made in S316 refers to the time when the previous sync signal was turned ON. If the count is 0, a temporary value is set for the time when the previous sync signal was turned ON (see S312). Since there is no need to make the determination in S316 using a temporary value, the determination in S316 is skipped if the count is 0.

[0110] In S316, the imaging control unit 202 calculates the elapsed time from the time the last sync signal ON was detected to the first reading time when the reading of the sync signal was resumed, and determines whether that elapsed time is longer than the period + pulse width of the sync signal. The period of the sync signal from the sync signal output unit 213 is a different value depending on the model of the imaging device 103. For models that can take 30 continuous images per second, the period of the sync signal is 1 / 30 second (≒33 milliseconds). The pulse width of the sync signal output from the imaging device 103 is the exposure time, and is a value that depends on the shutter speed setting of the imaging device 103. If the shutter speed is set to 1 / 125 second, the exposure time, i.e., the pulse width, is 1 / 125 second (=8 milliseconds). If the determination is YES, the process proceeds to S321; otherwise, the process proceeds to S322. Note that the elapsed time used for the determination in S316 may be the elapsed time from the last sync signal reading time to the latest sync signal reading time.

[0111] In S317, the control unit 209 outputs a message to the output unit 218 indicating that it may have missed the synchronization signal ON. The output unit 218 then controls the display 105 to display the message, for example, in the message area 510 shown in Figure 5.

[0112] In S318, the control unit 209 determines whether the sync signal read in S314 is ON. As mentioned above, in this embodiment, the sync signal changes from OFF to ON at the timing when still image capture begins. If the determination is YES, the process proceeds to S319; otherwise, the process returns to S314.

[0113] In S319, the control unit 209 increments the count.

[0114] In S320, the control unit 209 determines whether the count is equal to the default count. If the count is equal to the default count, the process proceeds to S321; otherwise, the process proceeds to S322.

[0115] In S321, the control unit 209 sends an imaging stop command to the release signal output unit 207. When the imaging stop command is sent, the release signal output unit 207 turns the release signal OFF. As shown in Figure 4(b), at time t9, the count becomes 9 at the moment the sync signal changes from OFF to ON. At this timing, the control unit 209 sends an imaging stop command to the release signal output unit 207. The release signal output unit 207 then turns the release signal OFF in accordance with the imaging stop command. At this timing, the imaging unit 203 stops continuous imaging.

[0116] In S322, the imaging control unit 202 sets the current time as the time when the sync signal was last turned ON.

[0117] In S323, the imaging control unit 202 waits for the duration of the light source switching time shown in Figure 4(a) after the last time the sync signal was turned ON. If the information processing device 102 is a PC with a general-purpose OS installed that is not for real-time control, it may not be able to wait for the specified time precisely because other processes are occupying computing resources. Therefore, the actual length of the waiting time is also measured.

[0118] In S324, the imaging control unit 202 determines whether the waiting time since the last time the sync signal was turned ON in S323 is longer than the period of the sync signal. If the determination is YES, the process proceeds to S325; otherwise, the process proceeds to S326.

[0119] In S325, the control unit 209 outputs a message to the output unit 218 indicating that there may not have been enough time to switch the light source. The output unit 218 displays the message on the display 105. For example, it is displayed in the message area 510 in Figure 5.

[0120] In S326, the control unit 209 controls the lighting signal output unit 210 to turn off the lighting signal corresponding to the count. For example, if the count is 3, the lighting signal LS3 is turned off.

[0121] In S327, the control unit 209 determines whether the count is less than a predetermined count. If the determination is YES, the process proceeds to S328; otherwise, the process proceeds to S313.

[0122] In S328, the control unit 209 controls the lighting signal output unit 210 to turn on the lighting signal corresponding to count + 1. For example, if the count is 3, the lighting signal LS4 is turned on.

[0123] In this embodiment, two types of determinations are made in S316 and S317, and in S324 and S325. However, only one of these processes may be performed. However, if only one is performed, the detection accuracy of synchronized imaging failure will be lower compared to when both are performed.

[0124] <Effects of this embodiment> According to the above-described embodiment, it is possible to detect discrepancies in the timing of imaging and light source illumination, which are likely to occur when performing synchronous imaging control in an information processing device using a general-purpose OS that is not dedicated to real-time control, by reducing the amount of computation and at low cost.

[0125] In this embodiment, if there is a possibility that the synchronization signal has been missed, a message to that effect is displayed on the display. However, if there is a possibility that the synchronization signal has been missed, the synchronized imaging may be stopped at that point. This prevents unnecessary inspection processing using images obtained from improperly synchronized imaging and further reduces the amount of computation.

[0126] Furthermore, in this embodiment, changes in the sync signal were monitored by polling, but changes in the sync signal may also be monitored by interrupts. Compared to polling, interrupts are less likely to cause the phenomenon of missing the sync signal ON due to other processes occupying computing resources. On the other hand, even when detecting the sync signal ON by an interrupt, computing resources may be occupy by other processes between S316 and S328, similar to the case of polling, preventing the start of S316 to S328 and causing the timing of imaging and light source illumination to become mismatched. Therefore, even when monitoring changes in the sync signal by interrupts, the processing in S320 and S324 is still effective.

[0127] [Embodiment 2] In Embodiment 1, synchronized imaging was performed by switching the lighting state of the light source of the illumination device 108 in accordance with the synchronization signal output from the imaging device 103. In this embodiment, synchronized imaging is performed by the illumination device 108 outputting a synchronization signal in accordance with the lighting of the light source, and the imaging device 103 performing imaging in accordance with the synchronization signal. The pulse width of the synchronization signal output by the illumination device 108 is set to any width shorter than the lighting interval of the light source.

[0128] <Hardware Configuration> Figure 8 is a block diagram showing an example of the hardware configuration of the image processing system 1, including the information processing device 102 according to this embodiment. Parts similar to those in Figure 1 are omitted from the explanation.

[0129] <Hardware configuration of the information processing device> The information processing device 102 includes a control unit 114, a release output interface 116, a sync input interface 117, and a USB interface 118. Each of these functional units is connected to each other via an internal bus so that they can communicate with one another.

[0130] The release output interface 116 outputs a release signal to the release input interface 120 of the imaging device 103, instructing the imaging of the object (subject). The control unit 114 turns on the release signal to be output to the imaging device 103 via the release output interface 116 each time the input sync signal is turned ON. In Embodiment 1, the imaging device 103 started continuous imaging when the input release signal turned ON. In this embodiment, where imaging is performed in accordance with the timing of the light source illumination, the imaging device 103 performs a single image when the release signal is turned ON in order to perform imaging in accordance with the timing of the light source illumination. The release output interface 116 turns OFF after a predetermined period of time after turning on the release signal.

[0131] The sync input interface 117 acquires the sync signal output from the lighting device 108 via the sync output interface 122 provided by the lighting device 108, in accordance with the timing of switching the light source.

[0132] <Hardware configuration of the imaging device> The imaging device 103 includes a release input I / F 120, an imaging optical system 121, an image processing engine 123, a USB I / F 124, and a control unit 125. Each of these functional units is connected to each other via an internal bus so as to be able to communicate with one another.

[0133] The release input interface 120 acquires the release signal output from the release output interface 116 provided by the information processing device 102.

[0134] The control unit 125 controls the imaging optical system 121, which consists of a lens or an image sensor, etc., when a release signal is input, to image the object to be inspected 113-2. Specifically, the control unit 125 takes a single image when the release signal is turned ON.

[0135] <Hardware configuration of the lighting system> The lighting device 108 includes a light source 109, a start input interface 115, a sync output interface 122, and a control unit 801. Each of these functional units is connected to each other via an internal bus so as to be able to communicate with one another.

[0136] The start input interface 115 acquires the imaging start signal output from the start output interface 101.

[0137] When the imaging start signal is input, the control unit 801 starts executing a program code pre-stored in its ROM. The control unit 801 lights up the multiple light sources 109 that make up the multi-lamp illumination individually or simultaneously, in a predetermined order, for a predetermined time. It also outputs a synchronization signal in accordance with the timing of when the light sources are turned on. Specifically, the control unit 801 outputs a synchronization signal to the synchronization input I / F 117 of the information processing device 102 via the synchronization output I / F 122.

[0138] <Processing Flow> In Embodiment 1, the light source was switched in accordance with the sync signal output simultaneously with imaging. In this embodiment, imaging is performed in accordance with the sync signal output simultaneously with the switching of the light source. Specifically, the illumination device 108 lights the light sources in a predetermined order for a predetermined time that is at least longer than the exposure time, turns on the sync signal when the light source is turned on, and turns it off at an arbitrary time shorter than the interval between the light source lights. In the synchronous imaging control shown in Figure 3, changes in the sync signal are monitored by polling, similar to Embodiment 1. When it is detected that the sync signal has turned ON, the device waits for the waiting time until imaging starts, which is given as a parameter, then turns on the release signal, and turns it off after a predetermined time.

[0139] The detection of missed synchronization signal ON in S316 of Embodiment 1 can be performed in the same manner in this embodiment. Also, the waiting time from synchronization signal ON to switching of the light source in S323 of Embodiment 1 is read as the waiting time from synchronization signal ON to the start of imaging in this embodiment. Furthermore, since imaging must be completed before the light source switches, the waiting time until the start of imaging must be less than or equal to the period of the synchronization signal minus the exposure time. That is, in S323 of this embodiment, if the waiting time from synchronization signal ON to the start of imaging is longer than the period of the synchronization signal minus the exposure time, it is determined that synchronized imaging has failed. Also, in S324 of this embodiment, a message is output indicating that the completion of imaging was not completed in time for the switching of the light source.

[0140] <Effects of this embodiment> With this configuration, even in a general-purpose OS, failures in synchronous imaging control, which performs imaging in accordance with the timing of lighting device activation, can be detected at low cost.

[0141] [Embodiment 3] In Embodiment 1, synchronized imaging was performed by switching the illumination state of the light source of the illumination device 108 in accordance with the synchronization signal output from the imaging device 103. In this embodiment, a synchronization signal generator specializing in generating synchronization signals outputs a synchronization signal, and the imaging device 103 and the illumination device 108 perform synchronized imaging by taking images and switching the illumination state of the light source in accordance with the synchronization signal.

[0142] <Hardware Configuration> Figure 9 is a block diagram showing an example of the hardware configuration of the image processing system 1, including the information processing device 102 according to this embodiment. Parts similar to those in Figure 1 are omitted from the explanation.

[0143] <Hardware configuration of the information processing device> The information processing device 102 includes a control unit 114, a release output I / F 116, a sync input I / F 117, a USB I / F 118, and a lighting output I / F 119. Each of these functional units is connected to each other via an internal bus so that they can communicate with one another.

[0144] The release output interface 116 outputs a release signal to the release input interface 120 of the imaging device 103, instructing the imaging of the object (subject). The control unit 114 turns on the release signal to be output to the imaging device 103 via the release output interface 116 each time the input sync signal is turned ON. In Embodiment 1, the imaging device 103 started continuous imaging at the moment the input release signal was turned ON. In this embodiment, imaging is performed in accordance with the sync signal generated by the sync signal generator 901, so the imaging device 103 performs a single image at the moment the release signal is turned ON. The release output interface 116 turns OFF after a predetermined period of time after turning on the release signal.

[0145] The sync input I / F 117 acquires the sync signal from the sync output I / F 122 provided by the sync signal generator 901. The control unit <Hardware configuration of the imaging device> The imaging device 103 includes a release input I / F 120, an imaging optical system 121, an image processing engine 123, a USB I / F 124, and a control unit 125. Each of these functional units is connected to each other via an internal bus so as to be able to communicate with one another.

[0146] The release input interface 120 acquires the release signal output from the release output interface 116 provided by the information processing device 102.

[0147] The control unit 125 controls the imaging optical system 121, which consists of a lens or an image sensor, etc., when a release signal is input, to image the object to be inspected 113-2. Specifically, the control unit 125 takes a single image when the release signal is turned ON.

[0148] <Hardware configuration of the synchronous signal generator> The synchronization signal generator 901 includes a start input interface 115, a synchronization output interface 122, and a control unit 902. Each of these functional units is connected to each other via an internal bus so that they can communicate with one another.

[0149] The start input interface 115 acquires the imaging start signal output from the start output interface 101.

[0150] When the imaging start signal is input, the control unit 902 begins executing the program code pre-stored in its ROM. The control unit 902 also outputs a sync signal to synchronize imaging and light source illumination. Specifically, the control unit 902 outputs a sync signal at regular intervals to the sync input I / F 117 of the information processing device 102 via the sync output I / F 122.

[0151] <Processing Flow> In Embodiment 1, the light source was switched in accordance with the sync signal output simultaneously with imaging. In this embodiment, imaging and the switching of the light source are performed in accordance with the sync signal output by the sync signal generator 901. Specifically, the sync signal generator 901 turns the sync signal ON. In the synchronous imaging control shown in Figure 3, the change in the sync signal is monitored by polling, similar to Embodiment 1. When it is detected that the sync signal has turned ON, the system waits for the waiting time for the light source switching given as a parameter, then turns the light source corresponding to the count OFF, and turns the light source corresponding to count + 1 ON (same as Embodiment 1). In addition, the system waits for the waiting time until imaging starts, given as a parameter, then turns the release signal ON, and turns it OFF after a predetermined time.

[0152] The detection of missed sync signals in S316 of Embodiment 1 can be performed in the same manner in this embodiment. Furthermore, in S324 and S325, it is sufficient to check whether the actual waiting time for the release signal waiting time and the lighting signal waiting time is longer than the predetermined time described below. Note that the release signal waiting time is the waiting time from when the sync signal is turned ON until the start of imaging, and the lighting signal waiting time is the waiting time from when the sync signal is turned ON until the switching of the lighting source. In S324, if the waiting time until the start of imaging is longer than the waiting time until the switching of the lighting source, it is determined that synchronous imaging has failed, and the process proceeds to S325 to output a message to that effect. Also, in S324, if the waiting time until the switching of the lighting source is longer than the period of the sync signal + the pulse width of the sync signal or the period of the sync signal + the waiting time until the start of imaging, it is determined that synchronous imaging has failed, and the process proceeds to S325 to output a message to that effect. The predetermined time corresponding to the waiting time until the switching of the lighting source is set to be less than or equal to the period of the sync signal + the pulse width of the sync signal when the waiting time until the start of imaging is longer than the pulse width of the sync signal. The predetermined time corresponding to the waiting time until the switching of the light source is set to be less than or equal to the period of the sync signal plus the waiting time until imaging starts, when the waiting time until imaging starts is shorter than the pulse width of the sync signal.

[0153] <Effects of this embodiment> With this configuration, even in a general-purpose OS, it is possible to detect failures in synchronous imaging control, which synchronizes imaging and light source illumination in accordance with the sync signal generated by the sync signal generator, at low cost.

[0154] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0155] This disclosure includes the following configurations and methods: [Configuration 1] A reading means for reading changes in a synchronization signal for generating a control signal to synchronize the turning on of an illumination device for illuminating an object to be inspected with the imaging device for imaging the object to be inspected, If the time between detecting that the synchronization signal has been turned ON and the next reading of the change in the synchronization signal is longer than a predetermined time, an output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed. An information processing device characterized by comprising: [Configuration 2] The reading means stops reading the change in the synchronization signal from the time it detects that the synchronization signal has been turned ON until at least one of the imaging device and the illumination device is turned on. The information processing device according to configuration 1, characterized by the above. [Configuration 3] The predetermined time is the sum of the period of the synchronization signal and the pulse width of the synchronization signal. An information processing apparatus according to configuration 1 or 2, characterized by the above. [Structure 4] The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the sync signal output when the imaging device performs imaging, The generation means generates the control signal for controlling the lighting of the lighting device in synchronization with the synchronization signal. An information processing device according to any one of configurations 1 to 3. [Composition 5] The lighting device comprises a plurality of light sources with different irradiation angles relative to the installation surface of the object to be inspected. The generating means generates the control signal for switching the light source to be lit in the lighting device. The information processing apparatus according to configuration 4, characterized by the features described above. [Composition 6] The generation means generates the control signal for switching the light source to be lit in the lighting device after detecting that the synchronization signal has been turned ON. The output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed if the time from when the synchronization signal is turned ON in the control signal until a change occurs that instructs the switching of the light source to be lit is longer than the period of the synchronization signal. The information processing apparatus according to configuration 3 or 4, characterized by the above. [Composition 7] The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the synchronization signal output when the lighting device is turned on, The generation means generates a control signal for controlling imaging of the imaging device in synchronization with the synchronization signal. An information processing device according to any one of configurations 1 to 3. [Structure 8] The generation means generates the control signal to cause the imaging device to take an image after detecting that the sync signal has been turned ON. The output means outputs a signal indicating that synchronization between the imaging of the imaging device and the lighting of the illumination device has failed if the time from the detection of the synchronization signal being turned ON in the control signal until a change instructing the start of imaging occurs is longer than the time obtained by subtracting the exposure time of the imaging from the period of the synchronization signal. The information processing apparatus according to configuration 7, characterized by the features described above. [Composition 9] The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the synchronization signal output by the synchronization signal generator, The generation means generates a control signal for controlling imaging of the imaging device and a control signal for controlling the lighting of the illumination device in synchronization with the synchronization signal. An information processing device according to any one of configurations 1 to 3. [Configuration 10] The generation means generates the control signal for causing the imaging device to take an image after detecting that the synchronization signal has been turned ON, and the control signal for switching the light source to be turned on in the illumination device after the imaging is completed. The output means outputs a signal indicating that synchronization between the imaging of the imaging device and the illumination of the lighting device has failed if the time from the detection of the sync signal being turned ON in the control signal until a change instructing the start of imaging occurs is longer than the time from the detection of the sync signal being turned ON in the control signal until a change instructing the switching of the light source to be illuminated occurs, or if the time from the detection of the sync signal being turned ON in the control signal until a change instructing the switching of the light source to be illuminated occurs is longer than the sum of the period of the sync signal and the time from the detection of the sync signal being turned ON until a change instructing the start of imaging occurs. The information processing apparatus according to configuration 9, characterized by the features described therein. [Composition 11] The information processing apparatus according to any one of configurations 1 to 7, further comprising a display control means that causes a display means to display that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, based on a signal output by the output means. [Composition 12] If the output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, the reading means shall cease reading the change in the synchronization signal. An information processing device according to any one of configurations 1 to 11, characterized by the above. [Composition 13] If the output means outputs a signal indicating that the synchronization of the imaging and the illumination has failed, the generation means shall cease generating the control signal. An information processing device according to any one of configurations 1 to 12, characterized by the above. [Composition 14] A step of reading a change in a synchronization signal for generating a control signal to synchronize the turning on of an illumination device that illuminates the object to be inspected with the imaging device that captures the object to be inspected, If the time between detecting that the synchronization signal has been turned ON and the next reading of the change in the synchronization signal is longer than a predetermined time, the step of outputting a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, An information processing method characterized by comprising: [Composition 15] A program for causing a computer to function as an information processing device as described in any one of items 1 to 13.

Claims

1. A reading means for reading changes in a synchronization signal for generating a control signal to synchronize the turning on of an illumination device for illuminating an object to be inspected with the imaging device for imaging the object to be inspected, If the time between detecting that the synchronization signal has been turned ON and the next reading of the change in the synchronization signal is longer than a predetermined time, an output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed. An information processing device characterized by comprising:

2. The reading means stops reading the change in the synchronization signal from the time it detects that the synchronization signal has been turned ON until at least one of the imaging device taking an image and the illumination device turning on is performed. The information processing apparatus according to feature 1.

3. The predetermined time is the sum of the period of the synchronization signal and the pulse width of the synchronization signal. The information processing apparatus according to feature 1.

4. The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the sync signal output when the imaging device performs imaging, The generation means generates the control signal for controlling the lighting of the lighting device in synchronization with the synchronization signal. The information processing apparatus according to feature 1.

5. The lighting device comprises a plurality of light sources with different irradiation angles relative to the installation surface of the object to be inspected. The generating means generates the control signal for switching the light source to be lit in the lighting device. The information processing apparatus according to feature 4.

6. The generation means generates the control signal for switching the light source to be lit in the lighting device after detecting that the synchronization signal has been turned ON. The output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed if the time from when the synchronization signal is turned ON in the control signal until a change occurs that instructs the switching of the light source to be lit is longer than the period of the synchronization signal. The information processing apparatus according to claim 3.

7. The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the synchronization signal output when the lighting device is turned on, The generation means generates a control signal for controlling imaging of the imaging device in synchronization with the synchronization signal. The information processing apparatus according to feature 1.

8. The generation means generates the control signal to cause the imaging device to take an image after detecting that the synchronization signal has been turned ON. The output means outputs a signal indicating that synchronization between the imaging of the imaging device and the lighting of the illumination device has failed if the time from when the synchronization signal is detected to when a change instructing the start of imaging occurs in the control signal is longer than the time obtained by subtracting the exposure time of the imaging from the period of the synchronization signal. The information processing apparatus according to feature 7.

9. The system further comprises a generation means for generating the aforementioned control signals, The reading means reads the synchronization signal output by the synchronization signal generator, The generation means generates a control signal for controlling imaging of the imaging device and a control signal for controlling the lighting of the illumination device in synchronization with the synchronization signal. The information processing apparatus according to feature 1.

10. The generation means generates the control signal for causing the imaging device to take an image after detecting that the synchronization signal has been turned ON, and the control signal for switching the light source to be lit in the illumination device after the imaging is completed. The output means outputs a signal indicating that synchronization between the imaging of the imaging device and the illumination of the lighting device has failed if the time from the detection of the sync signal being turned ON in the control signal until a change instructing the start of imaging occurs is longer than the time from the detection of the sync signal being turned ON in the control signal until a change instructing the switching of the light source to be illuminated occurs, or if the time from the detection of the sync signal being turned ON in the control signal until a change instructing the switching of the light source to be illuminated occurs is longer than the sum of the period of the sync signal and the time from the detection of the sync signal being turned ON until a change instructing the start of imaging occurs. The information processing apparatus according to feature 9.

11. The information processing apparatus according to claim 1, further comprising a display control means that causes a display means to display that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, based on a signal output by the output means.

12. If the output means outputs a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, the reading means shall cease reading the change in the synchronization signal. The information processing apparatus according to claim 1, characterized by the following:

13. If the output means outputs a signal indicating that the synchronization of the imaging and the illumination has failed, the generation means shall cease generating the control signal. The information processing apparatus according to claim 1, characterized by the following:

14. A step of reading a change in a synchronization signal for generating a control signal to synchronize the turning on of an illumination device that illuminates the object to be inspected with the imaging device that captures the object to be inspected, If the time between detecting that the synchronization signal has been turned ON and the next reading of the change in the synchronization signal is longer than a predetermined time, the step of outputting a signal indicating that the synchronization between the imaging of the imaging device and the lighting of the illumination device has failed, An information processing method characterized by comprising:

15. A program for causing a computer to function as an information processing device according to any one of claims 1 to 12.

Images