Visual inspection support device

Abstract

To provide a technique capable of specifying a visual position on a surface of an inspection target without using a camera capable of eye tracking.SOLUTION: A visual inspection support device for supporting a visual inspection of an inspection target by an inspector comprises: a light source which irradiates light toward the inspection target from near the eyes of the inspector; an imaging unit which is disposed near an illumination unit that illuminates the inspection target and images reflected light that has been irradiated from the light source and reflected by the inspection target; a specification unit which specifies a trajectory of a reflection position of the reflected light on the surface of the inspection target by using an image captured by the imaging unit; and a determination unit which determines, with reference to the specified trajectory, an inspected region and an uninspected region of the surface.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a visual inspection support device. [Background technology]

[0002] For inspection objects where defects that cannot be captured by a camera may occur, visual inspection by an inspector may be performed. As a technology used for visual inspection by an inspector, for example, Patent Document 1 discloses a visual inspection work management system that measures the time during which the inspector's gaze position signal is within the video signal of the inspection area in order to prevent the outflow of defective products due to inspection errors. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-163380 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, a gaze position signal indicating the gaze position on the surface of an object to be inspected is determined by a camera capable of so-called eye tracking. Such cameras are generally expensive, and in order to accurately determine the gaze position, they must be periodically calibrated. Furthermore, the calibration may involve complicated work. For this reason, there has been a demand for a technology that can determine the gaze position on the surface of an object to be inspected without using a camera capable of eye tracking.

[0005] The present invention has been made to solve at least some of the above-mentioned problems, and aims to provide a technology that can identify the visual position on the surface of an object to be inspected without using a camera capable of eye tracking. [Means for solving the problem]

[0006] The present invention has been made to solve at least part of the above-mentioned problems, and can be realized in the following aspects.

[0007] (1) According to one aspect of the present invention, there is provided a visual inspection support device for supporting an inspector in visually inspecting an object to be inspected, the visual inspection support device including: a light source that irradiates light from near the eyes of the inspector toward the object to be inspected; an image capture unit that is disposed near an illumination unit that illuminates the object to be inspected and captures the light that is irradiated from the light source and then reflected by the object to be inspected; an identification unit that identifies a trajectory of the reflected light on the surface of the object to be inspected using an image captured by the image capture unit; and a discrimination unit that identifies inspected areas and uninspected areas on the surface by referring to the identified trajectory.

[0008] According to this configuration, the light source emits light from near the inspector's eyes, and the imaging unit is located near the illumination unit. In this case, the path of light emitted from the illumination unit, reflected by the object under inspection, and incident on the inspector's eyes generally coincides with the path of light emitted from the light source, reflected by the object under inspection, and incident on the imaging unit. Therefore, the reflection position of the reflected light (light emitted from the light source and reflected by the object under inspection) on the surface of the object under inspection identified using the image captured by the imaging unit can be considered to be the visual position observed by the inspector. Therefore, by using this image, the visual position on the surface of the object under inspection can be identified without using an eye-tracking camera. Furthermore, by discriminating between inspected and uninspected areas by referring to the trajectory of the reflection position that can be considered as the visual position, inspected and uninspected areas can be accurately distinguished.

[0009] (2) According to another aspect of the present invention, there is provided a visual inspection support device for supporting an inspector in visually inspecting an object to be inspected, the visual inspection support device including: an infrared light source that irradiates infrared light from near the eyes of the inspector toward the object to be inspected; a first image capture unit that is attached with a filter that limits incident light to only infrared light and is positioned near an illumination unit that illuminates the object to be inspected and that captures the reflected infrared light that is irradiated from the infrared light source and reflected by the object to be inspected; a second image capture unit that is positioned near the first image capture unit and that captures the reflected light that is irradiated from the illumination unit and reflected by the object to be inspected; an identification unit that identifies a trajectory of reflection positions of the reflected infrared light on the surface of the object to be inspected by using an image captured by the first image capture unit, by superimposing the trajectory on the surface of the object to be inspected by using an image captured by the second image capture unit; and a discrimination unit that identifies inspected areas and uninspected areas on the surface by referring to the identified trajectory.

[0010] According to this configuration, the infrared light source emits infrared light from near the inspector's eyes, and the first image capture unit is located near the illumination unit. The path of the light emitted from the illumination unit, reflected by the object under inspection, and incident on the inspector's eyes generally coincides with the path of the infrared light emitted from the infrared light source, reflected by the object under inspection, and incident on the first image capture unit. Therefore, the reflection position of the reflected infrared light on the surface of the object under inspection identified using the image captured by the first image capture unit can be considered to be the visual position of the inspector, and the visual position can be identified using the image. Furthermore, since the second image capture unit is located near the first image capture unit, the visual position on the surface of the object under inspection identified using the image captured by the second image capture unit can be superimposed on the identified reflection position of the reflected infrared light, thereby identifying the visual position on the surface of the object under inspection without using an eye-tracking camera. Furthermore, by discriminating between inspected and uninspected areas by referring to the trajectory of the reflection position that can be considered as the visual position, inspected and uninspected areas can be accurately distinguished.

[0011] (3) The visual inspection support device of the above aspect may further include a notification unit that notifies the user of the presence of the uninspected area when the uninspected area exists on the surface. With this configuration, the notification unit notifies the presence of an uninspected area, reducing the possibility that an inspector will end the visual inspection without inspecting the uninspected area, thereby improving the accuracy of detecting defective inspection objects during visual inspection.

[0012] The present invention can be realized in various forms, for example, in the form of a visual inspection support device, a visual inspection support system, a visual inspection support method, a method for controlling visual inspection support operations, a computer program for executing these devices and methods, a server device for distributing this computer program, a non-transitory storage medium on which a computer program is stored, etc. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is an explanatory diagram illustrating the configuration of a visual inspection support device according to a first embodiment of the present invention; [Figure 2] FIG. [Figure 3] FIG. 10 is an explanatory diagram of the relationship between the position visually observed by an inspector and the reflection position. [Figure 4] 10 is a flowchart showing the procedure of a support process executed by a control unit. [Figure 5] FIG. 10 is an explanatory diagram of a method for distinguishing between inspected and uninspected areas. [Figure 6] FIG. 10 is an explanatory diagram illustrating the configuration of a visual inspection support device according to a second embodiment of the present invention. [Figure 7] 3A and 3B are explanatory diagrams of images captured by a first imaging unit and a second imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0014] First Embodiment FIG. 1 is an explanatory diagram illustrating the configuration of a visual inspection support device 1 according to a first embodiment of the present invention. FIG. 1 illustrates mutually orthogonal X, Y, and Z axes. These X, Y, and Z axes are common to all figures following FIG. 1. The visual inspection support device 1 is a device that supports an inspector in visually inspecting an object under inspection OB. In detail, the visual inspection support device 1 supports the inspector so that the visual inspection is not completed without leaving any uninspected areas on the surface of the object under inspection OB.

[0015] FIG. 2 is an explanatory diagram of visual inspection. An illumination unit 15 is attached above a workbench where visual inspection is performed and illuminates an inspection object OB. In visual inspection, the inspection object OB illuminated by the illumination unit 15 is inspected by an inspector's eye E. In more detail, visual inspection is a process in which the inspector checks for defects on the surface SF of the inspection object OB in a state where the inspector can visually recognize the inspection object OB by illuminating light Li from the illumination unit 15, which is reflected by the inspection object OB and then incident on the inspector's eye E as reflected light Lr. A visual inspection support device 1 supports such visual inspection. As shown in FIG. 1, the visual inspection support device 1 includes a light source 10, an imaging unit 20, a control unit 30, and a notification unit 40.

[0016] The light source 10 is a surface light source that irradiates light from the vicinity of the inspector's eye E toward the object under inspection OB. The vicinity of the inspector's eye E refers to both an area within 15 cm of the pupil center of the inspector's left eye and an area within 15 cm of the pupil center of the inspector's right eye. As shown in FIG. 1 , the light irradiated by the light source 10 is designated as irradiated light Vi. In this embodiment, the light source 10 is attached to a mounting member that can be worn on the inspector's head. When the inspector wears this mounting member on his or her head, the light source 10 is positioned near the inspector's eye E. It is desirable that the position of the light source 10 be adjustable on this mounting member so that it can be positioned near the inspector's eye E. It is also desirable that the orientation of the light source 10 be adjustable so that light can be irradiated from the vicinity of the inspector's eye E toward the object under inspection OB. In this embodiment, the light irradiated from the light source 10 is visible light. It is desirable that the hue of the visible light emitted from the light source 10 is different from the hue of the surface SF of the inspection object OB and the hue present in the environment in which the inspector performs the inspection. For example, the hue of the visible light emitted from the light source 10 is red light.

[0017] The image capture unit 20 is a camera located near the illumination unit 15, which illuminates the inspection object OB. It captures the reflected light Vr emitted from the light source 10 and reflected by the inspection object OB. The vicinity of the illumination unit 15 refers to a range within 15 cm from the surface of the illumination unit 15. The reflected light Vr is the light emitted from the light source 10 and reflected by the inspection object OB. The image capture unit 20 is a visible light camera that captures not only the reflected light Vr but also the reflected illumination light Le of the illumination light Li that is reflected toward the image capture unit 20 from the inspection object OB when the illumination light Li is emitted from the illumination unit 15. That is, when the illumination light Vi and the illumination light Li are irradiated, the image capture unit 20 captures an image PC that shows the surface SF of the inspection object OB and the reflection position RP of the reflected light Vr on the surface SF. Information about the image PC captured by the image capture unit 20 is transmitted to the control unit 30, which will be described later. The reflection position RP and the image PC will be described in detail with reference to FIG. 3.

[0018] FIG. 3 is an explanatory diagram of the relationship between the position visually observed by an inspector and the reflection position RP. FIG. 3(A) shows the inspection object OB as seen by the inspector under irradiation with the irradiation light Vi and the illumination light Li. FIG. 3(B) shows an image PC of the inspection object OB of FIG. 3(A) captured by the image capture unit 20. In this embodiment, the image PC captured by the image capture unit 20 is a color moving image, but it may also be a color still image. The reflection position RP shown in FIG. 3(A) is the position on the surface SF of the inspection object OB where the irradiation light Vi is reflected and becomes reflected light Vr. As described above, the light source 10 emits the irradiation light Vi from near the inspector's eye E, and the image capture unit 20 is disposed near the illumination unit 15 (see FIG. 1). At this time, the path of light (illumination light Li and reflected light Lr) that is irradiated from the illumination unit 15, reflected by the inspection object OB, and incident on the inspector's eye E is approximately the same as the path of light (irradiation light Vi and reflected light Vr) that is irradiated from the light source 10, reflected by the inspection object OB, and incident on the photographing unit 20. Therefore, the reflection position RP on the surface SF of the inspection object OB can be considered to be the visual position observed by the inspector. The reflection position RP is captured in the image PC captured by the photographing unit 20. Therefore, by identifying the trajectory of the reflection position RP in the image PC, the trajectory of the visual position on the surface SF of the inspection object OB can be identified.

[0019] Returning to the explanation of FIG. 1, the control unit 30 is a computer including a ROM, a RAM, and a CPU, and performs various controls of the visual inspection support device 1. The control unit 30 also functions as an identification unit that receives information about the image PC captured by the imaging unit 20 from the imaging unit 20 and identifies the trajectory of the reflection position RP of the reflected light Vr on the surface SF of the inspection object OB using the image PC. The control unit 30 also functions as a discrimination unit that refers to the identified trajectory of the reflection position RP and discriminates between inspected and uninspected areas on the surface SF of the inspection object OB. The method for identifying the trajectory of the reflection position RP and the method for discriminating between inspected and uninspected areas will be described later with reference to FIG. 5.

[0020] When an uninspected area exists in the inspection object OB, the notification unit 40 notifies the presence of the uninspected area. The notification unit 40 may be a device that notifies by display, a device that notifies by sound, or a device that notifies by display and sound, as long as the content of the notification can be recognized by an inspector or a visual inspection manager.

[0021] 4 is a flowchart showing the procedure of the support process executed by the control unit 30 in the visual inspection support device 1. The support process is a process for supporting an inspector so that the visual inspection is not terminated without leaving an uninspected area on the surface SF of the inspection object OB. The control unit 30 starts the support process when the inspection object OB is placed on the workbench where the visual inspection is performed. The placement of the inspection object OB on the workbench may be detected by the imaging unit 20, by an infrared sensor whose detection range is on the workbench, or by a weight sensor that detects the load on the workbench.

[0022] From the start of the assistance process until the end of the assistance process, the control unit 30 causes the photographing unit 20 to photograph an image PC (see FIG. 3(B)) and sequentially receives information about the image PC from the photographing unit 20. As described above, the image PC is a color moving image that shows the surface SF of the inspection object OB and the reflection position RP of the reflected light Vr on the surface SF. The control unit 30 uses the received image PC to distinguish between inspected areas and uninspected areas on the surface SF of the inspection object OB by processing steps S11 and S12 described below.

[0023] When the support process is started, the control unit 30 uses the image PC received from the image capture unit 20 between the start of the support process and the execution of step S11 to identify the trajectory of the reflection position RP of the reflected light Vr on the surface SF of the inspection object OB (step S11). Specifically, the control unit 30 converts each frame of the image PC, which is a color moving image, from the RGB color space to the HSV color space, and then identifies the reflection position RP of the reflected light Vr on the surface SF captured in each frame based on the H phase (hue). At this time, the control unit 30 also simultaneously identifies the position and orientation of the inspection object OB in each frame. That is, the control unit 30 identifies the trajectory of the reflection position RP of the reflected light Vr on the surface SF by identifying the position and orientation of the inspection object OB in each frame and then identifying the reflection position RP on the surface SF captured in each frame (illustrated as trajectories TJ1 and TJ2 in FIGS. 5(B) and 5(C) described later). The position and orientation of the inspection object OB can be identified by, for example, template matching, which compares the contour shape of the inspection object OB in the image PC with a number of pre-stored templates. However, the method for identifying the position and orientation of the inspection object OB is not particularly limited.

[0024] Next, the control unit 30 refers to the trajectory of the identified reflection position RP and distinguishes between inspected areas and uninspected areas on the surface SF of the inspection object OB (step S12). At this time, the control unit 30 also functions as a storage unit that stores the number of distinguished inspected areas and the number of distinguished uninspected areas.

[0025] Fig. 5 is an explanatory diagram of a method for distinguishing between inspected and uninspected areas. The XYZ axes shown in the lower right of Fig. 5 are common to Figs. 5(A), (B), and (C). Fig. 5(A) shows areas R1 to R6, which are examples of areas into which the surface SF of the inspection object OB is divided. Information about areas R1 to R6 is stored in advance in the control unit 30.

[0026] 5(B) illustrates an example of the trajectory of the reflection position RP of the reflected light Vr on the surface SF of the inspection object OB. The control unit 30 refers to the position and posture of the inspection object OB in each frame and determines, among the regions R1 to R6, the region through which the trajectory has passed as an inspected region, and determines the region through which the trajectory has not passed as an uninspected region. In the case of the trajectory TJ1 illustrated in FIG. 5(B), all of the regions R1 to R6 are regions through which the trajectory TJ1 has passed, and therefore all of the regions R1 to R6 are determined to be inspected regions.

[0027] Figure 5(C) illustrates a different trajectory of reflection position RP from that shown in Figure 5(B). In the case of trajectory TJ2 illustrated in Figure 5(C), of the regions R1 to R6, regions R1, R3 to R6 are regions through which trajectory TJ2 has passed, and therefore regions R1, R3 to R6 are determined to be inspected regions. On the other hand, of the regions R1 to R6, region R2 is a region through which trajectory TJ2 has not passed, and therefore region R2 is determined to be an uninspected region.

[0028] Note that the trajectory of the reflection position RP is a collection of points indicating the reflection position RP identified in each frame, and therefore, more precisely, is represented by countless points, rather than a line like the trajectories TJ1 and TJ2 shown in FIGS. 5B and 5C. Therefore, the determination of whether an area is an inspected area or an uninspected area is made precisely based on whether or not at least one point indicating the reflection position RP is included within the area to be determined. More specifically, if at least one point indicating the reflection position RP is included within the area to be determined, the area is determined as an inspected area. On the other hand, if no point indicating the reflection position RP is included within the area to be determined, the area is determined as an uninspected area. On the other hand, if the trajectory of the reflection position RP is processed into a line connecting each of the points indicating the reflection position RP (for example, a line like the trajectories TJ1 and TJ2), the determination of whether an area is an inspected area or an uninspected area is made based on whether or not the trajectory is included within the area to be determined, as shown in FIGS. 5B and 5C.

[0029] Returning to the explanation of Figure 4, after distinguishing between inspected areas and uninspected areas (step S12), the control unit 30 determines whether the visual inspection has ended (step S13). Specifically, the control unit 30 determines that the visual inspection has ended if the inspection object OB has disappeared from the work table. On the other hand, the control unit 30 determines that the visual inspection has not ended if the inspection object OB is still placed on the work table.

[0030] If it is determined that the visual inspection has not ended (step S13: NO), the control unit 30 executes the process of step S11 again. At this time, the control unit 30 identifies the trajectory of the reflection positions RP of the reflected light Vr on the surface SF of the inspection object OB using the image PC used when step S11 was executed last time, as well as the image PC received from the image capture unit 20 between the execution of step S11 last time and the execution of step S11 this time. That is, the control unit 30 updates the trajectory of the reflection positions RP identified in step S11 last time to the trajectory of the reflection positions RP identified in step S11 this time. If the trajectory of the reflection positions RP is a set of points indicating the reflection positions RP, the number of points constituting the trajectory of the reflection positions RP after the update increases compared to the number of points constituting the trajectory of the reflection positions RP before the update. If the trajectory of the reflection positions RP is a line connecting the points indicating the reflection positions RP, the total length of the trajectory of the reflection positions RP after the update becomes longer than the total length of the trajectory of the reflection positions RP before the update. Then, the control unit 30 refers to the updated trajectory of the reflection position RP and distinguishes between inspected areas and uninspected areas on the surface SF of the inspection object OB (step S12). At this time, the control unit 30 updates the number of inspected areas and the number of uninspected areas stored at the time of the previous discrimination at step S12 to the number of inspected areas and the number of uninspected areas stored at the time of the current discrimination at step S12. After the update, the number of inspected areas is the same as before the update or is greater than before the update. Similarly, the number of uninspected areas is the same as before the update or is less than before the update.

[0031] On the other hand, if it is determined that the visual inspection has ended (step S13: YES), the control unit 30 determines whether or not an uninspected area exists on the surface SF of the inspection object OB (step S14). Specifically, the control unit 30 determines whether or not an uninspected area exists on the surface SF of the inspection object OB by referring to the number of inspected areas and the number of uninspected areas stored the last time step S12 was executed. In more detail, if the number of uninspected areas stored is one or more, the control unit 30 determines that an uninspected area exists on the surface SF of the inspection object OB (step S14: YES). If the number of uninspected areas stored is zero, the control unit 30 determines that no uninspected area exists on the surface SF of the inspection object OB (step S14: NO). If no uninspected area exists (step S14: NO), the control unit 30 ends the assistance process.

[0032] On the other hand, if there are uninspected areas (step S14: YES), the control unit 30 causes the notification unit 40 to notify the presence of uninspected areas (step S15). Thereafter, the control unit 30 ends the support process. Note that when the support process ends, the information indicating the number of inspected areas and the number of uninspected areas stored when step S12 was last executed is erased.

[0033] As described above, in the visual inspection support device 1 of the first embodiment, the light source 10 emits illumination light Vi from near the inspector's eye E, and the photographing unit 20 is disposed near the illumination unit 15 (see FIG. 1 ). At this time, the path of light (illumination light Li and reflected light Lr) emitted from the illumination unit 15, reflected by the inspection object OB, and incident on the inspector's eye E is approximately the same as the path of light (illumination light Vi and reflected light Vr) emitted from the light source 10, reflected by the inspection object OB, and incident on the photographing unit 20. Therefore, the reflection position RP on the surface SF of the inspection object OB can be considered to be the visual position observed by the inspector. The reflection position RP is captured in the image PC captured by the photographing unit 20. Therefore, by using the image PC, the visual position on the surface SF of the inspection object OB can be identified without using a camera capable of eye tracking. Furthermore, by discriminating between inspected and uninspected areas by referring to the trajectory of the reflection position RP, which can be regarded as the visual position, it is possible to discriminate between inspected and uninspected areas with high accuracy.

[0034] Furthermore, the visual inspection support device 1 of the first embodiment includes a notification unit 40 that notifies the user of the presence of an uninspected area in the inspection object OB when the uninspected area exists. Therefore, since the notification unit 40 notifies the user of the presence of an uninspected area, it is possible to reduce the possibility that an inspector will end the visual inspection while leaving an uninspected area. As a result, it is possible to improve the accuracy of detecting defective inspection objects OB during visual inspection.

[0035] Second Embodiment 6 is an explanatory diagram illustrating the configuration of a visual inspection support device 1a according to a second embodiment of the present invention. The visual inspection support device 1a of the second embodiment differs from the visual inspection support device 1 of the first embodiment (see FIG. 1) in that it includes an infrared light source 10a instead of the light source 10, and a first photographing unit 21 and a second photographing unit 22 instead of the photographing unit 20.

[0036] The infrared light source 10a is a surface light source that irradiates infrared light from the vicinity of the inspector's eye E toward the inspection object OB. As shown in FIG. 6, the light irradiated by the infrared light source 10a is referred to as irradiated light Ri. The vicinity of the inspector's eye E has the same meaning as described for the light source 10 of the first embodiment. Furthermore, like the light source 10 of the first embodiment, the infrared light source 10a is attached to a mounting member that can be worn on the inspector's head.

[0037] The first image capturing unit 21 is a camera disposed near the illumination unit 15 and captures reflected infrared light Rr that is emitted from the infrared light source 10a and then reflected by the inspection object OB. The term "near the illumination unit 15" has the same meaning as that described for the image capturing unit 20 in the first embodiment. The first image capturing unit 21 is also equipped with a filter F that limits incident light to only infrared light. That is, since the first image capturing unit 21 is an infrared camera capable of capturing infrared light, even when the first image capturing unit 21 is irradiated with the irradiation light Ri and the illumination light Li, the image P1 captured by the first image capturing unit 21 (shown in FIG. 7, described later) does not capture visible light (e.g., reflected illumination light Le) but clearly captures the reflected infrared light Rr. In this embodiment, the image P1 is a moving image, but may also be a still image. Information about the image P1 captured by the first image capturing unit 21 is transmitted to the control unit 30 each time.

[0038] The second image capturing unit 22 is a camera disposed near the first image capturing unit 21 and captures reflected light (reflected illumination light Le) that is irradiated from the illumination unit 15 and then reflected by the inspection object OB. The vicinity of the first image capturing unit 21 refers to a range within 15 cm from the surface of the first image capturing unit 21. The angle of view of the second image capturing unit 22 is the same as that of the first image capturing unit 21. Because the second image capturing unit 22 is a visible light camera capable of capturing visible light, even when the illumination light Ri and illumination light Li are irradiated, the image P2 captured by the second image capturing unit 22 (shown in FIG. 7 described later) does not capture reflected infrared light Rr, but clearly captures visible light such as reflected illumination light Le (light of the illumination light Li that is reflected toward the second image capturing unit 22 via the inspection object OB). In this embodiment, the image P2 is a color moving image, but may also be a color still image. Information about the image P2 captured by the second image capturing unit 22 is transmitted to the control unit 30 each time.

[0039] FIG. 7 is an explanatory diagram of images P1 and P2. The X, Y, and Z axes shown in the lower right of FIG. 7 are common to FIGS. 7(A) and 7(B). FIG. 7(A) illustrates image P1. In image P1 of FIG. 7(A), a dashed line is shown representing the object under inspection OB, which is not actually captured in image P1. In image P1, the reflection position Rp of reflected infrared light Rr is captured, but the object under inspection OB is not captured. FIG. 7(B) illustrates image P2. In image P2 of FIG. 7(B), a dashed line is shown representing the reflection position Rp of reflected infrared light Rr, which is not actually captured in image P2. In image P2, the reflection position Rp of reflected infrared light Rr is not captured, but the object under inspection OB is captured. Images P1 and P2 shown in FIG. 7 are images captured at the same time by the first imaging unit 21 and the second imaging unit 22. As described above, the second photographing unit 22 is positioned near the first photographing unit 21, and the angle of view of the first photographing unit 21 and the angle of view of the second photographing unit 22 are the same, so images P1 and P2 are images of approximately the same area on the workbench where visual inspection is performed.

[0040] In the second embodiment, the control unit 30 functions as an identification unit that receives information about the image P1 captured by the first imaging unit 21 from the imaging unit 20 and identifies the trajectory of the reflection position Rp of the reflected infrared light Rr using the image P1. Specifically, the control unit 30 performs binarization on each frame that makes up the image P1, and then identifies an area made up of pixels brighter than a brightness threshold as the reflection position Rp of the reflected infrared light Rr in each frame, thereby identifying the trajectory of the reflection position Rp of the reflected infrared light Rr.

[0041] Furthermore, in the second embodiment, the control unit 30 functions as an identification unit that receives information about the image P2 captured by the second imaging unit 22 from the imaging unit 20 and identifies the surface SF of the inspection object OB using the image P2. Specifically, the control unit 30 identifies the position and orientation of the inspection object OB in each frame constituting the image P2 by template matching or the like, thereby identifying the surface SF captured in each frame.

[0042] Furthermore, the control unit 30 functions as an identification unit that identifies the trajectory of the reflection positions Rp of the reflected infrared light Rr on the surface SF by superimposing the trajectory of the reflection positions Rp of the reflected infrared light Rr identified using image P1 on the surface SF of the inspection object OB identified using image P2. Specifically, the control unit 30 identifies the trajectory of the reflection positions Rp of the reflected infrared light Rr on the surface SF by superimposing image P1, in which the trajectory of the reflection positions Rp of the reflected infrared light Rr has been identified, on image P2, in which the surface SF of the inspection object OB has been identified. Then, the control unit 30 functions as a discrimination unit that discriminates between inspected areas and uninspected areas on the surface SF of the inspection object OB by referring to the identified trajectory of the reflection positions Rp.

[0043] 4 is executed in the visual inspection support device 1a of the second embodiment, similarly to the visual inspection support device 1 of the first embodiment. Note that in step S11 of the support process in the second embodiment, the control unit 30 identifies the trajectory of the reflection position Rp of the reflected infrared light Rr on the surface SF of the inspection object OB using the images P1 and P2 received from the first photographing unit 21 and the second photographing unit 22 between the start of the support process and the execution of step S11. Specifically, as described above, the control unit 30 identifies the trajectory of the reflection position Rp of the reflected infrared light Rr on the surface SF by superimposing the trajectory of the reflection position Rp of the reflected infrared light Rr identified using image P1 on the surface SF of the inspection object OB identified using image P2. In addition, in step S11, which is executed again when it is determined that the visual inspection has not been completed (step S13: NO), the control unit 30 updates the trajectory of the reflection position Rp of the reflected infrared light Rr on the surface SF of the object to be inspected OB using not only the images P1 and P2 used when the previous step S11 was executed, but also the images P1 and P2 received from the first photographing unit 21 and the second photographing unit 22 between the time the previous step S11 was executed and the time the current step S11 is executed.

[0044] In the visual inspection support device 1a of the second embodiment described above, the infrared light source 10a irradiates infrared light from near the inspector's eye E, and the first photographing unit 21 is disposed near the illumination unit 15. In this case, the path of light (illumination light Li and reflected light Lr) irradiated from the illumination unit 15, reflected by the inspection object OB, and incident on the inspector's eye E generally coincides with the path of infrared light (irradiation light Ri and reflected infrared light Rr) irradiated from the infrared light source 10a, reflected by the inspection object OB, and incident on the first photographing unit 21. Therefore, the reflection position Rp of the reflected infrared light Rr on the surface SF of the inspection object OB identified using the image P1 captured by the first photographing unit 21 can be considered to be the visual position visually observed by the inspector, and therefore the visual position can be identified using the image P1. Furthermore, since the second photographing unit 22 is disposed near the first photographing unit 21, it is possible to identify the visual observation position on the surface SF of the inspection object OB without using a camera capable of eye tracking by superimposing the identified reflection position Rp of the reflected infrared light Rr on the surface SF of the inspection object OB identified using the image P2 photographed by the second photographing unit 22. Furthermore, since the inspected area and the uninspected area are distinguished in this way by referring to the trajectory of the reflection position Rp that can be regarded as the visual observation position, it is possible to accurately distinguish the inspected area from the uninspected area.

[0045] In the first embodiment, the reflection position RP is identified using an image PC that captures reflected light Vr, which is visible light that is irradiated from the light source 10 and then reflected by the inspection object OB, whereas in the second embodiment, the reflection position Rp is identified using an image P1 that captures reflected infrared light Rr, which is infrared light that is irradiated from the infrared light source 10a and then reflected by the inspection object OB. Therefore, in the second embodiment, unless infrared light or near-infrared light is present around the workbench where the visual inspection is performed, no light other than the reflected infrared light Rr is captured in the captured image P1, so the reflection position Rp can be identified with higher accuracy than in the first embodiment, in which the reflection position RP is identified using the image PC.

[0046] <Modification of this embodiment> The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0047] In the above-described embodiment, the light source 10 and the infrared light source 10a are surface light sources, but they are not limited to this. They may be any of parallel light sources, linear light sources, and point light sources, as long as they can emit light that is sufficiently reflected by the surface SF of the inspection object OB.

[0048] In the above-described embodiment, the light source 10 and the infrared light source 10a are attached to a mounting member that can be worn on the inspector's head, but this is not limited to this. The light source 10 and the infrared light source 10a may be attached to any member as long as they can be placed near the inspector's eyes E. For example, if a columnar member that is about the same height as or taller than the inspector is provided around the workbench where the visual inspection is performed, the light source 10 and the infrared light source 10a may be attached to that columnar member.

[0049] In the above-described embodiment, the visual inspection support device 1, 1a does not include an illumination unit 15, and supports the visual inspection using an illumination unit 15 attached above the workbench where the visual inspection is performed, but this is not limited to this. The visual inspection support device 1, 1a may also include an illumination unit 15. With such a visual inspection support device 1, 1a, even in an environment where an illumination unit 15 is not attached above the workbench where the visual inspection is performed, the illumination unit 15 included in the visual inspection support device 1, 1a can be attached above the workbench to support the visual inspection.

[0050] In the second embodiment described above, the infrared light source 10a is used, but this is not limiting. For example, an ultraviolet light source may be used instead of the infrared light source 10a. In this case, the filter F attached to the first image capturing unit 21 is a filter that limits the incident light to only ultraviolet light.

[0051] In the above-described embodiment, the locus of reflection positions RP and Rp is determined when the image PC and the images P1 and P2 are moving images. However, the image PC and the images P1 and P2 may be still images. In such a case, the control unit 30 causes the image capturing unit 20 (first image capturing unit 21 and second image capturing unit 22) to capture successive still images PC (images P1 and P2), and determines the locus of reflection positions RP (reflection positions Rp) of the reflected light Vr (reflected infrared light Rr) on the surface SF.

[0052] This aspect has been described above based on embodiments and modifications. However, the above-described embodiments are intended to facilitate understanding of this aspect and are not intended to limit this aspect. This aspect may be modified or improved without departing from the spirit and scope of the claims, and equivalents thereof are included in this aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate. [Explanation of symbols]

[0053] 1, 1a...Visual inspection support device 10…Light source 10a…Infrared light source 15...Lighting section 20...Photography Department 21...First Filming Section 22...Second Filming Section 30...Control unit 40...Information Department F...Filter

Claims

1. A visual inspection support device that supports an inspector in visually inspecting an object to be inspected, a light source that irradiates light from near the eyes of the inspector toward the inspection object; an imaging unit that is disposed near an illumination unit that illuminates the inspection object and that images light that is irradiated from the light source and then reflected by the inspection object; an identification unit that identifies a trajectory of a reflection position of the reflected light on the surface of the inspection object using the image captured by the imaging unit; a discrimination unit that discriminates between inspected and uninspected areas on the surface by referring to the identified trajectory.

2. A visual inspection support device that supports an inspector in visually inspecting an object to be inspected, an infrared light source that irradiates infrared light from near the eyes of the inspector toward the inspection object; a first image capturing unit that is fitted with a filter that limits incident light to only infrared light, is disposed near an illumination unit that illuminates the inspection object, and captures an image of reflected infrared light that is irradiated from the infrared light source and then reflected by the inspection object; a second photographing unit disposed near the first photographing unit and configured to photograph light irradiated from the illumination unit and then reflected by the inspection object; an identification unit that identifies a trajectory of the reflection positions of the reflected infrared light, which is identified using the image captured by the first image capture unit, on the surface of the object to be inspected, which is identified using the image captured by the second image capture unit; and a discrimination unit that discriminates between inspected and uninspected areas on the surface by referring to the identified trajectory.

3. 3. The visual inspection support device according to claim 1 or 2, further comprising: A visual inspection support device comprising: a notification unit that notifies the presence of an uninspected area when the uninspected area exists on the surface.

Images