Inspection system

Abstract

To provide an inspection system with which it is possible to stably inspect points of inspection using captured images.SOLUTION: An inspection system 100 according to the present disclosure comprises: an image-capturing device 1 that captures an image of the object to be inspected; an image-capturing information acquisition unit 22 that acquires image-capturing information that includes an image-capturing position linked to an inspection point of the object to be inspected, an image-capturing timing, and the number of captured images; a movement device 3 that moves the image-capturing device 1 on the basis of the image-capturing position; an image acquisition unit 21 that acquires a plurality of images captured by the image-capturing device 1; and an image determination unit 23 that determines the plurality of images. The image determination unit 23 includes an image determination processing unit 232 and an acceptability determination unit 234. The image determination processing unit 232 determines whether the inspection point indicated by each of the plurality of images is normal, abnormal, or uninspected. The acceptability determination unit 234 determines that the object being inspected is unacceptable, when the acquired plurality of images include at least one image in which the inspection point was determined as being abnormal.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an inspection system.

Background Art

[0002] An inspection system that captures an image while relatively moving an imaging device and an inspection object, and performs an inspection using the captured image is widely used. For example, the image inspection system disclosed in Patent Document 1 acquires inspection target images at respective viewpoints at a plurality of imaging timings while changing the position of the viewpoint with respect to the inspection object. The image inspection system supports identification of the cause when a defect occurs by capturing a plurality of recording images at a timing different from the imaging timing for acquiring a plurality of inspection images.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The inventors of the present application have discovered the following problems. In such an inspection system, since the imaging is performed while relatively moving the imaging device and the inspection object, there are cases where the inspection location cannot be stably imaged, such as the inspection object being displaced from a predetermined position or halation occurring.

[0005] The present disclosure has been made in view of the above-described problems, and provides an inspection system capable of performing an inspection using an image obtained by stably imaging an inspection location.

Means for Solving the Problems

[0006] The inspection system according to the present disclosure is an imaging device that images an inspection object, and An imaging information acquisition unit that acquires imaging information including an imaging position, an imaging timing, and the number of imaging shots associated with an inspection location of the object to be inspected; A moving device that moves the imaging device based on the acquired imaging position; An image acquisition unit that acquires a plurality of images captured by the imaging device; An image determination unit that determines the acquired plurality of images, and includes: The image determination unit includes an image determination processing unit and a pass / fail determination unit; The image determination processing unit determines whether the inspection locations indicated by the acquired plurality of images are normal, abnormal, or undetected; The pass / fail determination unit determines that the object to be inspected is unqualified when at least one of the acquired plurality of images is determined to have an abnormal inspection location, or when all of the acquired plurality of images are determined to have an undetected inspection location.

[0007] According to such a configuration, by acquiring a plurality of images of the object to be inspected for each inspection location, even if there is an image in which the inspection location is not detected, an image with a normal inspection location can be discriminated. Therefore, the inspection of the object to be inspected can be performed using the images in which the inspection location is stably imaged.

[0008] Also, the number of imaging shots may be set to be larger as the imaging range in which the inspection location is imaged by the imaging device becomes narrower.

[0009] According to such a configuration, according to the imaging range in which the inspection location is imaged by the imaging device, a larger number of images with a high difficulty level of imaging the inspection location can be acquired. Therefore, the undetected inspection location can be suppressed.

[0010] Also, the image determination unit further includes a pre-determination processing unit that converts the image to grayscale. The image determination processing unit may be characterized by determining, using the grayscale-converted image, whether the inspection locations indicated by the acquired plurality of images are normal, abnormal, or undetected, respectively.

[0011] According to such a configuration, since the grayscale-converted image is determined, the influence on the background portion due to the color of the inspection object can be suppressed, and a decrease in determination accuracy can be suppressed.

[0012] Further, the image determination unit further includes an edge determination removal unit that determines whether the determined inspection location is included in the determination effective range of the image. The edge determination removal unit may be characterized by determining that the inspection locations indicated by the acquired plurality of images are undetected when it is determined that the determined inspection location is not included in the determination effective range of the image.

[0013] According to such a configuration, it is possible to determine that the inspection location is undetected for an image in which the entire inspection location could not be sufficiently imaged.

[0014] Further, the image determination unit determines the acquired plurality of images by image recognition using deep learning.

Advantages of the Invention

[0015] According to the present disclosure, an inspection can be performed using an image in which an inspection location is stably imaged.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0017] Hereinafter, specific embodiments to which the present disclosure is applied will be described in detail with reference to the drawings. However, the present disclosure is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings are simplified as appropriate.

[0018] (Embodiment 1) <Configuration of the Inspection System> The configuration of the inspection system according to Embodiment 1 will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing the configuration of the inspection system according to Embodiment 1.

[0019] As shown in FIG. 1, the inspection system 100 includes an imaging device 1, an automatic inspection device 2, a moving device 3, and a control device 4. The inspection system 100 is used, for example, in the visual inspection process on a vehicle production line. The imaging device 1, the automatic inspection device 2, the moving device 3, and the control device 4 can exchange various data with each other through a communication line or the like.

[0020] The control device 4 controls the imaging device 1, the automatic inspection device 2, and the moving device 3. The control device 4 is, for example, a computer including a central processing unit, a memory, various interfaces, a communication module, etc. The storage stores various programs. The processor reads and executes the program loaded onto the memory, thereby providing the processing and functions of the control device 4. Also, the automatic inspection device 2 and the control device 4 may be configured by a single computer. This computer may use, for example, a personal computer. The control device 4 may hold information on inspection items to be carried out on the inspection line. The information on the inspection items includes, for example, imaging information associated with the inspection locations of the objects to be inspected. The control device 4 holds, for example, inspection items for each vehicle to be inspected. The control device 4 may hold, as inspection items, the automatic inspection items carried out by the automatic inspection device 2 and the manual inspection items carried out by the inspector, respectively. Note that the manual inspection is, as an example, a visual inspection. The control device 4 is configured to be able to communicate with the imaging device 1, the automatic inspection device 2, and the moving device 3, and may output information on inspection items or imaging information to each device.

[0021] The imaging device 1 images the object to be inspected and generates an image. The imaging device 1 includes, for example, an optical system and an image sensor. The imaging device 1 may further include a personal computer. The object to be inspected is conveyed, for example, by a conveyor. The object to be inspected is, for example, a vehicle. The image is preferably, for example, a color image. The imaging device 1 may image the object to be inspected according to the object to be inspected, the inspection items, the imaging information corresponding to the inspection items, etc. The imaging information includes, for example, the imaging position, the imaging timing, and the number of images taken. The imaging timing may be determined according to, for example, conveyor control information, sensors, and the camera angle. The number of images taken is preferably set to be larger as the imaging range in which the inspection location can be imaged by the imaging device 1 becomes narrower. When it is impossible or difficult to view the inspection location from above, or when the time during which the inspection location can be imaged by the imaging device 1 is short, the imaging range tends to become narrower.

[0022] The moving device 3 moves the imaging device 1 based on the imaging position acquired from the control device 4. The moving device 3 is, for example, an actuator, a movable table, an articulated robot, or the like.

[0023] The automatic inspection device 2 includes an image acquisition unit 21, an imaging information acquisition unit 22, and an image determination unit 23. The automatic inspection device 2 is a device that is introduced into a part of the inspection line and automatically performs part of the inspection items. The automatic inspection device 2 is, for example, AVI (Automatic Visual Inspection).

[0024] The automatic inspection device 2 is configured to be communicable with the control device 4 and acquires automatic inspection items from the control device 4. The inspection items of the inspection performed by the automatic inspection device 2 are stored as automatic inspection items in the control device 4. The automatic inspection device 2 automatically performs the inspection based on the acquired automatic inspection items.

[0025] Note that the automatic inspection device 2 may notify the inspector of the inspection items reflecting the inspection results on the inspection line through an inspection result display terminal or a mobile terminal. Further, the automatic inspection device 2 may also notify the inspector of imaging parameters corresponding to the inspection items, the vehicle of the inspection object, conveyor values, inspection items, images captured by the imaging device 1 in real time, etc. Also, for example, the automatic inspection device 2 may have a server function that outputs the inspection results based on a request from the outside. The automatic inspection device 2 is configured to be communicable with an inspection result display terminal and a mobile terminal. The automatic inspection device 2 transmits the inspection result information to the inspection result display terminal based on a request from the inspection result display terminal.

[0026] The image acquisition unit 21 acquires a plurality of images captured by the imaging device 1.

[0027] The imaging information acquisition unit 22 acquires imaging information from the imaging device 1 or the control device 4. The acquired imaging information includes the imaging position, imaging timing, and number of imaging frames associated with the inspection location of the inspection object.

[0028] The image determination unit 23 determines a plurality of images acquired by the image acquisition unit 21. The image determination unit 23 includes an image determination processing unit 232 and a pass / fail determination unit 234. The image determination unit 23 may further include a pre-determination processing unit 231 and an edge determination removal unit 233.

[0029] The pre-determination processing unit 231 converts the images acquired by the image acquisition unit 21 into grayscale.

[0030] The image determination processing unit 232 determines whether the inspection locations indicated by the plurality of images acquired by the image acquisition unit 21 are normal, abnormal, or undetected. When the image determination unit 23 includes the pre-determination processing unit 231, the image determination processing unit 232 determines whether the inspection locations indicated by the plurality of images grayscale-converted by the pre-determination processing unit 231 are normal, abnormal, or undetected.

[0031] The image determination processing unit 232 stores, for each of the automatic inspection items stored in the control device 4, the conditions for determining whether the inspection location of the inspection object is normal, abnormal, or undetected. The image determination processing unit 232 automatically performs an inspection on the inspection location, and outputs a normal determination if the inspection location is determined to be normal, an abnormal determination if it is determined to be abnormal, and an undetected determination if it is determined to be undetected. Normal, for example, is a state similar to the production instruction. Abnormal, for example, means different from the production instruction. Undetected, for example, means that the determination value is below the threshold or that the imaging by the imaging device 1 has failed. The failure of the imaging by the imaging device 1 is, for example, the occurrence of halation.

[0032] The image determination processing unit 232 may perform image recognition by deep learning, for example. The image determination processing unit 232 learns a normal inspection location and an abnormal inspection location based on a large number of image data. The image determination processing unit 232 analyzes an image obtained by imaging an inspection object based on the learned result, and outputs a normal determination if the inspection location of the inspection object is determined to be in a normal state, and outputs an abnormal determination if it is determined to be in an abnormal state. Note that, as will be described later, when the inspection result of a normal determination is corrected to a normal determination by an inspector, the image determination processing unit 232 may learn the normal or abnormal state of the inspection location in the inspection based on the information of the corrected inspection result.

[0033] The edge determination removal unit 233 determines whether the inspection location determined by the image determination processing unit 232 is included in the effective range of image determination. When the edge determination removal unit 233 determines that the inspection location determined by the image determination processing unit 232 is not included in the effective range of image determination, it determines that the inspection locations indicated by the plurality of acquired images are not detected.

[0034] The pass / fail determination unit 234 determines that the inspection object is unqualified when at least one of the plurality of acquired images includes an image in which the inspection location is determined to be abnormal, or when all of the plurality of acquired images are images in which the inspection location is determined to be not detected. Note that the pass / fail determination unit 234 may determine that the inspection object is unqualified when at least one of the plurality of acquired images includes an image in which the inspection location is determined to be abnormal, or when the plurality of acquired images do not include any image in which the inspection location is determined to be normal.

[0035] <An example of the processing by the inspection system> Next, with reference to FIG. 2, an example of the processing by the inspection system 100 will be described.

[0036] First, when entering the inspection process by the automatic inspection device 2 on the inspection line, the automatic inspection device 2 acquires inspection item information from the control device 4 (step ST1). The inspection item information is associated with imaging timing information.

[0037] Subsequently, the moving device 3 is controlled according to the imaging information for each inspection location (step ST21). Specifically, the moving device 3 moves the imaging device 1 to the imaging position associated with the inspection location of the inspection object.

[0038] Subsequently, the imaging device 1 images the inspection location and generates an image (step ST22). Specifically, the imaging device 1 images the inspection location at the imaging timing associated with the inspection location of the inspection object, and generates n images for each inspection location. The number of imaging n can be changed according to each inspection location.

[0039] The loop processing is performed for the above-described steps ST21 and ST22 until all inspection locations are photographed. When the loop processing is completed, the photographing of all inspection locations is completed.

[0040] On the other hand, image determination is performed on the images generated in step ST22 (step ST30). Specifically, image determination is performed on n images for each inspection location.

[0041] Subsequently, the pre-determination processing unit 231 converts the n images into grayscale (step ST31). The n inspection objects respectively indicated by the n images may emit different colors due to paint or the like. In such a case, although the same object is shown in the background part of the n inspection objects, the background part in the image may change according to the color of the inspection object. The change in the background part of the n images converted into grayscale is small according to the color of the inspection object.

[0042] Subsequently, the image determination processing unit 232 determines whether the inspection location shown in the n images converted into grayscale is normal, abnormal, or undetected (step ST32).

[0043] Subsequently, the edge determination removal unit 233 determines whether the inspection location determined in step ST32 is included in the determination effective range of the image (step ST33). Specifically, when the edge determination removal unit 233 determines that the inspection location is not included in the determination effective range of the image, regardless of the determination result in step ST32, the inspection location is determined to be undetected. When the edge determination removal unit 233 determines that the inspection location is included in the determination effective range of the image, the inspection location remains determined to be normal, abnormal, or undetected as per the determination result in step ST32.

[0044] Subsequently, the pass / fail determination unit 234 determines whether the n images for each inspection location include at least one image in which the inspection location is determined to be abnormal (step ST34). Specifically, when the pass / fail determination unit 234 determines that the n images include at least one image in which the inspection location is determined to be abnormal (step ST34: YES), it determines that the inspection object is non-conforming. Subsequently, the inspector performs visual confirmation (step ST36). Here, for example, when the inspector finds that the inspection location determined by the pass / fail determination unit 234 to be non-conforming is actually normal and the inspection object should be determined to be conforming, the inspector may input the inspection result to the interface of the control device 4.

[0045] On the other hand, when the pass / fail determination unit 234 determines that the n images do not include any image in which the inspection location is determined to be abnormal (step ST34: NO), it determines whether all of the n images are images in which the inspection location is determined to be undetected (step ST35).

[0046] When the pass / fail determination unit 234 determines that all of the n captured images are images in which the inspection location is determined to be undetected (step ST35: YES), it determines that the inspection object is non-conforming. In step ST35, if the pass / fail determination unit 234 determines that the n captured images do not include any images in which the inspection location is determined to be abnormal, it may determine that the inspection object is non-conforming. Subsequently, the inspector performs a visual check (step ST36). The inspector can perform a visual check to determine whether the inspection locations determined to be undetected in steps ST32 and ST33 are normal or abnormal.

[0047] On the other hand, when the pass / fail determination unit 234 determines that all of the n captured images are images in which the inspection location is determined to be undetected (step ST35: NO), it determines that the inspection object is conforming.

[0048] As described above, loop processing is performed for steps ST21 and ST22 to capture all inspection locations. Also, for all inspection locations, image determination is performed on the n captured images (step ST30). Therefore, it is possible to determine the pass / fail of the inspection of the vehicle that is the inspection object, and the vehicle inspection is completed.

[0049] From the above, according to the above configuration, by acquiring a plurality of images of the inspection object, even if there is an image in which the inspection location is not detected, an image in which the inspection location is normal can be discriminated. Therefore, the inspection of the inspection object can be performed using the images in which the inspection location is stably imaged.

[0050] Also, according to the above configuration, in step ST31, the determination preprocessing unit 231 converts the n captured images into grayscale. The background portion of the n captured images that have been converted to grayscale has a small change according to the color of the inspection object. Thereby, the influence of the color of the inspection object on the background portion can be suppressed, and a decrease in determination accuracy can be suppressed.

[0051] In many cases, inspections are also carried out on manufactured products such as vehicles to check for product defects. For example, in vehicle inspections, an inspection line is configured, and a plurality of inspection processes are sequentially carried out by a plurality of inspectors. Here, each inspector is assigned the inspection items to be carried out by himself / herself. For the inspection of some inspection items, it may be carried out using an inspection system introduced on the inspection line. In this case, depending on the inspection results of the inspection system, differences may occur in the inspection items to be subsequently carried out by the inspector. Here, when the above-described inspection system 100 is introduced into such an inspection process, the inspector is notified of the inspection results and the inspection items to be subsequently carried out by the inspector accordingly. Therefore, the inspector can efficiently grasp the inspection items reflecting the inspection results of the inspection system 100.

[0052] <Specific examples of each step> Next, with reference to FIGS. 3 to 6, specific examples of each step of the processing by the inspection system 100 will be described.

[0053] In one specific example of step ST22, the captured images are shown in FIGS. 3(a) and 3(b). The inspection locations are the vehicle's mechanical key and camera within frames f31 and f32 shown in FIGS. 3(a) and 3(b), respectively. The imaging range in which this mechanical key and camera can be imaged by the imaging device 1 has a certain size, and the imaging time is 5 seconds. Therefore, the number of captured images is set to 2.

[0054] In another specific example of step ST22, the captured images are shown in FIGS. 4(a) to 4(e). The inspection location is the vehicle's air cleaner case within frames f41 to f45 shown in FIGS. 4(a) to 4(e), respectively. The imaging range in which this air cleaner case can be imaged by the imaging device 1 is smaller than the imaging range of the mechanical key and camera shown in FIGS. 3(a) and 3(b), and the imaging time is 0.8 to 1.5 seconds. Therefore, the number of captured images is set to 5, which is more than 2. In this specific example, since the number of captured images is set to be large, a large number of images of this air cleaner case captured stably can be obtained.

[0055] In an example of step ST33, the determined images are shown in FIGS. 5(a) to 5(c). The images shown in FIG. 5(a), the image shown in FIG. 5(b), and the image shown in FIG. 5(c) were captured in this order. That is, the imaging timing is in the order of the image shown in FIG. 5(a), the image shown in FIG. 5(b), and the image shown in FIG. 5(c). The inspection location is the air cleaner case of the vehicle within frames f51 to f53 shown in FIGS. 5(a) to 5(c), respectively.

[0056] The entire air cleaner case within frame f52 is reflected in the image shown in FIG. 5(b). Similarly, the entire air cleaner case within frame 53 is reflected in the image shown in FIG. 5(c). Therefore, the images shown in FIGS. 5(b) and 5(c) are determined to have the detection location detected, and a preferable inspection is being performed.

[0057] On the other hand, a part of the air cleaner case within frame f51 is reflected in the image shown in FIG. 5(a), and the remaining part is presumed to be outside the image shown in FIG. 5(a). In other words, the entire air cleaner case within frame f51 is not reflected in the image shown in FIG. 5(a) and is cut off. Therefore, the image shown in FIG. 5(a) is determined to not have the detection location detected, and a preferable inspection is not being performed.

[0058] In an example of step ST33, as shown in FIG. 6(b), on the image shown in FIG. 5(b), a determination valid range E6 is set. The determination valid range E6 is inside a frame smaller than the outer edge of the image shown in FIG. 5(b). As shown in FIG. 6(b), since frame f52 is located inside the determination valid range E6, it can be determined that the detection location is detected. On the other hand, as shown in FIG. 6(a), since frame f51 is located outside the determination valid range E6, it is determined that the detection location is detected.

[0059] Each configuration in the above-described embodiment is constituted by hardware or software, or both, and may be constituted by one piece of hardware or software, or may be constituted by a plurality of pieces of hardware or software. The functions (processes) of each device may be realized by a computer having a CPU (Central Processing Unit), a memory, and the like. For example, a program for performing the method (e.g., control method) in the embodiment may be stored in a storage device, and each function may be realized by the CPU executing the program stored in the storage device.

[0060] These programs can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). Also, the programs may be supplied to a computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can supply the programs to a computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0061] Note that the present disclosure is not limited to the above-described embodiments, and can be appropriately modified without departing from the gist. Also, the present disclosure may be implemented by appropriately combining the above-described embodiments and examples thereof. Further, although an example in which the inspection system 100 described above is used in the visual inspection process in a vehicle production line has been described, it is not limited thereto. Also, the automatic inspection device 2 in the inspection system 100 is not limited to a device that performs image recognition by deep learning. The inspection system 100 can be arbitrarily applied in an inspection line that performs other inspections other than visual inspection and includes a process of performing automatic inspection.

Explanation of Reference Numerals

[0062] 100 Inspection system 1 Imaging device 2 Automatic inspection device 21 Image acquisition unit 22 Imaging information acquisition unit 23 Image determination unit 231 Pre-determination processing unit 232 Image determination processing unit 233 Edge determination removal unit 234 Pass / fail determination unit 3 Moving device 4 Control device f31~f35, f41, f42, f51~f53, f61, f62 Frame E6 Determination effective range ST1, ST21, ST22, ST30~ST36 Step

Claims

1. An imaging device that images an object to be inspected, an imaging information acquisition unit that acquires imaging information including an imaging position, an imaging timing, and the number of imaging times associated with an inspection location of the object to be inspected, a moving device that moves the imaging device based on the acquired imaging position, an image acquisition unit that acquires a plurality of images captured by the imaging device, and an image determination unit that determines the acquired plurality of images, and includes an image determination processing unit and a pass / fail determination unit, The image determination unit includes an image determination processing unit and a pass / fail determination unit, The image determination processing unit determines whether the inspection locations indicated by the acquired plurality of images are normal, abnormal, or undetected, When the acquired plurality of images include at least one image in which the inspection location is determined to be abnormal, or when all of the acquired plurality of images are images in which the inspection location is determined to be undetected, the pass / fail determination unit determines that the object to be inspected is unqualified. An inspection system.

2. The number of imaging times is set to be larger as the imaging range in which the inspection location is imaged by the imaging device becomes narrower. The inspection system according to claim 1.

3. The image determination unit further includes a pre-determination processing unit that converts the image to grayscale, The image determination processing unit determines whether the inspection locations indicated by the acquired plurality of images are normal, abnormal, or undetected using the grayscale-converted image. The inspection system according to claim 1 or 2.

4. The image determination unit further includes an edge determination removal unit that determines whether the determined inspection location is included in the effective range of image determination, When the edge determination removal unit determines that the determined inspection location is not included in the effective range of image determination, the edge determination removal unit determines that the inspection locations indicated by the acquired plurality of images are undetected. The inspection system according to claim 1 or 2.

5. The image determination unit determines the acquired plurality of images by image recognition using deep learning. The inspection system according to claim 1 or 2.

Images