Visual inspection system

Abstract

To enable inspection regarding a plurality of directions to be realized simply, when conducting visual inspection of a product based on the image of the product captured by a camera.SOLUTION: A visual inspection system according to one embodiment of the present invention comprises: a placement unit 20 on which a product 10 to be visually inspected is placed; a camera 30 that captures the image of the product 10 placed on the placement unit 20 from a specific direction; and a visual inspection device 40 that executes the visual inspection process of the product 10 on the basis of the image captured by the camera 30. The placement unit 20 includes a mirror 23 that is located so as to reflect the image of the product 10 placed thereon as seen from another direction different than the specific direction toward the camera 30, and the camera 30 captures an image including an image section in which the product 10 seen from the specific direction is captured and an image section in which the product 10 seen from the another direction via the mirror 23 is captured.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an appearance inspection system for inspecting the appearance of a product based on an image of the product taken by a camera.

Background Art

[0002] Conventionally, in order to check whether there are defects such as scratches and distortions in manufactured products, appearance inspection of products has been carried out. Appearance inspection of products is realized, for example, by placing the product to be inspected on a dedicated stand, taking a picture with a camera, and analyzing the image. Conventionally, the stand generally has only the function of fixing the product. Also, the camera for taking pictures of the product is often fixed so as to take pictures of the product on the stand from a specific position and direction.

[0003] When performing appearance inspection of products in the above-described manner, in order to inspect the appearance of the product in a plurality of directions, it is necessary to perform the operation of changing the orientation of the product placed on the stand and taking pictures a plurality of times, and the inspection takes a long time. Also, although the number of cameras may be increased to take pictures from a plurality of directions, not only does the equipment cost increase, but it is also necessary to perform the operation of installing each camera at an appropriate position, which is troublesome. Further, since it is necessary to individually analyze each image taken of the product from a plurality of directions and integrate the analysis results to obtain a final inspection result, there is also a concern that the processing becomes complicated.

[0004] As prior art in the technical field related to the present invention, there are the following. For example, Patent Document 1 discloses an image analysis system that takes pictures of an object moving on a production line such as a food factory with a camera and inspects it. On a GUI analysis screen, an operator arbitrarily sets items to be inspected, automatically reads information about the set items and compares them, and records the results together with an image.

Prior Art Documents

Patent Documents

[0005] [[ID= Japanese Patent Publication No. 2013-205182 [Overview of the project] [Problems that the invention aims to solve]

[0006] This invention has been made in view of the above-mentioned conventional circumstances, and aims to enable easy inspection of the appearance of a product based on an image of the product taken with a camera, by performing inspection from multiple directions. [Means for solving the problem]

[0007] To achieve the above objective, an appearance inspection system according to one aspect of the present invention is configured as follows. In other words, the visual inspection system according to the present invention comprises a mounting unit on which a product to be visually inspected is placed, a camera that images the product mounted on the mounting unit from a first direction, and a visual inspection device that performs a visual inspection process of the product based on the image captured by the camera, wherein the mounting unit has a mirror arranged to reflect an image of the mounted product viewed from a second direction different from the first direction toward the camera, and the camera captures an image that includes a first image portion showing the product viewed from the first direction and a second image portion showing the product viewed from the second direction via the mirror.

[0008] Herein, the appearance inspection system according to the present invention further includes a first detection unit that detects when a product is placed on a mounting unit, and a second detection unit that detects when an operating body used to perform the product placement operation on the mounting unit has moved out of the camera's shooting range, and the appearance inspection device may be configured to perform an appearance inspection process based on an image captured by the camera when the first detection unit has detected that a product has been placed on the mounting unit, and the second detection unit has detected that the operating body has moved out of the camera's shooting range.

[0009] Furthermore, in the visual inspection system according to the present invention, inspection frames for the first image portion and the second image portion are predetermined, and the visual inspection device may be configured to perform visual inspection processing on the inspection frame of the first image portion and visual inspection processing on the inspection frame of the second image portion, respectively, for the image captured by the camera.

[0010] Furthermore, in the visual inspection system according to the present invention, the visual inspection device may be configured to determine the position of the product on the mounting unit, adjust the image or inspection frame according to the position of the product, and then perform the visual inspection process.

[0011] Furthermore, in the visual inspection system according to the present invention, the visual inspection device may be configured to determine the orientation of the product in the mounting unit, adjust the image or inspection frame according to the orientation of the product, and then perform the visual inspection process. [Effects of the Invention]

[0012] According to the present invention, it becomes possible to easily perform inspections of the appearance of a product based on images of the product taken with a camera, from multiple directions. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows an example configuration of a visual inspection system according to one embodiment of the present invention. [Figure 2] Figure 1 is a plan view of an exemplary mounting unit used in the visual inspection system. [Figure 3] This is a longitudinal cross-sectional view of the exemplary mounting unit shown in Figure 2, along the line A-A'. [Figure 4] This figure shows an example of an image captured by a camera in the visual inspection system shown in Figure 1. [Figure 5] This figure shows an example of an inspection frame that indicates the scope of the visual inspection process performed by the visual inspection system in Figure 1. [Figure 6] This figure shows an example of the screen display of the results of the visual inspection process performed by the visual inspection system shown in Figure 1. [Figure 7] This figure shows an example of the screen display of the results of the visual inspection process performed by the visual inspection system shown in Figure 1. [Figure 8] This figure shows an example flowchart of the visual inspection process using the visual inspection system shown in Figure 1. [Figure 9] This is a longitudinal cross-sectional view of a modified mounting unit. [Modes for carrying out the invention]

[0014] One embodiment of the present invention will be described with reference to the drawings. Figure 1 shows an example of the configuration of a visual inspection system according to one embodiment of the present invention. The visual inspection system shown in Figure 1 includes a mounting unit 20 on which a product 10 to be inspected is placed, a camera 30 that images the product 10 placed on the mounting unit 20 from a specific direction (in this example, the direction of the top surface), a visual inspection device 40 that performs a visual inspection process of the product 10 based on the image captured by the camera 30, and a server 45 that stores data such as the results of the visual inspection process performed by the visual inspection device 40.

[0015] In the visual inspection system of this example, the mounting unit 20 has a mirror 23 positioned to reflect an image of the mounted product 10 viewed from a direction other than a specific direction toward the camera 30. The camera 30 is configured to capture an image that includes an image portion showing the product 10 viewed from a specific direction (the direction toward the camera) and an image portion showing the product 10 viewed from a different direction via the mirror 23. In other words, the camera 30 can capture an image that includes multiple image portions of the product 10 viewed from multiple directions in a single shot. Therefore, the visual inspection device 40 can perform not only a visual inspection process of the product 10 viewed from a specific direction, but also a visual inspection process of the product 10 viewed from a different direction, based on a single image captured by the camera 30.

[0016] FIG. 2 shows a plan view of an exemplary mounting unit 20 used in the appearance inspection system of FIG. 1. FIG. 3 shows a longitudinal sectional view taken along line A-A' of the exemplary mounting unit 20 shown in FIG. 2. The mounting unit 20 of this example includes a base portion 21, a mounting table 22, four mirrors 23A to 23D, four unnecessary light prevention plates 24A to 24D, and two sensor portions 25A and 25B.

[0017] The mounting table 22 is a member that serves as a set position for the product 10 and is provided at the center of the upper surface of the base portion 21. As an example, the mounting table 22 has an upper surface portion formed with a recess corresponding to the shape of the lower portion of the product 10. By using such a mounting table 22, the position and orientation of the product 10 on the mounting table 22 can be fixed simply by fitting the lower portion of the product 10 into the recess of the upper surface portion of the mounting table 22, so that subsequent processing can be simplified.

[0018] Mirrors 23A to 23D provided with an inclination of about 45 degrees with respect to the horizontal plane are arranged in four directions around (the side surface) of the mounting table 22. Further, unnecessary light prevention plates 24A to 24D for preventing the reflection of the background and the intrusion of light are arranged on the back surfaces of the mirrors 23A to 23D, respectively. In this example, unnecessary light prevention plates 24A to 24D that are slightly larger than the mirrors 23A to 23D are used, but as long as the reflection of the background and the intrusion of light can be appropriately prevented, the size, shape, material, etc. are not particularly limited.

[0019] The sensor portion 25A is an example of the first detection portion according to the present invention and detects that the product 10 is mounted on the mounting unit 20. As the sensor portion 25A, for example, a pressure sensor incorporated in the surface portion of the mounting table 22 is used. The sensor portion 25B is an example of the second detection portion according to the present invention and detects that an operating body (for example, the hand of an operator or the arm of a machine) that performs the mounting operation of the product 10 on the mounting unit 20 has moved out of the imaging range of the camera 30. As the sensor portion 25B, for example, a set of an infrared ray emitting portion that emits horizontal infrared rays into the space above the product 10 on the mounting table 22 and an infrared sensor that detects the emitted infrared rays, provided on the diagonal line of the upper surface of the base portion 21, is used.

[0020] The camera 30 is positioned above the mounting unit 20, overlooking it, so that the mounting base 22 and the surrounding mirrors 23A to 23D are within its shooting range (angle of view). Therefore, as shown in Figure 4, the image 50 captured by the camera 30 includes not only an image portion 51 showing the top surface of the product 10 placed on the mounting base 22, but also an image portion 52 showing the image reflected by mirror 23A of the front of the product 10, an image portion 53 showing the image reflected by mirror 23B of the left side of the product 10, an image portion 54 showing the image reflected by mirror 23C of the right side of the product 10, and an image portion 55 showing the image reflected by mirror 23D of the back of the product 10. In this way, by using the mounting unit 20, it is possible to obtain an image 50 showing each side of the product 10 except the bottom surface (i.e., the top, front, left, right, and back) with just one shot.

[0021] The image 50 captured by the camera 30 is supplied to the visual inspection device 40. The visual inspection device 40 performs a visual inspection process based on the image 50 supplied from the camera 30. In this example, there are one or more inspection items for each of the top, front, left, right, and back surfaces of the product 10, and a visual inspection process is performed for each inspection item.

[0022] Figure 5 shows an example of an inspection frame indicating the scope of each inspection item. In the example in Figure 5, inspection frames 61 and 62 are shown for the area of ​​the image showing the top surface of product 10, inspection frame 63 is shown for the area of ​​the image showing the front of product 10, inspection frames 64 and 65 are shown for the area of ​​the image showing the left side of product 10, inspection frame 66 is shown for the area of ​​the image showing the right side of product 10, and inspection frame 67 is shown for the area of ​​the image showing the back of product 10. These inspection frames 61 to 67 are assumed to be pre-set before the operation of the visual inspection system begins.

[0023] The visual inspection device 40 also receives the detection results from sensors 25A and 25B. The visual inspection device 40 controls the operation of the visual inspection process based on the detection results from sensors 25A and 25B. In this example, the visual inspection process is controlled to be executed based on the image captured by camera 30 when sensor 25A detects that the product 10 has been placed on the mounting unit 20, and sensor 25B detects that the operating body has moved out of the camera's shooting range, Y. By performing this control, it is possible to automatically perform the visual inspection process on the image of the product 10 placed on the mounting unit 20, and to eliminate the appearance of operating bodies such as workers' hands or machine arms in the image.

[0024] Here, the image to be subject to the visual inspection process (i.e., an image taken in state X and state Y) can be acquired in various ways. For example, camera 30 may repeatedly take images at predetermined intervals and transmit them to the visual inspection device 40, and the visual inspection device 40 may identify the image taken at the timing of state X and state Y and use that image as the subject of the visual inspection process. As another example, camera 30 may be in a standby state for taking pictures, and the visual inspection device 40 may instruct camera 30 to take pictures at the timing of state X and state Y, and camera 30 may transmit the image it took in response to this instruction to the visual inspection device 40 as the subject of the visual inspection process.

[0025] The visual inspection device 40 inspects the appearance of the product 10 by performing AI analysis on each region of inspection frames 61 to 67 in the image captured by the camera 30 using a pre-trained learning model. If the inspection results for all inspection frames are OK (no defects), the overall inspection result is determined to be OK. On the other hand, if the inspection result for any inspection frame is NG (defect present), the overall inspection result is determined to be NG. The visual inspection device 40 may also perform visual inspection processing using methods other than AI analysis. For example, the visual inspection device 40 may calculate the difference between each region of inspection frames 61 to 67 in the image captured by the camera 30 and a reference image prepared by photographing a defect-free product in advance, and determine that it is OK if the difference is less than a predetermined threshold, and NG otherwise.

[0026] In the above description, states X and Y are detected using sensors 25A and 25B, but this is merely one example, and states X and Y may be detected by other methods. For example, the camera 30 can repeatedly take pictures at predetermined intervals and transmit images to the visual inspection device 40, and the visual inspection device 40 can detect states X and Y by analyzing each image at predetermined intervals. In other words, the visual inspection device 40 may be configured to include the first detection unit and the second detection unit according to the present invention.

[0027] When the visual inspection device 40 performs the visual inspection process as described above, it displays the results of the visual inspection process on a screen. The results of the visual inspection process may be displayed on the display of the visual inspection device 40, or on the display of another device capable of receiving the results of the visual inspection process from the visual inspection device 40.

[0028] Figures 6 and 7 show examples of screen displays of the results of the visual inspection process. Figure 6 is an example of a screen display when all inspection items are OK (no defects). Figure 7 is an example of a screen display when any of the inspection items are NG (defective). The display screen 70 of the results of the visual inspection process has a first display area 71 that displays images captured by the camera 30, a second display area 72 that displays the overall inspection results, and a third display area 73 that displays the inspection results for each inspection item (inspection frames 61-67 in Figure 5) in list format.

[0029] In the first display area 71, not only images captured by the camera 30 but also graphics visualizing the inspection results for each inspection item are overlaid. In this example, white frame displays 81-87 are superimposed on the areas of inspection items where the inspection result was OK, as shown by the dashed lines in Figure 6. Red frame displays 81', 83' are superimposed on the areas of inspection items where the inspection result was NG, as shown by the thick solid lines in Figure 7. Therefore, inspectors can understand which parts of the product 10 have defects simply by looking at the display in the first display area 71.

[0030] The visual inspection device 40 not only displays the results of the visual inspection process on the screen, but also transmits the data to the server 45 for storage. The server 45 stores, for example, data of the inspection results (overall inspection results and inspection results for each inspection item), images captured by the camera 30, and images in which the inspection results for each inspection item are visualized and superimposed on these images, all associated with a product ID that identifies the visually inspected product 10. Therefore, by accessing the server 45, it is possible to utilize the results of the visual inspection process retrospectively.

[0031] The flow of the visual inspection process using the visual inspection system in this example will be explained with reference to the flowchart example shown in Figure 8. Here, the product 10 to be inspected has a barcode (including a two-dimensional barcode) indicating the product ID, and the product ID is detected by reading the barcode with a barcode reader before (or after) placing the product 10 on the placement unit 20 and stored in the server 45. Alternatively, instead of reading with a barcode reader, the visual inspection device 40 may detect the product ID by analyzing the barcode portion of an image taken of the product 10. Or, instead of a barcode for the product ID, the product ID itself may be attached to the product 10, and the product ID may be detected by OCR (Optical Character Reader) processing of an image taken of the product 10.

[0032] When the product 10 to be inspected is placed on the placement unit 20, this is detected by the sensor unit 25A, and the detection result of the sensor unit 25A is input to the visual inspection device 40 (step S11). Also, when the operator that performed the product placement operation moves out of the shooting range of the camera 30, this is detected by the sensor unit 25B, and the detection result of the sensor unit 25B is input to the visual inspection device 40 (step S12).

[0033] Camera 30 repeatedly takes pictures at predetermined intervals, and the images obtained from the pictures are sequentially supplied to the visual inspection device 40. The visual inspection device 40 determines whether the sensor 25A has detected that the product 10 has been placed on the mounting unit 20, and the sensor 25B has detected that the operating object has moved out of the camera 30's shooting range, and performs a visual inspection of the product 10 using the image when the determination is successful (step S13). After that, the visual inspection device 40 displays the results of the visual inspection on the screen (step S14). The visual inspection device 40 also sends the visual inspection result data and image data to the server 45 and stores them in association with the product ID (step S15).

[0034] As described above, the visual inspection system in this example comprises a mounting unit 20 on which the product 10 to be inspected is placed, a camera 30 that images the product 10 placed on the mounting unit 20 from a specific direction, and a visual inspection device 40 that performs a visual inspection process of the product 10 based on the image captured by the camera 30. The mounting unit 20 has a mirror 23 positioned to reflect an image of the mounted product 10 viewed from a direction other than the specific direction toward the camera 30, and the camera 30 is configured to capture an image that includes an image portion showing the product 10 viewed from the specific direction and an image portion showing the product 10 viewed from the other direction via the mirror 23.

[0035] Therefore, with the visual inspection system in this example, by taking a photograph of the product to be inspected only once, it is possible to capture an image that includes multiple image portions of the product viewed from multiple directions. Furthermore, the visual inspection process for multiple directions of the product can be performed using only the captured image. In this way, the visual inspection system in this example makes it possible to easily perform visual inspection of a product from multiple directions.

[0036] Here, the above explanation assumes that the position and orientation of the product 10 on the base 22 of the mounting unit 20 are fixed by forming a recess on the upper surface of the base 22. However, a base 22 having a flat upper surface without such a recess may be used, allowing the position and orientation of the product 10 to be freely changed. That is, the product 10 may be positioned slightly shifted forward or backward or left or right, the product 10 may be positioned slightly tilted relative to the reference direction, or the front of the product 10 may be positioned facing upward.

[0037] However, if the position and orientation of the product 10 are to be changed in this manner, it is necessary to adjust the image captured by the camera 30 or the inspection frame (inspection frames 61-67 shown in Figure 5) according to the position and orientation of the product 10 placed on the stand 22. In this case, for example, the contour lines of the product 10 when the product 10 is placed on the stand 22 in a reference position and orientation are detected in each direction and set as reference contour lines, and the inspection frame is set corresponding to the reference contour lines in each direction. The visual inspection device 40 then detects the contour lines of the product 10 from the image portions in each direction included in the captured image of the product 10 to be inspected, and after adjusting the image portions in each direction by moving, rotating, or deforming so that the contour lines in each direction match the reference contour lines, it performs the visual inspection process. Alternatively, the reference contour lines are adjusted by moving, rotating, or deforming so that the reference contour lines match the contour lines in each direction, and the same adjustment is made to the inspection frame before the visual inspection process is performed. Note that the above adjustment method is merely an example, and adjustments may be made by other methods.

[0038] Furthermore, in the above explanation, images are taken of each surface of product 10 except the bottom surface (i.e., top, front, left, right, and back) and the visual inspection process is performed. However, if there are directions that do not require inspection, the photography of those directions may be omitted. In this case, among the mirrors 23A to 23D, the mirrors in the directions that do not require inspection are unnecessary and may be removed.

[0039] Furthermore, while the above explanation does not allow for visual inspection of the underside of product 10, by modifying or expanding the mounting unit 20, it becomes possible to perform a comprehensive visual inspection of the underside of product 10. This mechanism will be explained with reference to Figure 9. Figure 9 shows a vertical cross-sectional view illustrating a modified version of the mounting unit 20.

[0040] The mounting unit 20 in Figure 9 has an opening 26 that penetrates vertically through the central part of the base 21 and the mounting platform 22 so that the underside of the product 10 can be photographed. An extended base 27 equipped with two mirrors 28A and 28B is attached to the underside of the base 21. Mirror 28A is positioned directly below the opening 26 and is tilted at approximately 45 degrees with respect to the horizontal plane. Mirror 28B is tilted at approximately 45 degrees with respect to the horizontal plane in the opposite direction to mirror 28A so that it reflects the image of the underside of the product 10 seen through the opening 26 towards the camera 30. By using such an extended base 27, it is possible to obtain an image showing all surfaces of the product 10, including the underside, in a single shot.

[0041] Although embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take many other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention as described herein, and are included in the scope of the invention and its equivalents as described in the claims.

[0042] Furthermore, the present invention can be provided not only as the devices described above or as systems composed of such devices, but also as methods executed by these devices, programs for a processor to realize the functions of these devices, and storage media for storing such programs in a computer-readable manner. [Industrial applicability]

[0043] This invention can be used in a visual inspection system that inspects the appearance of a product based on an image of the product taken with a camera. [Explanation of Symbols]

[0044] 10: Product, 20: Mounting unit, 21: Base, 22: Stand, 23A~23D: Mirror, 24A~24D: Anti-flash plate, 25A,25B: Sensor unit, 26: Opening, 27: Extension base, 28A,28B: Mirror, 30: Camera, 40: Visual inspection device, 45: Server

Claims

1. A mounting unit on which the product subject to visual inspection is placed, A camera that images the product placed on the mounting unit from a first direction, The system includes an appearance inspection device that performs an appearance inspection process of the product based on an image captured by the camera, The mounting unit has a mirror positioned to reflect an image of the mounted product viewed from a second direction different from the first direction toward the camera, The camera captures an image that includes a first image portion showing the product as viewed from the first direction and a second image portion showing the product as viewed from the second direction via the mirror. The inspection frame for each of the regions of the first and second image portions is predetermined. The visual inspection device performs the visual inspection process on the inspection frame of the first image portion and the visual inspection process on the inspection frame of the second image portion, respectively, on the image captured by the camera, and displays the results of the visual inspection process on the display of the device itself or another device. The aforementioned screen display includes a display area in which the inspection frame is superimposed on the image captured by the camera, and the display mode of the inspection frame differs depending on the result of the visual inspection process applied to the inspection frame, characterized in that the visual inspection system is characterized in that the screen display includes a display area in which the inspection frame is superimposed on the image captured by the camera, and the display mode of the inspection frame differs depending on the result of the visual inspection process applied to the inspection frame.

2. In the visual inspection system described in claim 1, A first detection unit that detects that the product has been placed on the mounting unit, The system further includes a second detection unit that detects when the operating body used to place the product onto the mounting unit moves out of the camera's shooting range, The appearance inspection system is characterized in that the appearance inspection device performs the appearance inspection process based on an image captured by the camera when the first detection unit detects that the product has been placed on the aforementioned placement unit, and the second detection unit detects that the operating body has moved out of the camera's shooting range.

3. In the visual inspection system described in claim 1, The appearance inspection system is characterized in that the appearance inspection device determines the position of the product in the aforementioned placement unit, adjusts the image or the inspection frame according to the position of the product, and then performs the appearance inspection process.

4. In the visual inspection system described in claim 1, The appearance inspection system is characterized in that the appearance inspection device determines the orientation of the product in the aforementioned placement unit, adjusts the image or the inspection frame according to the orientation of the product, and then performs the appearance inspection process.

5. In the visual inspection system according to Claim 1, An appearance inspection system characterized by further comprising a light-blocking plate larger than the mirror on the back side of the mirror as seen from the camera.

Images