Workpiece inspection apparatus and workpiece inspection method

Abstract

To provide a workpieces inspection technique which is effective in facilitating inspection of a plurality of kinds of workpieces.SOLUTION: A workpiece inspection apparatus 101 includes: a camera 21 for imaging; a monitor 22 which can display a three-dimensional real image Mb captured by the camera 21; a control unit 40 for controlling the camera 21 and the monitor 22; and a memory 50 storing three-dimensional master images of a plurality of kinds of vehicle bodies 1. The control unit 40 includes: an image processing unit which performs shape fitting processing of reading from a memory 50, when an arbitrary workpiece W is selected as an inspection object from among the plurality of kinds of vehicle bodies 1, a master image of the workpiece W in a state where a real image Mb when capturing the workpiece W by the camera 21 is displayed on the monitor 22, and superimposing the master image on the real image Mb; and a comparison information output unit which outputs comparison information between the real image Mb and the master image, to the monitor 22, on the basis of a result of the shape fitting.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a work inspection device and a work inspection method for inspecting a work. [Background technology]

[0002] The following Patent Document 1 discloses a three-dimensional measuring method for measuring and analyzing a completed vehicle using an existing three-dimensional measuring machine. This three-dimensional measuring method uses a lift device that supports the vehicle body and a three-dimensional measuring machine installed at fixed positions on both the left and right sides of the lift device.

[0003] The following Patent Document 2 discloses a component inspection method and system for easily detecting abnormalities in captured images of a component to be inspected. In this technology, an image capturing device captures images of the component from multiple locations at a fixed position, and it is determined whether the captured image data is defective image data or not. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-3536 A [Patent Document 2] WO2020 / 031984 A1 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology disclosed in Patent Document 1, the coordinate measuring machine used for inspection is fixed in advance to a facility such as a factory. Therefore, when inspecting a vehicle, it is necessary to align the relative position of the inspection target part of the vehicle supported by the lift device with the coordinate measuring machine, which causes a problem that the vehicle inspection work takes time. To solve this problem, a portable image capturing device can be adopted as the image capturing device disclosed in Patent Document 2. By detecting defects in the inspection target part using image data obtained by capturing an image of the inspection target part with this image capturing device, it becomes possible to reduce the time required for the inspection work.

[0006] However, the technology disclosed in Patent Document 2 is premised on inspecting parts with a fixed shape. Therefore, it has a problem that it is not possible to select and inspect an arbitrary inspection target from among multiple inspection targets. For example, this is disadvantageous when performing continuous inspection work on multiple types of workpieces, such as in a production line for car bodies.

[0007] The present invention has been made in consideration of the above problems, and aims to provide a work inspection technique that is effective for easily inspecting a plurality of types of work. [Means for solving the problem]

[0008] One aspect of the present invention is A work inspection device for inspecting a plurality of types of work, A camera for taking pictures, A monitor capable of displaying a three-dimensional actual image captured by the camera; A control unit for controlling the monitor; A storage unit for storing three-dimensional master images of each of the plurality of types of workpieces; Equipped with The control unit, when an arbitrary workpiece is selected from the plurality of types of workpieces as an inspection object, performs a shape fitting process of reading out the master image of the inspection object from the storage unit and superimposing the master image on the actual image while displaying the actual image of the inspection object photographed by the camera on the monitor; a comparison information output unit that outputs a comparison information between the actual image and the master image on the monitor based on a result of the shape fitting process by the image processing unit; A workpiece inspection apparatus having is located.

[0009] Another aspect of the present invention is A work inspection method for inspecting a plurality of types of work, comprising the steps of: A preparation step of storing three-dimensional master images of each of the plurality of types of workpieces in a storage unit; When an arbitrary workpiece is selected from the plurality of types of workpieces as an inspection object, an image processing step is performed in which a shape fitting process is performed in which the master image of the inspection object is read from the storage unit and superimposed on the actual image while an actual image of the inspection object photographed by a camera is displayed on a monitor. a comparison information output step of outputting comparison information between the actual image and the master image to the monitor based on a result of the shape fitting process by the image processing step; A workpiece inspection method comprising: is located. Effect of the Invention

[0010] In each of the above-mentioned aspects, when an arbitrary workpiece is selected from among a plurality of types of workpieces as an inspection target, a shape fitting process is executed. In this shape fitting process, a master image of the inspection target is read from a storage unit and superimposed on the actual image while an actual image of the inspection target is displayed on a monitor when the inspection target is photographed by a camera. Then, based on the result of the shape fitting process at this time, comparison information between the actual image and the master image is output to the monitor. This allows the inspection of the workpiece W to be completed in a short time.

[0011] Users can simply take a picture of the object to be inspected using a camera and check the comparison information between the actual image and the master image on the monitor. This reduces the time required for workpiece inspection work, making it possible to easily inspect workpieces. In addition, by selecting any workpiece from multiple types of workpieces as the inspection object, it becomes possible to easily inspect multiple types of workpieces in the same way.

[0012] As described above, according to each of the above-mentioned aspects, it is possible to provide a work inspection technique that is effective for easily inspecting a plurality of types of workpieces. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a perspective view showing the work inspection device of the first embodiment in a state before a work is placed on a base. [Diagram 2] FIG. 2 is a diagram showing the state when a workpiece is photographed using the workpiece inspection device of FIG. 1. [Diagram 3] FIG. 2 is a system configuration diagram of the mobile terminal in FIG. [Figure 4] FIG. 4 is a system configuration diagram of a control unit of the mobile terminal of FIG. 3 . [Diagram 5] 3 is a flowchart of a workpiece inspection method according to the first embodiment. [Figure 6] FIG. 13 is a diagram showing an example of a database of inspection object names and master images. [Figure 7] FIG. 6 is a diagram for explaining the shape fitting process in FIG. 5 . [Figure 8] FIG. 4 is a diagram showing an example of comparison information output to a monitor in the first embodiment. [Figure 9] FIG. 11 is a diagram showing a state in which a workpiece is photographed using the workpiece inspection device of the second embodiment. [Figure 10] 10 is a flowchart of a workpiece inspection method according to a second embodiment. [Figure 11] FIG. 11 is a diagram for explaining the shape fitting process in FIG. 10 . [Figure 12] FIG. 11 is a diagram showing an example of comparison information output to a monitor in the second embodiment. [Figure 13]13A and 13B are diagrams showing actual images and master images of cross-sectional shapes of distortion. [Figure 14] 11A to 11C are diagrams for explaining the process of changing the superimposition position in FIG. 10; [Figure 15] 11 is a diagram showing a cross-sectional shape extracted in the overlapping position change process in FIG. 10 . [Figure 16] FIG. 11 is a diagram showing a state in which a workpiece is photographed using the workpiece inspection device of the third embodiment. [Figure 17] 11 is a flowchart of a workpiece inspection method according to a third embodiment. [Figure 18] FIG. 18 is a diagram for explaining the shape fitting process in FIG. 17 . [Figure 19] FIG. 11 is a diagram showing an example of comparison information output to a monitor in the third embodiment. [Figure 20] FIG. 11 is a diagram showing an example of comparison information output to a monitor in the third embodiment. [Figure 21] FIG. 13 is a diagram showing a state in which a workpiece is photographed using the workpiece inspection device of embodiment 4. [Figure 22] 10 is a flowchart of a workpiece inspection method according to a fourth embodiment. [Figure 23] FIG. 13 is a diagram showing an example of comparison information output to a monitor in the fourth embodiment. [Figure 24] FIG. 13 is a diagram showing an example of comparison information output to a monitor in the fourth embodiment. [Diagram 25] FIG. 13 is a diagram showing an example of comparison information output to a monitor in the fourth embodiment. [Figure 26] FIG. 13 is a diagram showing an example of information output to a monitor in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Preferred embodiments of the above aspects are described below.

[0015] It is preferable that the work inspection device of the above-mentioned aspect is provided with a base for placing the work, and both the actual image and the master image are images including the work placed on the base and a reference portion provided on the base, and that the image processing unit uses the reference portion of the base as a reference for overlaying the master image on the actual image in the shape fitting process, and that the comparison information output unit outputs information regarding the amount of positional deviation of the actual image relative to the master image to the monitor as the comparison information.

[0016] According to this workpiece inspection device, the correct position of the workpiece in the master image can be guaranteed by using the reference part of the base as the reference for overlaying the master image on the actual image in the shape fitting process for overlaying the master image on the actual image. This makes it possible to accurately measure information regarding the amount of positional deviation of the actual image from the master image and output it to the monitor as comparison information.

[0017] In the workpiece inspection device of the above aspect, it is preferable that the base has wheels for running.

[0018] According to this workpiece inspection device, the platform can be easily moved using wheels as needed to inspect the workpiece. This means that the location where the workpiece can be inspected is less likely to be limited, and it is possible to quickly respond to sudden requests for workpiece inspection.

[0019] In the work inspection device of the above aspect, it is preferable that the comparison information output section outputs information relating to an external appearance of the work to the monitor as the comparison information.

[0020] According to this workpiece inspection device, it is possible to output information about the external appearance of the workpiece to a monitor as comparison information based on the results of a shape fitting process in which a master image of the workpiece is superimposed on an actual image.

[0021] In the work inspection device of the above-mentioned aspect, it is preferable that the image processing unit changes the relative position of the overlay direction of the overlay surface of the master image with respect to the actual image in response to a user's operation to change the overlay position after the shape fitting process.

[0022] According to this workpiece inspection device, the user can know the overlapping state between the overlapping surfaces of the real image and the master image at any relative position by executing an overlapping position change operation, which allows the user to easily grasp the three-dimensional shape of the real image.

[0023] In the work inspection device of the above aspect, it is preferable that the work includes an attachment part, and the comparison information output unit outputs information about the attachment part in the work to the monitor as the comparison information.

[0024] According to this workpiece inspection device, it is possible to output information about the attached parts of the workpiece as comparison information to a monitor based on the results of a shape fitting process in which a master image of the workpiece is superimposed on an actual image.

[0025] In the work inspection device of the above-mentioned aspect, when there is a difference between the actual image and the master image based on the result of the shape fitting process, it is preferable that the comparison information output unit outputs information that exaggerates the difference as the comparison information to the monitor.

[0026] This workpiece inspection device allows the user to easily visualize the differences between the actual image and the master image on a monitor.

[0027] In the work inspection device of the above-mentioned aspect, when there is a difference between the actual image and the master image based on the result of the shape fitting process, it is preferable that the comparison information output unit outputs information about the cause of the difference to the monitor as the comparison information.

[0028] This workpiece inspection device enables the user to visually confirm on the monitor the cause of the difference between the actual image and the master image.

[0029] In the workpiece inspection device of the above aspect, it is preferable that the camera, the monitor, and the control unit are incorporated into one mobile terminal.

[0030] According to this workpiece inspection device, by using a portable terminal that is easy to carry and has excellent operability, it is possible to easily and quickly inspect the workpiece.

[0031] In the work inspection method of the above-mentioned aspect, a pedestal is used for placing the work, and both the actual image and the master image are images including the work placed on the pedestal and a reference portion provided on the pedestal, and in the shape fitting process, the reference portion of the pedestal is used as a reference for overlaying the master image with the actual image, and in the comparison information output step, it is preferable that information regarding the amount of positional deviation of the actual image with respect to the master image is output to the monitor as the comparison information.

[0032] According to this workpiece inspection method, the reference part of the base is used as the reference for overlaying the master image on the actual image in the shape fitting process, thereby ensuring the correct position of the workpiece in the master image. This makes it possible to accurately measure information regarding the amount of positional deviation of the actual image from the master image and output it to the monitor as comparative information.

[0033] In the workpiece inspection method according to the above aspect, it is preferable that in the comparison information output step, information relating to an external appearance of the workpiece is output to the monitor as the comparison information.

[0034] According to this workpiece inspection method, it is possible to output information about the external surface of the workpiece as comparison information to a monitor based on the results of a shape fitting process in which a master image of the workpiece is superimposed on an actual image.

[0035] It is preferable that the workpiece inspection method of the above-mentioned aspect includes, after the shape fitting process, an overlay position changing step of changing the relative position of the overlay direction of the overlay surface of the master image with respect to the actual image in accordance with an overlay position changing operation by a user.

[0036] According to this workpiece inspection method, the user can know the overlapping state between the overlapping surfaces of the real image and the master image at any relative position by executing an overlapping position change operation, which allows the user to easily grasp the three-dimensional shape of the real image.

[0037] In the work inspection method of the above aspect, it is preferable that the work includes an attachment part, and in the comparison information output step, information relating to the attachment part of the work is output to the monitor as the comparison information.

[0038] According to this workpiece inspection method, it is possible to output information about the attached parts of the workpiece as comparison information to a monitor based on the results of a shape fitting process in which a master image of the workpiece is superimposed on an actual image.

[0039] In the work inspection method of the above aspect, in the comparison information output step, if there is a difference between the actual image and the master image based on the result of the shape fitting process, it is preferable that information in which the difference is exaggerated is output to the monitor as the comparison information.

[0040] According to this workpiece inspection method, the user can easily visualize the differences between the actual image and the master image on the monitor.

[0041] In the work inspection method of the above-mentioned aspect, in the comparison information output step, if there is a difference between the actual image and the master image based on the result of the shape fitting process, it is preferable that information regarding the cause of the difference is output to the monitor as the comparison information.

[0042] According to this workpiece inspection method, the user can visually confirm on the monitor the cause of the difference between the actual image and the master image.

[0043] Hereinafter, an embodiment for realizing an inspection technique for inspecting a plurality of types of workpieces will be described with reference to the drawings.

[0044] In addition, in the drawings for explaining the embodiments, unless otherwise specified, a first direction which is the horizontal direction of the work inspection device and the work is indicated by arrow X, a second direction which is the horizontal direction of the work inspection device and the work and perpendicular to the first direction is indicated by arrow Y, and a third direction which is the vertical direction of the work inspection device and the work is indicated by arrow Z.

[0045] (Embodiment 1) As shown in Fig. 1, a work inspection device (hereinafter, for convenience, simply referred to as "inspection device") 101 of embodiment 1 is a device for inspecting multiple types of workpieces W. This inspection device 101 is used in particular to inspect the positional deviation of each workpiece W. In this embodiment, the workpiece W is a vehicle body 1 of an automobile.

[0046] 1. Structure of Pedestal 10 The inspection device 101 includes a pedestal 10 for placing a workpiece W thereon. The pedestal 10 includes a main body 11 having a plurality of wheels 11a for traveling, a plurality of support parts 12, and a plurality of reference parts 13. The plurality of support parts 12 are provided on the main body 11, and function to support the workpiece W from below. The pedestal 10 also includes reference pins (not shown) for determining the position of the workpiece W in the first direction X and the second direction Y.

[0047] The base 10 is also called a "cart" because it is a movable type with multiple wheels 11a provided on the main body 11. If the base 10 with wheels 11a is used, the location where the workpiece W is inspected is not limited, and it can be moved to an appropriate location as needed. In the case of a stationary structure in which the base 10 does not need to be moved, the wheels 11a of the main body 11 may be omitted.

[0048] The multiple reference parts 13 are provided on the main body 11 of the pedestal 10 and are used when photographing the workpiece W. The positional accuracy of the multiple reference parts 13 in the main body 11 is guaranteed. Therefore, when the position of the workpiece W in the first direction X and the second direction Y relative to the pedestal 10 is determined by the reference pin, the relative position between the workpiece W and the multiple reference parts 13 is uniquely determined. In this embodiment, the multiple reference parts 13 are attached to both sides of the main body 11 in the first direction X so as not to interfere with the photographing of the vehicle body 1 placed on the pedestal 10, and each is formed in a substantially L-shape. Note that, as long as the condition of not interfering with the photographing of the workpiece W is satisfied, the mounting position and shape of each reference part 13 are not limited to those in FIG. 1 and can be appropriately changed as necessary.

[0049] 2. Structure of the mobile terminal 20 1 and 2, the inspection device 101 includes one mobile terminal 20. The mobile terminal 20 is a thin and lightweight information device, and is also called a mobile terminal or a tablet terminal. The mobile terminal 20 is integrally equipped with a camera 21 for photographing, a monitor 22 capable of displaying a three-dimensional actual image Mb photographed by the camera 21, a control unit 40 for controlling the camera 21 and the monitor 22, and a memory 50 serving as a storage unit. The camera 21 is exposed on one surface of the mobile terminal 20 in the thickness direction, and the monitor 22 is exposed on the other surface of the mobile terminal 20 in the thickness direction.

[0050] The camera 21 has a built-in imaging element (not shown) that receives light reflected from the surface of the workpiece W. Typically, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used as the imaging element.

[0051] The mobile terminal 20 is preferably attached to a fixed member 20a such as a tripod when used. This makes it possible to prevent the effects of camera shake when taking pictures with the camera 21, and enables the workpiece W to be photographed with high accuracy by the camera 21. In addition, the mobile terminal 20 is preferably equipped with a lighting function or a flash function to be used together with the camera 21. This makes it possible to prevent the effects of backlighting and the like when the camera 21 is used.

[0052] 3, the mobile terminal 20 incorporates a camera 21, a monitor 22, an image capture board 30, a control unit 40, a memory 50, a graphic memory 51, and a communication unit 52. The mobile terminal 20 is provided with a user interface (e.g., a mouse, a keyboard, a touch panel, a touch pad, etc.) that is operated by a user. An input operation through the user interface is converted into an electrical signal and input to the control unit 40.

[0053] The image capture board 30 has a function of outputting image information transmitted from the camera 21 to the control unit 40. The image capture board 30 is provided with an input unit 31 and a frame buffer 32. The input unit 31 acquires image information transmitted from the camera 21 and outputs it to the frame buffer 32. The frame buffer 32 converts the image information output from the input unit 31 into a data format that allows the control unit 40 to execute various processes. The frame buffer 32 stores the image information after the data format is converted and outputs it to the control unit 40. Note that, although the present embodiment illustrates a case in which the image capture board 30 is built into the mobile terminal 20, the image capture board 30 may be separate from the mobile terminal 20 as long as it is capable of communicating with the control unit 40.

[0054] The control unit 40 is a processor configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor) or an FPGA (Field-Programmable Gate Array), and has a function of controlling the operation of each part of the mobile terminal 20. The control unit 40 cooperates with the memory 50 to comprehensively perform various processes and controls. Specifically, the processor 40 refers to the programs and data stored in the memory 50 and executes the programs to realize the functions of each part. The control unit 40 is capable of transmitting and receiving data to and from an external database 60 via the communication unit 52.

[0055] The memory 50 as an example of a storage unit includes, for example, a RAM (Random Access Memory) used when executing each process of the control unit 40, and a ROM (Read Only Memory) that stores programs and data that define the operation of the control unit 40. The RAM temporarily stores data or information generated or acquired by the control unit 40. The ROM has a program that defines the operation of the control unit 40 written therein. The memory 50 stores three-dimensional master images of each of the multiple types of workpieces W. The memory 50 may also store AI software that includes a trained model (not shown) that is used to compare the actual image Mb with the master image Ma.

[0056] The graphic memory 51 is, for example, a VRAM (Video Random Access Memory) and functions as a memory used when executing display processing of an image (screen). The graphic memory 51 executes display processing of an image generated and output by the control unit 40, and displays it on the monitor 22.

[0057] The monitor 22 is configured using a display such as an LCD (Liquid Crystal Display) or an organic EL (electro-luminescence) display, etc. The monitor 22 displays images output from the graphic memory 51, etc.

[0058] 4, the control unit 40 includes an image acquisition unit 41, a master image acquisition unit 42, an image processing unit 43, and a comparison information output unit 44. The image acquisition unit 41 has a function of acquiring image information (i.e., the actual image Mb captured by the camera 21) from the frame buffer 32. The master image acquisition unit 42 has a function of acquiring the master image Ma read out from the memory 50. The image processing unit 43 has a function of performing a shape fitting process of superimposing the master image Ma acquired by the master image acquisition unit 42 on the actual image Mb acquired by the image acquisition unit 41. The comparison information output unit 44 has a function of outputting comparison information R between the actual image Mb and the master image Ma to the monitor 22 via the graphic memory 51 based on the result of the shape fitting process by the image processing unit 43.

[0059] 3. Workpiece inspection method A workpiece inspection method (hereinafter, for convenience, simply referred to as "inspection method") according to embodiment 1 will be described with reference to Fig. 5 to Fig. 8. This inspection method is a method for inspecting an inspection object that is an arbitrary workpiece W among a plurality of types of workpieces W. In this inspection method, the inspection device 101 having the above-mentioned configuration is used.

[0060] In the inspection method of the first embodiment, first, a preparation step S101 is executed in which a three-dimensional master image Ma of each of a plurality of types of workpieces W is stored in the memory 50. The master image Ma is a model image (reference image) including the workpiece W placed on the pedestal 10 and the reference portion 13 provided on the pedestal 10. An inspection target is selected from the plurality of types of workpieces W.

[0061] 6, according to the preparation step S101, for example, a master image Ma_1 of the inspection object T-1, a master image Ma_2 of the inspection object T-2, a master image Ma_3 of the inspection object T-3, and a master image Ma_4 of the inspection object T-4 are stored in the memory 50. After that, the processes from step S102 to step S105 are executed in sequence.

[0062] Step S101a is a step of setting an inspection object on the base 10. Step S102 is a step of the user selecting an inspection object on the monitor 22. As shown in FIG. 7, in this step S102, the user selects an inspection object on the selection unit 23 on the monitor 22. The inspection object is any workpiece W selected from a plurality of types of workpieces W. An identification code such as a QR code (registered trademark) may be used to select the inspection object. According to this step S102, the master image Ma to be used in step S104 is determined. For example, if the user selects T-1 as the inspection object, the master image Ma_1 is used (see FIG. 6).

[0063] In step S103, the user photographs the workpiece W, which is the inspection target selected in step S102, with the camera 21 of the mobile terminal 20. At this time, the user adjusts the orientation of the camera 21 to a desired orientation while checking the actual image Mb displayed on the monitor 22 in real time. According to this step S103, the name of the workpiece W, which is the inspection target, is displayed on the selection section 23 of the monitor 22, and the actual image Mb of the workpiece W photographed by the camera 21 is displayed on the monitor 22 (see FIG. 2). The user photographs the workpiece W and the multiple reference parts 13 of the base 10 with the camera 21. Therefore, the actual image Mb is an image including the workpiece W placed on the base 10 and the multiple reference parts 13 provided on the base 10. At this time, the relative positions of the workpiece W and the multiple reference parts 13 are uniquely determined as described above. In step S103, in order to improve the inspection accuracy of the workpiece W, it is preferable to use a lighting function or a flash function when photographing with the camera 21.

[0064] Step S104 is an image processing step for performing a shape fitting process when a workpiece W to be inspected is selected in step S102. The shape fitting process at this time is a process for superimposing a master image Ma of the workpiece W on the actual image Mb of the workpiece W in a state in which the actual image Mb of the workpiece W is displayed on the monitor 22 (see FIG. 7). This shape fitting process is executed using a known program for AR (augmented reality) stored in the memory 50. As shown in FIG. 8, in this embodiment, in this shape fitting process, a plurality of reference portions 13 of the base 10 are used as a reference for superimposing the master image Ma on the actual image Mb.

[0065] Step S105 is a comparison information output step in which comparison information R between the actual image Mb and the master image Ma is output to the monitor 22 based on the result of the shape fitting process in step S104. The comparison information R at this time is, for example, the positional deviation amount Δd between the correct position and the actual measured position of the B pillar of the workpiece W, as shown in Fig. 8. In this embodiment, the correct position of the workpiece W (B pillar) in the master image Ma can be guaranteed by always performing image recognition of the multiple reference portions 13 of the base 10. This makes it possible to improve the measurement accuracy of the positional deviation amount Δd of the workpiece W.

[0066] In addition, when a different work W is to be selected from among multiple types of work W, or when a different work W of the same type is to be selected, the processing of step S101a and steps S102 to S105 in FIG. 5 is also executed for the selected work W.

[0067] 4. Effects Next, the effects of the above-mentioned embodiment 1 will be described.

[0068] In the first embodiment, when an arbitrary workpiece W is selected as an inspection target from among a plurality of types of workpieces W, a shape fitting process is executed. In this shape fitting process, a master image Ma of the inspection target is read from the memory 50 and superimposed on the actual image Mb while an actual image Mb of the inspection target photographed by the camera 21 of the mobile terminal 20 is displayed on the monitor 22 of the mobile terminal 20. Then, based on the result of the shape fitting process at this time, comparison information R between the actual image Mb and the master image Ma is output to the monitor 22 of the mobile terminal 20. This allows the inspection of the workpiece W to be completed in a short time.

[0069] The user can simply photograph the inspection target using the camera 21 of the mobile terminal 20, and check the comparison information R between the actual image Mb and the master image Ma on the monitor 22 of the mobile terminal 20. Therefore, the time required for the inspection of the workpiece W can be reduced, and the inspection of the workpiece W can be easily performed. In addition, by selecting an arbitrary workpiece W from among multiple types of workpieces W as the inspection target, it becomes possible to easily perform the inspection of multiple types of workpieces W in the same manner.

[0070] As described above, according to the first embodiment, a workpiece inspection technique that is effective for easily inspecting a plurality of types of workpieces W can be provided.

[0071] According to the first embodiment, in the shape fitting process of superimposing the master image Ma of the workpiece W on the actual image Mb, the correct position of the workpiece W in the master image Ma can be guaranteed by using the multiple reference portions 13 of the base 10 as a reference for superimposing the master image Ma on the actual image Mb. Therefore, it becomes possible to accurately measure information regarding the amount of positional deviation of the actual image Mb from the master image Ma as comparison information and output it to the monitor 22.

[0072] According to the first embodiment, the pedestal 10 can be easily moved by using the wheels 11a as necessary to perform the inspection of the workpiece W. This makes it difficult to limit the location where the workpiece W is inspected, and makes it possible to quickly respond to an urgent request to inspect the workpiece W. If a defect occurs in the workpiece W, the time required from the occurrence of the defect to the implementation of the inspection can be shortened.

[0073] According to the first embodiment, by using the portable terminal 20 that is easy to carry and has excellent operability, it is possible to easily and quickly inspect the workpiece W. As long as the base 10 provided with the reference portion 13 and the portable terminal 20 are prepared, the inspection of the workpiece W can be carried out anywhere. This has the advantage that no special skills are required to inspect the workpiece W, except for the task of placing the workpiece W on the base 10.

[0074] Hereinafter, other embodiments related to the above-mentioned embodiment 1 will be described with reference to the drawings. In the other embodiments, the same elements as those in embodiment 1 are given the same reference numerals, and the description of the same elements will be omitted.

[0075] (Embodiment 2) As shown in FIG. 9, the inspection device 102 of the second embodiment is used to inspect the exterior surface 2 of the workpiece W. In this embodiment, the workpiece W is an automobile body 1. The inspection device 102 includes a mobile terminal 20 similar to that of the inspection device 101 of the first embodiment. The specific structure of the mobile terminal 20 is similar to that of the first embodiment (see FIGS. 3 and 4). On the other hand, the inspection device 102 differs from the inspection device 101 of the first embodiment in that the reference portion 13 of the base 10 is not required.

[0076] Next, an inspection method of the second embodiment will be described with reference to Fig. 10 to Fig. 15. This inspection method is a method for inspecting the appearance surface 2 of an inspection target, which is an arbitrary workpiece W among a plurality of types of workpieces W. In this method, the inspection device 102 having the above-mentioned configuration is used.

[0077] 10, the processes from step S201 to step S205 of the inspection method of embodiment 2 are similar to the processes from step S101 and step S102 to step S105 (see FIG. 5) of embodiment 1. On the other hand, the inspection method of embodiment 2 further includes step S206.

[0078] In step S201, a three-dimensional master image Ma of each of the appearance surfaces 2 of the multiple types of workpieces W is stored in the memory 50. In step S202, the user selects an object to be inspected (i.e., the appearance surface 2 of any of the multiple types of workpieces W) using the selection unit 23 on the monitor 22.

[0079] 9 and 10, in step S203, the user, while holding the mobile terminal 20 with his / her fingers, photographs the side surface, which is the exterior surface 2, of the workpiece W to be inspected from the side with the camera 21 of the mobile terminal 20. At this time, the user adjusts the orientation of the camera 21 to a desired orientation while checking the actual image Mb displayed on the monitor 22 in real time.

[0080] Step S204 is an image processing step for performing a shape fitting process when a workpiece W to be inspected is selected in step S202. The shape fitting process at this time is a process for superimposing a master image Ma of the workpiece W on the actual image Mb of the workpiece W in a state in which the actual image Mb of the workpiece W is displayed on the monitor 22 (see FIG. 11). This shape fitting process is executed using a known AR program stored in the memory 50, as in the case of the first embodiment.

[0081] Step S205 is a comparison information output step in which comparison information R between the actual image Mb and the master image Ma is output to the monitor 22 based on the result of the shape fitting process in step S204. In this embodiment, the comparison information R is information about the appearance surface 2 of the workpiece W (e.g., the position, shape, classification, etc. of defects on the appearance surface 2 of the workpiece W) as shown in FIG. 12. In this case, the defects can be plotted with figures or symbols on the monitor 22, or the plots can be colored. The defects can be classified using names such as scratches, distortions, convexities, flares, and crooked edges, or numbers and symbols.

[0082] In step S205, as information on the appearance surface 2 of the workpiece W, for example, as shown in Fig. 13, the actual image Mb can be displayed together with the master image Ma for the cross-sectional shape of the distortion 3. In this case, it is preferable to apply highlighting processing such as coloring, shading, or filling in the areas where there are differences between the actual image Mb and the master image Ma. This allows the user to easily recognize the areas where there are differences between the actual image Mb and the master image Ma.

[0083] Step S206 is an overlapping position changing step for changing the relative position of the master image Ma with respect to the real image Mb in the overlapping direction (first direction X) in response to an overlapping position changing operation by the user after the shape fitting process of step S205. According to this step S206, it is possible to extract a cross-sectional shape when the real image Mb is cut by the overlapping plane of the master image Ma at each relative position.

[0084] Step S206 is executed by the image processing unit 43 (see FIG. 4) of the control unit 40. Note that, as shown in FIG. 14, in the shape fitting process of step S205, the first position P1 in the first direction X of the master image Ma approximately coincides with the position of the actual image Mb.

[0085] In this embodiment, the user performs an operation to change the superimposition position using the change unit 24 displayed on the monitor 22. In the change unit 24, the user selects an arbitrary defect from among a plurality of defects after the shape fitting process, and then inputs the amount of relative position change (shift amount) in the first direction X.

[0086] 14, for example, when a distortion 3 detected on the appearance surface 2 of the workpiece W is selected, the superimposed surface of the master image Ma can be shifted from a first position P1 to a second position P2 with respect to the actual image Mb of the distortion 3, and further shifted to a third position P3. The amount of shifting at this time can be set in units of 0.1 mm, for example.

[0087] According to step S206, the user can check the change in the overlap state between the superimposed surfaces of the actual image Mb and the master image Ma of the selected defect, distortion 3, on the monitor 22. At this time, the overlap dimensions between the superimposed surfaces of the actual image Mb and the master image Ma may be displayed numerically on the monitor 22 for each of the first position P1, the second position P2, and the third position P3. In this case, quantitative evaluation of the defect of the appearance surface 2 of the workpiece W becomes possible.

[0088] Also, it may be possible to confirm the change in the cross-sectional shape in the first direction X when the actual image Mb of the selected defect distortion 3 is cut by the superimposed surface of the master image Ma. As shown in Fig. 15, the cross-sectional shape of distortion 3 has a first shape 3a when the superimposed surface of the master image Ma is at a first position P1, a second shape 3b when the superimposed surface of the master image Ma is at a second position P2, and a third shape 3c when the superimposed surface of the master image Ma is at a third position P3.

[0089] The other configurations and methods are the same as those of the first embodiment.

[0090] According to the second embodiment, it is possible to output information about the appearance surface 2 of the workpiece W as comparison information to the monitor 22 based on the result of the shape fitting process in which the master image Ma of the workpiece W is superimposed on the actual image Mb. This allows the inspection of the appearance surface 2 of the workpiece W to be easily performed by anyone anywhere. In addition, it is possible to reduce the number of people required for the inspection process of the appearance surface 2 of the workpiece W.

[0091] According to the second embodiment, the user can know the overlapping state between the real image Mb and the master image Ma at any relative position by executing the overlapping position change operation in step S206. This allows the user to easily grasp the three-dimensional shape of the real image Mb.

[0092] In addition, the same effects as those of the first embodiment are achieved.

[0093] (Embodiment 3) The inspection device 103 of the third embodiment shown in FIG. 16 is used to inspect attachment parts 15, 16 (see FIGS. 18 to 20) of a workpiece W. In this embodiment, the workpiece W is a dash panel 1' (a partition plate separating an engine room from a cabin) constituting the body of an automobile. The inspection device 103 includes a mobile terminal 20 similar to that of the inspection device 101 of the first embodiment. The specific structure of the mobile terminal 20 is similar to that of the first embodiment (see FIGS. 3 and 4). On the other hand, the inspection device 103 differs from the inspection device 101 of the first embodiment in that the reference portion 13 of the base 10 is not required.

[0094] The workpiece W is photographed by the camera 21 while placed on the pedestal 10. At this time, it is preferable to use a flexible arm 14 to prevent the influence of camera shake when photographing with the camera 21 of the mobile terminal 20. The flexible arm 14 extends from a fixed part 10a provided on the pedestal 10, and serves to hold the mobile terminal 20.

[0095] Next, a work inspection method according to the third embodiment will be described with reference to Figs. 16 to 20. This work inspection method is a method for inspecting attachment parts 15, 16 (see Figs. 18 to 20) to be inspected, which are any of a plurality of types of workpieces W. In this method, the inspection device 103 having the above-mentioned configuration is used.

[0096] As shown in FIG. 17, the processes from step S301 to step S305 of the inspection method of the third embodiment are similar to the processes from step S101 and step S102 to step S105 of the first embodiment (see FIG. 5).

[0097] In step S301, a three-dimensional master image Ma of each of the multiple types of workpieces W in a state where the attachment parts 15, 16 are normally attached is stored in the memory 50. In step S302, the user selects an inspection target (i.e., any of the multiple types of workpieces W) in the selection unit 23 on the monitor 22. Also, as shown in FIG. 19, in step S302, the user selects a variation of the type, number, combination, etc. of the attachment parts 15, 16 from three variations A, B, and C in the selection unit 25 on the monitor 22. In this embodiment, a state in which "variation B" is selected in which the attachment parts 15 are stud bolts and the attachment parts 16 are nuts is illustrated.

[0098] In step S303, the workpiece W to be inspected is photographed by the camera 21 of the mobile terminal 20 held by the flexible arm 14 (see FIG. 16). At this time, the user adjusts the orientation of the camera 21 to a desired orientation while checking the actual image Mb displayed on the monitor 22 in real time.

[0099] Step S304 is an image processing step for performing a shape fitting process when a workpiece W to be inspected is selected in step S302. The shape fitting process at this time is a process for superimposing a master image Ma of the workpiece W on the actual image Mb of the workpiece W in a state in which the actual image Mb of the workpiece W is displayed on the monitor 22 (see FIG. 18). This shape fitting process is executed using a known AR program stored in the memory 50, as in the case of the first embodiment.

[0100] Step S305 is a comparison information output step in which comparison information R between the actual image Mb and the master image Ma is output to the monitor 22 based on the result of the shape fitting process in step S304. The comparison information R at this time is, for example, the presence or absence of the attachment parts 15, 16 in the workpiece W and the attachment positions of the attachment parts 15, 16, as shown in Figures 19 and 20. In this embodiment, the correct position of the attachment part 15 is the attachment positions Q1, Q2, Q3, Q4, Q5, and Q6, and the correct position of the attachment part 16 is the attachment positions Q7, Q8, and Q9. In addition, this comparison information R includes a result of determining whether the attachment of the attachment parts 15 and 16 is correct or incorrect.

[0101] As shown in FIG. 19, when the master image Ma is superimposed on the actual image Mb, if the installation of the attachment parts 15, 16 is normal (the presence or absence of the attachment parts 15, 16 and the installation positions of the attachment parts 15, 16 are both correct), the actual image Mb of the attachment parts 15, 16 is painted in a normal identification color (e.g., blue or green) that is effective in indicating that the installation is normal.

[0102] In addition, "OK" which is the result of the judgment as to whether the attachment parts 15, 16 are attached correctly is displayed on the display unit 26. At this time, it is preferable to display "OK" itself in the same normal identification color as the actual image Mb of the painted attachment parts 15, 16. This allows the user to easily visually recognize that the attachment parts 15, 16 are attached correctly.

[0103] In contrast, as shown in FIG. 20, when the master image Ma is superimposed on the actual image Mb, if the attachment of the attachment parts 15, 16 is not normal (either the presence or absence of the attachment parts 15, 16 or the attachment positions of the attachment parts 15, 16 are incorrect, or either one is correct), the actual image Mb of the attachment parts 15, 16 is painted in an abnormality identification color (e.g., red or orange) that is effective in indicating that the attachment is abnormal.

[0104] Also, the correct mounting positions of the mounting parts 15, 16 are surrounded by a box line in the same abnormality identification color as the painted actual images Mb of the mounting parts 15, 16. Furthermore, the details of the mounting abnormality of the mounting parts 15, 16 are displayed on the display unit 27. In this embodiment, if the position of the mounting part 15 is shifted from the correct mounting position Q3 to an incorrect mounting position Q3', a speech bubble saying "Misaligned" is displayed on the display unit 27 around the mounting part 15. If the mounting part 15 is not in the correct mounting position Q5, a speech bubble saying "No bolts" is displayed on the display unit 27 around the mounting position Q5. If the mounting part 16 is not in the correct mounting position Q7, a speech bubble saying "No nuts" is displayed on the display unit 27 around the mounting position Q7.

[0105] In addition, "NG" which is the result of the judgment as to whether the attachment of the attachment parts 15, 16 is correct or incorrect is displayed on the display unit 26. At this time, it is preferable to display "NG" itself in the same abnormality identification color as the actual image Mb of the painted attachment parts 15, 16. This allows the user to easily visually recognize that the attachment of the attachment parts 15, 16 is abnormal.

[0106] The other configurations and methods are the same as those of the first embodiment.

[0107] According to embodiment 3, based on the results of a shape fitting process in which a master image Ma of the workpiece W with the mounting parts 15, 16 attached is superimposed on an actual image Mb, it becomes possible to output information regarding the mounting parts in the workpiece as comparison information R to the monitor 22.

[0108] In addition, the same effects as those of the first embodiment are achieved.

[0109] (Embodiment 4) As shown in FIG. 21, the inspection device 104 of the fourth embodiment is used to inspect a workpiece W. In this embodiment, the workpiece W is an automobile body 1. The inspection device 102 includes a mobile terminal 20 similar to that of the inspection device 101 of the first embodiment. The specific structure of the mobile terminal 20 is similar to that of the first embodiment (see FIGS. 3 and 4). On the other hand, the inspection device 104 differs from the inspection device 101 of the first embodiment in that the reference portion 13 of the base 10 is not required.

[0110] Next, an inspection method according to the fourth embodiment will be described with reference to Fig. 22 to Fig. 26. This inspection method is a method for inspecting an inspection object that is an arbitrary workpiece W among a plurality of types of workpieces W. In this method, the inspection device 104 having the above-mentioned configuration is used.

[0111] As shown in FIG. 22, the processes from step S401 to step S405 of the inspection method of the fourth embodiment are similar to the processes from step S101 and step S102 to step S105 of the first embodiment (see FIG. 5).

[0112] In step S401, three-dimensional master images Ma of each of the multiple types of workpieces W are stored in the memory 50. In step S402, the user selects an inspection target (i.e., any of the multiple types of workpieces W) using the selection unit 23 on the monitor 22.

[0113] 21, in step S403, the user, while holding the mobile terminal 20 with his / her fingers, photographs the rear part of the workpiece W to be inspected from behind with the camera 21 of the mobile terminal 20. At this time, the user adjusts the orientation of the camera 21 to a desired orientation while checking in real time the actual image Mb displayed on the monitor 22. At the rear part of the workpiece W, for example, a lower back panel 1a, which is one of the components, and a reference hole 1b for mounting the lower back panel 1a are formed.

[0114] Step S404 is an image processing step for performing a shape fitting process when a workpiece W to be inspected is selected in step S402. The shape fitting process at this time is a process for superimposing a master image Ma of the workpiece W on the actual image Mb of the workpiece W in a state in which the actual image Mb of the workpiece W is displayed on the monitor 22. This shape fitting process is executed using a known AR program stored in the memory 50, as in the case of the first embodiment.

[0115] Step S405 is a comparison information output step of outputting comparison information R between the actual image Mb and the master image Ma based on the result of the shape fitting process in step S404 to the monitor 22. In this embodiment, the comparison information R is, for example, information about the lower back panel 1a, as shown in Fig. 23. When a positional deviation occurs in the lower back panel 1a (when there is a difference between the actual image Mb and the master image Ma), information 28 indicating the positional deviation direction (rotation direction, falling direction, etc.) of the lower back panel 1a is output as comparison information R to the monitor 22.

[0116] In step S405, as shown in Fig. 24, it is preferable to output information in which the positional deviation of the lower back panel 1a (the difference between the actual image Mb and the master image Ma) is exaggerated to the monitor 22 as the comparison information R. In this embodiment, the positional deviation of the lower back panel 1a is displayed in an exaggerated manner. That is, the position indicated by the solid line in Fig. 24 is the actual position of the lower back panel 1a in a misaligned state, whereas the position indicated by the two-dot chain line in Fig. 24 is an exaggerated position in which the positional deviation of the lower back panel 1a is exaggerated. This makes it easier for the user to check the difference between the actual image Mb and the master image Ma on the monitor 22.

[0117] To exaggerate the positional deviation of the lower back panel 1a, the actual image Mb may be displayed in a color map (for example, like a thermography image, the part of the workpiece W with the greatest amount of positional deviation is displayed in red, the part with the least amount of positional deviation is displayed in blue, and the change in the amount of positional deviation is displayed in a gradation from red to blue). To exaggerate the positional deviation of the lower back panel 1a, the lower back panel 1a may be shifted in each of the first direction X, the second direction Y, and the third direction Z, and the amount of shift of the lower back panel 1a at that time may be set arbitrarily.

[0118] The comparison information R output to the monitor 22 in step S405 may be information about the reference hole 1b. As shown in Fig. 25, when the position of the reference hole 1b is shifted (when there is a difference between the actual image Mb and the master image Ma), information 29 indicating the direction of the position shift of the reference hole 1b is output to the monitor 22 as the comparison information R.

[0119] When the cause of the misalignment of the lower back panel 1a is the misalignment of the reference hole 1b, the cause can be analyzed and information 29 about the cause can be output to the monitor 22. The cause analysis is performed, for example, using AI software including a trained model (not shown). In this embodiment, as shown in FIG. 26, an example is shown in which a message "The position of the reference hole is misaligned, so please check the reference pin in the process of attaching the lower back panel" is output to the monitor 22 as information 29.

[0120] The comparison information R output to the monitor 22 may be information related to the inclination of the vehicle body 1 or its components other than the positional deviation of the lower back panel 1a or the reference hole 1b.

[0121] The other configurations and methods are the same as those of the first embodiment.

[0122] According to embodiment 4, if there is a difference between the actual image Mb and the master image Ma based on the results of the shape fitting process, information that exaggerates the difference is output to the monitor 22 as comparison information R, allowing the user to easily visualize the difference between the actual image Mb and the master image Ma on the monitor 22.

[0123] According to embodiment 4, by outputting information about the cause of the difference between the actual image Mb and the master image Ma to the monitor 22, the user can visually confirm the cause of the difference between the actual image Mb and the master image Ma on the monitor 22.

[0124] In addition, the same effects as those of the first embodiment are achieved.

[0125] The present invention is not limited to the above-described embodiment, and various applications and modifications are possible without departing from the object of the present invention. For example, the following embodiments can be implemented.

[0126] In the above embodiment, an example is given of using a mobile terminal 20 incorporating at least a camera 21 and a monitor 22, but if necessary, a device having the functions of camera 21 and a device having the functions of monitor 22 may be used separately. [Explanation of symbols]

[0127] 10...base, 13...reference unit, 20...mobile terminal, 21...camera, 22...monitor, 40...control unit, 43...image processing unit, 44...comparison information output unit, 50...memory (storage unit), 101, 102, 103, 104...work inspection device (inspection device), Ma...master image, Mb...actual image, R...comparison information, S101, S101a, S102 to S105, S201 to S206, S301 to S305, S401 to S405...work inspection method (inspection method), S101, S201, S301, S401...preparation step, S104, S204, S304, S404...image processing step, S105, S205, S305, S405...comparison information output step, S206: Overlay position change step, W: Work

Claims

1. A work inspection device for inspecting a plurality of types of work, A camera for taking pictures, A monitor capable of displaying a three-dimensional actual image captured by the camera; A control unit that controls the camera and the monitor; A storage unit for storing three-dimensional master images of each of the plurality of types of workpieces; Equipped with The control unit, when an arbitrary workpiece is selected from the plurality of types of workpieces as an inspection object, performs a shape fitting process of reading out the master image of the inspection object from the storage unit and superimposing the master image on the actual image while displaying the actual image of the inspection object photographed by the camera on the monitor; a comparison information output unit that outputs comparison information between the actual image and the master image to the monitor based on a result of the shape fitting process by the image processing unit; A workpiece inspection device having the above structure.

2. A base for placing the workpiece is provided, Both the actual image and the master image are images including the workpiece placed on the pedestal and a reference portion provided on the pedestal, the image processing unit uses the reference portion of the base as a reference for overlaying the master image on the actual image in the shape fitting process; The workpiece inspection device according to claim 1 , wherein the comparison information output unit outputs information relating to an amount of positional deviation of the actual image relative to the master image as the comparison information to the monitor.

3. The workpiece inspection device according to claim 2 , wherein the base has wheels for running.

4. The workpiece inspection device according to claim 1 , wherein the comparison information output unit outputs information relating to an external appearance of the workpiece as the comparison information to the monitor.

5. The workpiece inspection device according to claim 4 , wherein the image processing unit changes a relative position of the superimposition direction of the superimposition surface of the master image with respect to the actual image in response to an operation by a user to change the superimposition position after the shape fitting process.

6. 2. The work inspection device according to claim 1, wherein the work includes an attachment part, and the comparison information output unit outputs information about the attachment part in the work to the monitor as the comparison information.

7. 2. The work inspection device according to claim 1, wherein when there is a difference between the actual image and the master image based on a result of the shape fitting process, the comparison information output unit outputs information that exaggerates the difference as the comparison information to the monitor.

8. 2. The work inspection device according to claim 1, wherein when there is a difference between the actual image and the master image based on the result of the shape fitting process, the comparison information output unit outputs information about a cause of the difference to the monitor as the comparison information.

9. The workpiece inspection device according to any one of claims 1 to 6, wherein the camera, the monitor, and the control unit are incorporated into a single mobile terminal.

10. A work inspection method for inspecting a plurality of types of work, comprising the steps of: A preparation step of storing three-dimensional master images of each of the plurality of types of workpieces in a storage unit; When an arbitrary workpiece is selected from the plurality of types of workpieces as an inspection object, an image processing step is performed in which a shape fitting process is performed in which the master image of the inspection object is read from the storage unit and superimposed on the actual image while an actual image of the inspection object photographed by a camera is displayed on a monitor. a comparison information output step of outputting comparison information between the actual image and the master image to the monitor based on a result of the shape fitting process by the image processing step; The workpiece inspection method includes:

11. A base is used for placing the workpiece, Both the actual image and the master image are images including the workpiece placed on the pedestal and a reference portion provided on the pedestal, In the shape fitting process, the reference portion of the base is used as a reference for overlaying the master image on the actual image; 11. The workpiece inspection method according to claim 10, wherein in the comparison information output step, information relating to an amount of positional deviation of the actual image with respect to the master image is output to the monitor as the comparison information.

12. The workpiece inspection method according to claim 10, wherein in the comparison information output step, information relating to an external appearance of the workpiece is output to the monitor as the comparison information.

13. 13. The workpiece inspection method according to claim 12, further comprising a superposition position changing step of changing a relative position of the superposition direction of the superposition surface of the master image relative to the actual image in response to a user's operation to change the superposition position after the shape fitting process.

14. 11. The workpiece inspection method according to claim 10, wherein the workpiece includes an attachment part, and in the comparison information output step, information about the attachment part in the workpiece is output to the monitor as the comparison information.

15. The workpiece inspection method according to claim 10, wherein in the comparison information output step, if there is a difference between the actual image and the master image based on the result of the shape fitting process, information in which the difference is exaggerated is output to the monitor as the comparison information.

16. The work inspection method according to claim 10, wherein in the comparison information output step, if there is a difference between the actual image and the master image based on the result of the shape fitting process, information regarding the cause of the difference is output to the monitor as the comparison information.

Images