Car body check support device and car body check support method

Abstract

To promptly and efficiently perform a check of a car body while displaying video information relating to the check on the car body.SOLUTION: In a car body check support device 10, a predetermined dot pattern 51b is projected on a calibration board 51 by a projector 30 and an imaging section 21, on which distortion correction is performed, is used to pick up an image of the calibration board 51. An imaging position of a portable terminal 20 on a virtual space is calculated on the basis of a checker pattern 51a constituting the image, and a relative position of the projector 30 with the imaging section 21 defined as a reference on the virtual space is calculated on the basis of the predetermined dot pattern 51b constituting the image. The car body check support device 10 identifies a projection location of an object 60 projecting an AR image by a shape recognition, generates a matching image 62 to which correction processing is applied based on the relative position of the projector 30 from the AR image, and projects the matching image 62.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle body inspection assistance device and a vehicle body inspection assistance method that are capable of quickly and efficiently carrying out inspection while displaying image information related to the inspection of a vehicle body on the vehicle body. [Background technology]

[0002] Conventionally, when checking for peeling of the underside of a vehicle floor, the worker needs to accurately grasp the position of the spot. For this reason, the worker often holds a tablet in one hand while inspecting the vehicle body. There are also cases where worker A holds a tablet and gives instructions to worker B while worker B inspects the vehicle body.

[0003] However, in order to perform such work accurately, the worker needs to have knowledge about the vehicle body, etc., and it is difficult for a worker who does not have sufficient knowledge to perform the work quickly and efficiently. For this reason, it is desirable to have the worker perform the inspection work while displaying image information on the vehicle body using AR (Augmented Reality) technology. In such a case, it is expected to use the AR technology described in Patent Document 1, for example. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020-261741 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned Patent Document 1 determines the position of the human body onto which the video information is projected using a marker that indicates a reference point, and therefore has the problem that it is not possible to identify the position at which the video information is to be displayed if the marker is not used.

[0006] Furthermore, in the technology of Patent Document 1, the camera that captures the image of the body resting on the operating table and the projector that displays the image information on the body are fixed in position, and therefore cannot be applied to situations where the positions of the camera and projector change from time to time, such as a vehicle body.

[0007] The present invention has been made to solve the problems (issues) associated with the above-mentioned conventional technology, and aims to provide a vehicle body inspection assistance device and a vehicle body inspection assistance method that are capable of performing inspection quickly and efficiently while displaying video information related to the inspection of the vehicle body on the vehicle body. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems and achieve the object, the present invention provides a vehicle body inspection assistance device that assists in inspection work of a vehicle body, and includes a projection unit that projects an image onto the vehicle body, an imaging unit that is fixed to a predetermined position of the projection unit and captures an image of the vehicle body, a calculation unit that calculates a relative position of the projection position projected onto the specified medium by the projection unit based on the position in virtual space of the specified medium imaged by the imaging unit, and a correction processing unit that corrects the projection position of the image by the projection unit based on the relative position calculated by the calculation unit.

[0009] In the present invention, in the above-mentioned invention, the medium has a first pattern applied to a predetermined position, and a second pattern is projected by the projection unit onto a position adjacent to the first pattern.

[0010] Furthermore, in the present invention, in the above invention, the calculation unit projects the second pattern onto the medium using the projection unit, captures an image including the first pattern and the second pattern of the medium using the imaging unit, calculates an imaging position of the imaging unit in the virtual space based on the first pattern on the image, and calculates a relative position of the projection unit in the virtual space based on the second pattern on the image and the imaging position of the imaging unit in the virtual space.

[0011] In addition, in the present invention, in the above invention, the correction processing unit performs correction processing of a geometric transformation of the image projected by the projection unit based on the relative position calculated by the calculation unit, and projects the image after the correction processing by the projection unit.

[0012] In the present invention, in the above-mentioned invention, the image forming apparatus further includes a distortion correction unit that captures an image of the medium for distortion correction having a third pattern thereon by the imaging unit and corrects distortion of the imaging unit.

[0013] In the present invention, in the above-mentioned invention, a mobile terminal having the imaging unit, the calculation unit, and the correction processing unit is fixed to an upper portion of a projector having the projection unit.

[0014] The present invention also provides a vehicle body inspection support method executed by a vehicle body inspection support device that supports inspection work of the vehicle body, the vehicle body having a projection unit that projects an image onto the vehicle body and an imaging unit that is fixed at a predetermined position on the projection unit and captures an image of the vehicle body, the method including a calculation step of calculating a relative position of the projection position projected onto the predetermined medium by the projection unit, based on a position in a virtual space of the predetermined medium imaged by the imaging unit, and a correction processing step of correcting the projection position of the image by the projection unit based on the relative position calculated by the calculation step. Effect of the Invention

[0015] According to the present invention, inspection can be performed quickly and efficiently while displaying image information relating to inspection of the vehicle body on the vehicle body. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an overview of a vehicle body inspection support device according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing a configuration of the mobile terminal shown in FIG. [Diagram 3] FIG. 3 is a diagram showing an example of the shape recognition data and the AR image data shown in FIG. [Figure 4]FIG. 4 is a diagram showing an overview of correction of the imaging unit parameters. [Diagram 5] FIG. 5 is an explanatory diagram for explaining correction of the imaging unit parameters. [Figure 6] FIG. 6 is a diagram showing an example of an image before and after the correction of the imaging unit parameters. [Figure 7] FIG. 7 is a diagram showing an outline of calculation of the imaging position of the imaging unit and the relative position of the projector. [Figure 8] FIG. 8 is a diagram showing an example of calculation of the relative positions of the projectors. [Figure 9] FIG. 9 is a diagram showing an overview of shape recognition and projection of extended realization of the vehicle body inspection assistance device. [Figure 10] FIG. 10 is a flowchart showing a processing procedure of the vehicle body inspection support device shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of a vehicle body inspection support device and a vehicle body inspection support method according to the present invention will be described in detail with reference to the accompanying drawings.

[0018] An overview of a vehicle body inspection support device 10 according to this embodiment will be described. Fig. 1 is a diagram showing an overview of the vehicle body inspection support device 10 according to the embodiment. As shown in Fig. 1, the vehicle body inspection support device 10 has a mobile terminal 20 and a projector 30. The mobile terminal 20 is fixed to an upper portion of the projector 30 by a fixing jig (not shown). In addition, the mobile terminal 20 and the projector 30 are connected by a cable (not shown).

[0019] As shown in FIG. 1(a), a calibration board 51 (corresponding to a medium in the claims) is prepared to calibrate the vehicle body inspection support device 10. A checkered pattern 51a (corresponding to the first pattern in the claims) for calculating the relative position of the mobile terminal 20 is provided on the calibration board 51. Note that the size (horizontal and vertical dimensions) of the lattice pattern of the predetermined checkered pattern 51a used on the calibration board 51 is assumed to be known in advance. Also, the calibration board 51 is provided with an area onto which the predetermined checkered pattern 51a and a predetermined dot pattern 51b (corresponding to the second pattern in the claims) are projected from the projector 30.

[0020] 1(b), the vehicle body inspection support device 10 projects a predetermined dot pattern 51b from a projection unit provided in the projector 30 onto a position adjacent to the checker pattern 51a of the calibration board 51 on which the predetermined checker pattern 51a is applied (S1). Then, the vehicle body inspection support device 10 captures an image of the checker pattern 51a and the predetermined dot pattern 51b of the calibration board 51 by the imaging unit 21 (S2).

[0021] Thereafter, the vehicle body inspection support device 10 calculates the imaging positions in the virtual space of the calibration board 51 and the imaging unit 21 provided in the mobile terminal 20 based on a predetermined checkered pattern 51a of the captured image (S3). Thereafter, the vehicle body inspection support device 10 calculates the relative position of the projector 30 in the virtual space with respect to the imaging unit 21 based on a predetermined dot pattern 51b of the captured image (S4).

[0022] Next, as shown in Fig. 1(c), the vehicle body inspection support device 10 captures an image of the object 60 onto which an augmented reality image (hereinafter, referred to as an "AR image") is to be projected by the imaging unit 21 included in the mobile terminal 20 (S5). Then, the vehicle body inspection support device 10 identifies a projection location onto the object 60 based on preset shape recognition data (S6).

[0023] Then, the vehicle body inspection support device 10 performs a correction process on the AR image to generate a matching image 62 based on the imaging position of the imaging unit 21 provided in the mobile terminal 20 and the relative position of the projector 30 (S7). Then, the vehicle body inspection support device 10 projects the matching image 62 onto the object (S8).

[0024] The vehicle body inspection support device 10 constantly performs the steps of (1) capturing an image of the object 60 using the imaging unit 21, (2) identifying the projection location on the object 60 based on shape recognition data, (3) correcting the AR image based on the relative positions of the mobile terminal 20 and the projector 30, and (4) projecting the matching image 62. Therefore, even if the position of the vehicle body inspection support device 10 is changed, the matching image 62 can be projected at the projection position specified by the shape recognition data.

[0025] <Configuration of the mobile terminal 20> Next, a description will be given of the configuration of the mobile terminal 20 shown in Fig. 1. Fig. 2 is a functional block diagram showing the configuration of the mobile terminal 20 shown in Fig. 1. As shown in Fig. 2, the mobile terminal 20 has an imaging unit 21, an operation display unit 22, an image transmission I / F unit 23, a storage unit 24, and a control unit 25. The imaging unit 21 is a camera that captures images of calibration boards 50, 51 and objects.

[0026] The operation display unit 22 is an input device and display device that performs display using a device such as a liquid crystal panel, a display, or an organic EL (Electro Luminescence) device, and receives operation input by a pressure-sensitive method, etc. The image transmission I / F unit 23 is an interface that transmits image data to the projector 30.

[0027] The storage unit 24 is a storage device consisting of a hard disk drive or a non-volatile memory, and stores shape recognition data 24a, AR image data 24b (corresponding to an image in the claims), imaging unit correction data 24c, mobile terminal imaging position data 24d, and projector relative position data 24e. The shape recognition data 24a is data of only the outline of the AR image created based on the AR image data 24b to be projected. The AR image data 24b is data of the AR image to be projected onto an object created in advance using a three-dimensional CAD or the like.

[0028] The imaging unit correction data 24c is data of internal parameters obtained by imaging a predetermined calibration pattern 50a with the imaging unit 21 and adjusting the internal parameters of the imaging unit 21 so as to correct distortion of the captured image. The mobile terminal imaging position data 24d is data obtained by imaging a predetermined checkered pattern 51a and calculating the imaging position of the imaging unit 21 in the virtual space calculated from the captured image. The projector relative position data 24e is data obtained by calculating the relative position of the projector 30 with respect to the imaging unit 21 in the virtual space based on images of the predetermined checkered pattern 51a and a predetermined dot pattern 51b projected by the projector 30.

[0029] The control unit 25 is a control unit that controls the entire mobile terminal 20, and has an image acquisition unit 25a, an imaging unit parameter correction processing unit 25b, a mobile terminal imaging position calculation processing unit 25c, a projector relative position calculation processing unit 25d (corresponding to a calculation unit in the claims), and an ARPM (Augmented Reality Projection Mapping) processing unit 26. In practice, by loading these programs into a CPU and executing them, the image acquisition unit 25a, the imaging unit parameter correction processing unit 25b, the mobile terminal imaging position calculation processing unit 25c, the projector relative position calculation processing unit 25d, and the ARPM processing unit 26 are caused to execute processes corresponding thereto, respectively.

[0030] The image acquisition unit 25a performs a process of acquiring an image by controlling the imaging unit 21. The imaging unit parameter correction processing unit 25b is a processing unit that captures the calibration pattern 50a by the imaging unit 21, calculates imaging unit correction parameters for correcting distortion of the imaging unit 21 based on the captured image, and stores the image in the storage unit 24 as imaging unit correction data 24c, and after calculating the imaging unit correction parameters, reads out the imaging unit correction data 24c stored in the storage unit 24 from the image captured by the imaging unit 21, and constantly performs distortion correction.

[0031] The mobile terminal imaging position calculation processing unit 25c performs processing for capturing an image of a predetermined checkered pattern 51a by the imaging unit 21, and calculating the imaging position of the imaging unit 21 in the virtual space of the mobile terminal 20 and the calibration board 51, based on the captured image and the dimensions of the lattice pattern of the predetermined checkered pattern 51a that has been set in advance. The calculated imaging position of the imaging unit 21 is stored in the storage unit 24 as mobile terminal imaging position data 24d.

[0032] The projector relative position calculation processing unit 25d projects a predetermined dot pattern 51b from the projection unit of the projector 30 onto the calibration board 51. Then, the imaging unit 21 captures an image of the predetermined checker pattern 51a and the predetermined dot pattern 51b on the calibration board 51, and performs a process of calculating the relative position of the projector 30 based on the imaging unit 21 provided in the mobile terminal 20, based on the captured image. The calculated relative position of the projector 30 is stored in the storage unit 24 as projector relative position data 24e.

[0033] The ARPM processing unit 26 includes a shape recognition processing unit 26a, a matching image generating unit 26b (corresponding to a correction processing unit in the claims), and a display processing unit 26c. The shape recognition processing unit 26a reads out shape recognition data 24a stored in advance in the storage unit 24, compares an image of an object captured by the imaging unit 21 with the shape recognition data 24a, and identifies a location to project the AR image data 24b. The shape recognition is performed by extracting local features of the captured image and comparing them with the shape recognition data 24a. As the features, SHOT (Signature of Histograms of Orientations) features, PFH (Point Feature Histogram) features, FPFH (Fast Point Feature Histogram) features, etc. can be used.

[0034] The matching image generation unit 26b is a processing unit that performs processing to read out the mobile terminal imaging position data 24d and the projector relative position data 24e, and apply geometric transformation correction such as enlargement / reduction and rotation to the AR image data 24b so that the AR image data 24b stored in advance in the storage unit 24 can be projected onto the projection location of the object recognized by the shape recognition processing unit 26a, thereby generating a matching image 62. The display processing unit 26c is a processing unit that sends the generated matching image 62 to the projector 30 via the image transmission I / F unit 23, and projects the matching image 62 onto the object.

[0035] <Regarding the shape recognition data 24a and the AR image data 24b> Next, the shape recognition data 24a and the AR image data 24b will be described. Fig. 3 is a diagram showing an example of the shape recognition data 24a and the AR image data 24b shown in Fig. 2. As shown in Fig. 3(a), the shape recognition data 24a is data obtained by extracting the outline of the AR image data 24b to be projected. For example, when a three-dimensional model is used for the AR image, the three-dimensional model can be created using a three-dimensional CAD, and contour data can be created from the three-dimensional model using a contour extraction technique.

[0036] As shown in Fig. 3(b), the AR image data 24b is image data for projecting information required for inspection onto an object to be inspected, etc. For example, when a structure is manufactured by combining several metal plates, the AR image data 24b may be image data for displaying welding positions, bolt positions on mating surfaces, etc.

[0037] <Calibration of the imaging unit 21 of the mobile terminal 20> Next, calibration for correcting distortion of the imaging unit 21 of the mobile terminal 20 will be described. Fig. 4 is a diagram showing an overview of correction of imaging unit parameters. As shown in Fig. 4, the vehicle body inspection support device 10 captures an image of the calibration pattern 50a (corresponding to the third pattern in the claims) using the imaging unit 21 provided in the mobile terminal 20 (S11). Then, the vehicle body inspection support device 10 corrects distortion of the imaging unit 21 based on the image (S12).

[0038] Next, the calibration of the imaging unit 21 will be described in detail. FIG. 5 is an explanatory diagram for explaining the correction of the imaging unit parameters. As shown in FIG. 5(a), the vehicle body inspection support device 10 uses the imaging unit 21 provided in the mobile terminal 20 to capture images of a calibration board 50 (corresponding to a medium for distortion correction in claims) multiple times. Here, when capturing an image, the position of the calibration board 50 (the distance between the imaging unit 21 and the calibration board 50 and the orientation of the calibration board 50) is changed for each capture. Note that a predetermined calibration pattern 50a is attached to the calibration board 50 (see FIG. 5(b)).

[0039] If the captured image contains distortion (see FIG. 6(a)), the mobile terminal 20 calculates internal parameters for performing distortion correction by the imaging unit parameter correction processing unit 25b so that the captured image is free of distortion (see FIG. 6(b)). Specifically, the imaging unit 21 captures images of the calibration board 50 multiple times, calculates internal parameters for performing distortion correction for each image, and stores the average value of the calculated internal parameters in the storage unit 24 as imaging unit correction data 24c.

[0040] <Calculation of the imaging position of the imaging unit 21 and the relative position of the projector 30> Next, the calculation of the imaging position of the imaging unit 21 and the relative position of the projector 30 will be described. FIG. 7 is a diagram showing an overview of the calculation of the imaging position of the imaging unit 21 and the relative position of the projector 30. As shown in FIG. 7(a), the mobile terminal 20 captures an image of the calibration board 51 multiple times using the imaging unit 21 provided in the mobile terminal 20. Here, the predetermined checkered pattern 51a is prepared such that the dimensions (x) of the grid of the checkered pattern 51a are the same vertically and horizontally (see FIG. 6(b)), and the dimensions of the grid are measured in advance. Then, the dimensions are set in advance in the mobile terminal 20. In addition, a predetermined dot pattern 51b is projected on the calibration board 51 at a position near the predetermined checkered pattern 51a by the projection unit of the projector 30 (see FIG. 7(b)).

[0041] Then, the vehicle body inspection support device 10 calculates the imaging position of the mobile terminal 20 and the calibration board 51 in the virtual space based on the predetermined checker pattern 51a of the image of the calibration board 51. Specifically, the imaging position is calculated based on the predetermined checker pattern 51a attached to the calibration board 51, whose dimensions are known, and a plurality of pixels forming the checker pattern 51a of the captured image. Since the parameters (focal length) of the camera of the imaging unit 21 and the dimensions of the grid of the predetermined checker pattern 51a are known, the distance to the calibration board 51 can be calculated when the number of pixels forming the grid of the checker pattern 51a is known. In addition, the relative angle can also be calculated based on the number of pixels on the four sides of the number of pixels forming the grid.

[0042] Thereafter, the vehicle body inspection assistance device 10 calculates a relative position based on the imaging unit 21 of the projector 30, based on the predetermined dot pattern 51b of the captured image of the calibration board 51. Specifically, as shown in Fig. 8, an error e occurs between the predetermined dot pattern 51b actually projected by the projector 30 and the predetermined dot pattern that should be projected based on the predetermined checker pattern 51a.

[0043] Based on this error, the relative position of projector 30 in the virtual space with respect to imaging unit 21 is calculated. In reality, by adjusting the image data projected by projector 30 based on the calculated relative position of projector 30 with respect to imaging unit 21, a predetermined dot pattern 51b that is actually projected by projector 30 is projected at a position where a predetermined checkered pattern 51a should be projected with respect to the predetermined checkered pattern 51a.

[0044] Here, the vehicle body inspection support device 10 captures a plurality of images with the position of the calibration board 51 changed using the imaging unit 21 provided in the mobile terminal 20. Then, based on each image, the vehicle body inspection support device 10 calculates the relative position of the projector 30 with respect to the imaging unit 21 as a reference, calculates the average value of the calculated relative positions, and stores the average value in the storage unit 24 of the mobile terminal 20 as projector relative position data 24e.

[0045] <Projection of Extended Realization> Next, the projection of an AR image by the vehicle body inspection support device 10 will be described. Fig. 9 is a diagram showing an overview of shape recognition and projection by the vehicle body inspection support device 10 through augmented realization. As shown in Fig. 9(a), the vehicle body inspection support device 10 captures an image of an object 60 onto which an AR image is to be projected using an imaging unit 21 provided in a mobile terminal 20. Then, the vehicle body inspection support device 10 identifies a projection location 61 onto which the AR image of the object 60 is to be projected by shape recognition.

[0046] 9(b), the vehicle body inspection support device 10 calculates the imaging position of the imaging unit 21 of the mobile terminal 20 based on the data obtained by performing shape recognition. After that, the vehicle body inspection support device 10 reads out the AR image data 24b and the projector relative position data 24e, and generates a matching image 62 based on the imaging position of the imaging unit 21 and the projector relative position data 24e. Specifically, the AR image data 24b is subjected to geometric transformation such as enlargement / reduction and rotation based on the imaging position of the imaging unit 21 and the projector relative position data 24e to perform correction processing, and generates the matching image 62. Then, the vehicle body inspection support device 10 projects the generated matching image 62 to the projection location 61 by the projector 30.

[0047] <Processing procedure of the vehicle body inspection support device 10> Next, a processing procedure of the vehicle body inspection support device 10 will be described. Fig. 10 is a flowchart showing the processing procedure of the vehicle body inspection support device 10 shown in Fig. 1. As shown in Fig. 10, the vehicle body inspection support device 10 captures an image of the calibration pattern 50a by the imaging unit 21 provided in the mobile terminal 20 (step S101). Thereafter, the vehicle body inspection support device 10 calculates distortion correction parameters for the imaging unit 21 based on the captured calibration pattern 50a (step S102) and corrects the distortion of the imaging unit 21 (step S103).

[0048] Then, the vehicle body inspection support device 10 projects a predetermined dot pattern 51b onto the calibration board 51 using the projector 30 (step S104). After that, the vehicle body inspection support device 10 captures an image of the calibration board 51 using the imaging unit 21 provided in the mobile terminal 20 (step 105).

[0049] Then, the vehicle body inspection support device 10 calculates the imaging position of the imaging unit 21 provided in the mobile terminal 20 in the virtual space based on the predetermined checker pattern 51a constituting the image (step S106). After that, the vehicle body inspection support device 10 calculates the relative position of the projector 30 based on the imaging unit 21 provided in the mobile terminal 20 in the virtual space as a reference based on the predetermined dot pattern 51b projected by the projector 30 constituting the image (step S107).

[0050] Then, the vehicle body inspection support device 10 captures an image of the object 60 onto which the AR image is to be projected, and identifies the projection location of the object onto which the AR image is to be projected based on the shape recognition data 24a (step S108). After that, the vehicle body inspection support device 10 performs a correction process on the AR image data 24b based on the relative position of the projector 30 with respect to the mobile terminal 20, and generates a matching image 62 (step S109).

[0051] Then, the vehicle body inspection support device 10 projects the generated matching image 62 from the projector 30 onto the object 60 (step S110). After that, the vehicle body inspection support device 10 determines whether the projection has ended (step S111), and if the projection has not ended (step S111: No), the process proceeds to step S108. If the projection has ended (step S111: Yes), the process ends.

[0052] As described above, in this embodiment, the vehicle body inspection support device 10 captures a predetermined calibration pattern 50a using the imaging unit 21 provided in the mobile terminal 20, and performs distortion correction of the imaging unit 21 based on the calibration pattern 50a. Then, the vehicle body inspection support device 10 projects a predetermined dot pattern 51b onto another prepared calibration board 51 using the projector 30, captures an image of the calibration board 51 using the imaging unit 21, and calculates the imaging position of the mobile terminal 20 in the virtual space based on the checker pattern 51a constituting the image. After that, the vehicle body inspection support device 10 calculates the relative position of the projector 30 based on the imaging unit 21 provided in the mobile terminal 20 in the virtual space based on the predetermined dot pattern 51b constituting the image. Furthermore, the vehicle body inspection support device 10 is configured to identify the projection location of the object 60 onto which the AR image is projected by shape recognition, generate a matching image 62 from the AR image that has been corrected based on the relative position of the projector 30, and project the matching image 62. Therefore, even if the position of the vehicle body inspection support device 10 is changed, the AR image can be projected at a specified location on the object, and inspection can be performed quickly and efficiently while displaying video information related to the vehicle body inspection on the vehicle body.

[0053] In the above embodiment, the vehicle body inspection support device 10 is described as being composed of the mobile terminal 20 and the projector 30, but the present invention is not limited to this, and the device may be one in which the functions of the mobile terminal 20 and the projector 30 are integrated.

[0054] Further, in the above embodiment, the vehicle body inspection support device 10 is described as using the mobile terminal 20, but the present invention is not limited to this, and a tablet, a smartphone, a PC, or the like may be used.

[0055] In the above embodiment, the vehicle body inspection support device 10 has been described as specifying the projection location of the matching image 62 using shape recognition, but the projection location can also be specified using a technique other than shape recognition. For example, the projection location of the AR image can also be specified by attaching an AR marker, a two-dimensional barcode, a photograph, or the like.

[0056] In the above embodiment, the present invention is described as being used for vehicle body inspection, but it can also be applied to the fields of home appliances, medicine, and entertainment.

[0057] The configurations illustrated in the above embodiments are merely functional schematics, and are not necessarily physically configured as illustrated. In other words, the form of distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. [Industrial Applicability]

[0058] The vehicle body inspection assistance device and vehicle body inspection assistance method according to the present invention are suitable for cases in which inspection is carried out quickly and efficiently while image information relating to the inspection of the vehicle body is displayed on the vehicle body. [Explanation of symbols]

[0059] 10 Vehicle inspection support device 20 Mobile Devices 21 Imaging unit 22 Operation display section 23 Image transmission I / F section 24 Memory section 24a Shape recognition data 24b AR image data 24c Imaging unit correction data 24d Mobile device imaging position data 24e Projector relative position data 25 Control Unit 25a Image acquisition section 25b Imaging unit parameter correction processing unit 25c Mobile terminal imaging position calculation processing unit 25d Projector relative position calculation processing unit 26 ARPM processing section 26a Shape recognition processing section 26b Matching image generation unit 26c Display processing section 30 Projector 50 Calibration Board 50a Calibration Pattern 51 Calibration Board 51a Checkered pattern 51b Dot Pattern 60 Object 61 Projection Location 62 Matching Images

Claims

1. A vehicle body inspection assistance device that assists in inspection work of a vehicle body, A projection unit that projects an image onto the vehicle body; an imaging unit that is fixed to a predetermined position of the projection unit and captures an image of the vehicle body; a calculation unit that calculates a relative position of a projection position projected onto a predetermined medium by the projection unit, based on a position in a virtual space of the predetermined medium imaged by the imaging unit; a correction processing unit that corrects a projection position of the image by the projection unit based on the relative position calculated by the calculation unit; A vehicle body inspection assistance device comprising:

2. The medium is 2. The vehicle body inspection support device according to claim 1, wherein the medium has a first pattern attached to a predetermined position, and a second pattern is projected by the projection unit onto a position adjacent to the first pattern.

3. The calculation unit is projecting the second pattern onto the medium by the projection unit; capturing an image including the first pattern and the second pattern of the medium by the imaging unit; calculating an imaging position of the imaging unit in the virtual space based on the first pattern on the image; Calculating a relative position of the projection unit in the virtual space based on the second pattern on the image and an imaging position of the imaging unit in the virtual space.

3. The vehicle body inspection support device according to claim 2.

4. The correction processing unit: The vehicle body inspection support device according to claim 1, characterized in that a geometric transformation correction process is performed on the image projected by the projection unit based on the relative position calculated by the calculation unit, and the image after the correction process is projected by the projection unit.

5. 2. The vehicle body inspection support device according to claim 1, further comprising a distortion correction unit that uses the imaging unit to capture an image of a medium for distortion correction having a third pattern thereon and corrects distortion of the imaging unit.

6. 6. The vehicle body inspection support device according to claim 1, wherein a mobile terminal having the imaging unit, the calculation unit, and the correction processing unit is fixed to an upper portion of a projector having the projection unit.

7. A vehicle body inspection support method executed by a vehicle body inspection support device that supports an inspection work of the vehicle body, the vehicle body inspection support device having a projection unit that projects an image onto a vehicle body and an imaging unit that is fixed at a predetermined position of the projection unit and captures an image of the vehicle body, a calculation step of calculating a relative position of a projection position projected onto a predetermined medium by the projection unit, based on a position in a virtual space of the predetermined medium imaged by the imaging unit; a correction processing step of correcting a projection position of the image by the projection unit based on the relative position calculated by the calculation step; A vehicle body inspection support method comprising:

Images