Camera calibration confirmation method, calibration confirmation program, and calibration confirmation device

Abstract

To provide a technique for suppressing false recognition of a calibration result in verifying calibration of a camera.SOLUTION: A calibration verification method for verifying a calibration result of a camera mounted on a vehicle includes: an image acquiring step S103 of acquiring a photographed image obtained by photographing, with the camera, a state where a verification target with one or more marks put at predetermined heights is arranged around the vehicle; an overlay position calculation step S105 of calculating a display position of a required accuracy range representing a required accuracy of the camera corresponding to a height of a mark, based on positional information of the verification target and the required accuracy of the camera; overlay image creation steps S106-S108 of creating an overlay image indicating the display position of the required accuracy range, and causing a display device to display a verification image created by overlaying the overlay image on the photographed image; and a verification step S111 of determining that a calibration result of the camera satisfies the required accuracy of the camera when the mark of the verification target is displayed within the required accuracy range in the verification image.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] This invention relates to a technology for calibration confirmation that confirms the calibration results of a camera mounted on a vehicle. [Background technology]

[0002] Surroundings of a vehicle are monitored and driving assistance is performed based on images captured by a camera mounted on the vehicle. In order to properly perform surroundings monitoring and driving assistance, the correlation between the coordinate system of the image captured by the camera and the coordinate system of the image used for image processing must be matched with high precision, and the camera is calibrated to ensure this precision. After calibrating the camera, the calibration results can be checked to achieve proper camera calibration.

[0003] In Patent Document 1, checkered rectangular calibration markers are installed on the left and right front sides of a vehicle, and images are taken with an on-board camera to calibrate the on-board camera. The coordinates of a marker frame indicating the theoretical position of the calibration marker are calculated using the calibrated camera parameters, and the marker frame is superimposed on the captured image of the calibration marker displayed on the display. If the camera calibration is appropriate, the calibration marker is displayed within the marker frame. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4690476 Summary of the Invention [Problem to be solved by the invention]

[0005] In the method of Patent Document 1, if the position of the calibration marker is misaligned, the calibration marker may be displayed within the marker frame even if the camera calibration is appropriate, or may be displayed within the marker frame even if the camera calibration is inappropriate. In this case, the proofreader may not notice the misalignment of the calibration marker and may misinterpret the calibration results.

[0006] In view of the above, an object of the present invention is to provide a technique that can suppress erroneous recognition of calibration results when checking the calibration of a camera. [Means for solving the problem]

[0007] The present invention provides a calibration confirmation method for confirming the calibration results of a camera mounted on a vehicle, the calibration confirmation method including: an image acquisition step of acquiring, with the camera, a captured image of a state in which a confirmation target, each having one or more markers at a predetermined height, is arranged around the vehicle; a superimposition position calculation step of calculating, based on position information of the confirmation target and the required accuracy of the camera, a display position of a required accuracy range that indicates the required accuracy of the camera corresponding to the height of the markers; a superimposition image creation step of creating a superimposition image that indicates the display position of the required accuracy range and displaying, on a display device, a confirmation image in which the superimposition image is superimposed on the captured image; and a confirmation step of determining that the calibration result of the camera satisfies the required accuracy of the camera if the markers of the confirmation target are displayed within the required accuracy range in the confirmation image.

[0008] According to the above calibration confirmation method, a confirmation target having one or more markers at a predetermined height and arranged around a vehicle is photographed by a camera, and a superimposed image with a required accuracy range is superimposed on the photographed image and displayed on a display device as a confirmation image. According to this confirmation image, if the markers on the confirmation target are displayed within the required accuracy range, it can be determined that the camera calibration result satisfies the required accuracy of the camera. Therefore, an operator can easily confirm the calibration result by visually checking the confirmation image displayed on the display device. Furthermore, the display position of the required accuracy range is calculated based on the position information of the confirmation target, which prevents the required accuracy range from being displayed in an inappropriate position due to a misalignment of the confirmation target. This prevents erroneous recognition of the calibration results when confirming the camera calibration.

[0009] The present invention also provides a calibration confirmation program and a calibration confirmation device that display on a display device a confirmation image for confirming the calibration result of a camera mounted on a vehicle.

[0010] The calibration confirmation program causes a computer to execute an image acquisition step of acquiring a captured image taken by the camera of a state in which a confirmation target with one or more markers attached at a predetermined height is placed around the vehicle; a superimposition position calculation step of calculating a display position of a required accuracy range that represents the required accuracy of the camera corresponding to the height of the markers based on position information of the confirmation target and the required accuracy of the camera; and a superimposition image creation step of creating a superimposition image that indicates the display position of the required accuracy range and displaying a confirmation image in which the superimposition image is superimposed on the captured image on a display device.

[0011] The calibration confirmation device comprises an image acquisition unit that acquires a captured image taken by the camera of a state in which a confirmation target, with one or more markers attached at a predetermined height, is arranged around the vehicle; a superimposition position calculation unit that calculates the display position of a required accuracy range that represents the required accuracy of the camera corresponding to the height of the markers based on position information of the confirmation target and the required accuracy of the camera; and a superimposition image creation unit that creates a superimposition image that indicates the display position of the required accuracy range and displays a confirmation image on a display device in which the superimposition image is superimposed on the captured image.

[0012] By displaying a confirmation image on a display device using the above-mentioned calibration confirmation program and calibration confirmation device, an operator can easily determine whether the mark on the confirmation target is displayed within the required accuracy range, and therefore can easily determine whether the camera calibration result satisfies the required accuracy of the camera. Furthermore, because the display position of the required accuracy range is calculated based on the position information of the confirmation target, misinterpretation of the calibration result can be suppressed when confirming the camera calibration. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing the configuration of a calibration confirmation device according to a first embodiment. [Figure 2] 1 is a diagram showing a camera sensor mounted on a vehicle and verification targets positioned around the vehicle. [Figure 3] FIG. 10 is a diagram illustrating a required accuracy range. [Figure 4] 10A and 10B are diagrams illustrating the creation of a superimposed image in which a photographed image and a required accuracy range are superimposed. [Figure 5] FIG. 2 is a diagram showing a display device on which a confirmation image is displayed according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing a display device on which a confirmation image is displayed according to the first embodiment. [Figure 7] 3 is a flowchart of a method for checking calibration of a camera sensor according to the first embodiment. [Figure 8] 10 is a flowchart of a camera sensor calibration confirmation method according to a second embodiment. [Figure 9] 10A and 10B are diagrams for simply explaining the arrangement of confirmation targets and the procedure for creating a superimposed image according to the second embodiment. [Figure 10] 10 is a flowchart of a camera calibration confirmation method according to a third embodiment. [Figure 11] 10A and 10B are diagrams for simply explaining the arrangement of confirmation targets and the procedure for creating a superimposed image according to the third embodiment. [Figure 12] FIG. 10 is a diagram showing the configuration of a calibration confirmation device according to a fourth embodiment. [Figure 13] 10 is a flowchart of a camera sensor calibration confirmation method according to a fourth embodiment. [Figure 14] FIG. 10 is a diagram showing a display device on which a confirmation image according to a fourth embodiment is displayed. [Figure 15] FIG. 13 is a diagram showing a required accuracy range of a confirmation image according to the fifth embodiment. [Figure 16] 13A and 13B are diagrams for simply explaining the arrangement of confirmation targets and the procedure for creating a superimposed image according to the fifth embodiment. [Figure 17] FIG. 13 is a diagram showing a display device on which a confirmation image is displayed according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] (First embodiment) 1 shows a vehicle 10 equipped with an ECU 20 that functions as a calibration verification device according to the first embodiment. The vehicle 10 includes a radar sensor 12, a camera sensor 13, the ECU 20, and a display device 14 as an on-board system.

[0015] The radar sensor 12 is, for example, a known millimeter-wave radar that transmits high-frequency signals in the millimeter wave band. The vehicle 10 may be equipped with only one radar sensor 12 or multiple radar sensors 12. The radar sensor 12 is installed, for example, at the front or rear end of the vehicle 10. The radar sensor 12 defines an area within a predetermined detection angle as a detection range within which objects can be detected, and detects the position of an object within the detection range. Specifically, the radar sensor 12 transmits a search wave at a predetermined interval and receives reflected waves using multiple antennas. The distance to the object can be calculated based on the transmission time of the search wave and the reception time of the reflected wave. Furthermore, the relative velocity can be calculated based on the frequency of the reflected wave reflected by the object, which is changed by the Doppler effect. Furthermore, the direction of the object can be calculated based on the phase difference between the reflected waves received by the multiple antennas. Note that if the position and direction of the object can be calculated, the object's relative position with respect to the vehicle 10 can be identified.

[0016] The camera sensor 13 may be a monocular camera such as a CCD camera, a CMOS image sensor, or a near-infrared camera, or may be a stereo camera. The camera sensor 13 extracts feature points indicative of the presence of an object in the captured image. Specifically, edge points are extracted based on the luminance information of the captured image, and a Hough transform is performed on the extracted edge points. In the Hough transform, for example, points on a line where multiple edge points are continuously lined up, or points where lines intersect at right angles, are extracted as feature points. The camera sensor 13 sequentially outputs the captured images as sensing information.

[0017] Only one camera sensor 13 may be installed in the vehicle 10, or multiple camera sensors 13 may be installed. The camera sensor 13 is attached, for example, at a predetermined height in the center of the vehicle width direction of the vehicle 10, and captures an area extending in a predetermined angular range toward the front, rear, or sides of the vehicle from a bird's-eye view. For example, camera sensors 13F, 13L, 13R, and 13B shown in FIG. 2 can be installed at the front, left, right, and rear of the vehicle 10.

[0018] After calibrating the camera sensor 13, in order to confirm the calibration results, confirmation targets 2 are placed around the vehicle 10 as appropriate, and the camera sensor 13 captures an image of the confirmation targets 2. The confirmation targets 2 are placed around the vehicle 10 as appropriate, depending on the installation position and orientation of the camera sensor 13. The position of the confirmation targets 2 need only be within the imaging range of the camera sensor 13. When calibrating the camera sensors 13F, 13L, 13R, and 13B, it is preferable to place the confirmation targets 2 in the direction of the camera sensor 13, for example, at the front, left, right, and rear of the vehicle 10, as in confirmation targets 2F, 2L, 2R, and 2B.

[0019] The display device 14 is a display device that includes at least a display screen that can display images captured by the camera sensor 13, and can be, for example, a display screen (display) of a car navigation system installed in the passenger compartment of the vehicle 10. The display device 14 may further include an interface that allows an operator to view the display screen and perform operation input.

[0020] 1, the ECU 20 includes a position information acquisition unit 21, a required accuracy acquisition unit 22, a camera parameter acquisition unit 23, a camera image acquisition unit 24, a superimposition position calculation unit 25, and a superimposition image creation unit 26. The ECU 20 is a computer (more specifically, a microcomputer) equipped with a CPU, ROM, RAM, an input / output interface, etc. The CPU realizes these functions by executing programs stored in the ROM and RAM. As a result, the ECU 20 functions as a calibration confirmation device that displays, on the display device 14, a confirmation image for confirming the calibration result of the camera sensor 13.

[0021] The position information acquisition unit 21 acquires position information of the confirmation target 2. The position information of the confirmation target 2 may be measured by a predetermined measuring device, received via a communication device, or input by an operator. FIG. 1 illustrates an example in which the position information acquisition unit 21 acquires the position information of the confirmation target 2 from an on-board radar sensor 12 or a measuring instrument 1 installed outside the vehicle. The radar sensor 12 is an example of an on-board sensor that can acquire the position information of the confirmation target 2 other than the camera sensor 13. As such an on-board sensor, the vehicle 10 may be equipped with, for example, a sonar sensor or a sensor that transmits search waves, such as a LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). The measuring instrument 1 is a measuring device such as a laser rangefinder, or a measuring instrument such as a tape measure or ruler. The position information of the confirmation target 2 measured by the measuring instrument 1 may be input to the ECU 20 via a communication means or via the operator. The position information of the confirmation target 2 acquired by the position information acquisition unit 21 is output to the superposition position calculation unit 25.

[0022] The required accuracy acquisition unit 22 acquires the required accuracy, which is the accuracy required of the camera sensor 13. The required accuracy is set according to the distance from the camera sensor 13, and is set, for example, as an allowable error range in all directions depending on the distance. The required accuracy is set higher the closer to the camera sensor 13, for example, 1.3 m in all directions when the distance from the camera sensor 13 is 100 m, and 1.3 cm in all directions when the distance is 10 m. The required accuracy acquired by the required accuracy acquisition unit 22 is output to the superposition position calculation unit 25.

[0023] The camera parameter acquisition unit 23 acquires the camera parameters of the camera sensor 13. The camera parameters include, for example, the mounting position of the camera sensor 13 relative to the vehicle 10, the direction of the optical axis of the camera sensor 13, etc. The camera calibration process is a process of adjusting the camera parameters to an appropriate range. The camera parameters of the camera sensor 13 acquired by the camera parameter acquisition unit 23 are output to the superposition position calculation unit 25.

[0024] The camera image acquisition unit 24 acquires a captured image of the confirmation target 2 arranged around the vehicle 10, captured by the camera sensor 13. The captured image acquired by the camera image acquisition unit 24 is output to the superimposed image creation unit 26.

[0025] The confirmation target 2 is a target used to confirm the calibration results of the camera sensor 13. As shown in FIGS. 3 and 4(a), the confirmation target 2 is a columnar (more specifically, cylindrical) object with one or more marks at predetermined heights. The confirmation target 2 has a mark C2 at a height H2 from the ground 3 and a mark C3 at a height H3 from the ground 3. The marks C2 and C3 are cross-shaped and are positioned so that their centers are located at heights H2 and H3. As shown in FIG. 3, when the position where the confirmation target 2 is located is designated as position P1, the mark C2 is positioned at height H2, which is the intersection of the line segment connecting the camera sensor 13 to position P2 and the confirmation target 2, and the mark C3 is positioned at height H3, which is the intersection of the line segment connecting the camera sensor 13 to position P3 and the confirmation target 2. The heights H2 and H3 of the marks C2 and C3 may be adjusted according to the set required accuracy, or the set required accuracy may be converted according to the heights H2 and H3 of the marks C2 and C3.

[0026] The superimposition position calculation unit 25 calculates the display positions of the required accuracy ranges R2 and R3, which indicate the required accuracy of the camera sensor 13 corresponding to the heights of the markers C1 and C2, based on the position information of the confirmation target 2 output from the position information acquisition unit 21 and the required accuracy output from the required accuracy acquisition unit 22. The superimposition position calculation unit 25 further calculates the required accuracy range R1, which corresponds to the bottom of the confirmation target 2 (the position of the ground 3). The superimposition position calculation unit 25 calculates the positions of the required accuracy ranges R1, R2, and R3 in order to create the superimposed image shown in FIG. 4(b). The calculated positions of the required accuracy ranges R1, R2, and R3 are output to the superimposed image creation unit 26.

[0027] The required accuracy ranges R1 to R3 are ranges within which the bottom of the confirmation target 2 and the landmarks C2 and C3 may exist in the confirmation image if the calibration result of the camera sensor 13 is appropriate (i.e., if the camera sensor 13 is calibrated to satisfy the required accuracy). The required accuracy range R1 is calculated to be a predetermined range around the ground 3 at zero height, according to the required accuracy corresponding to position P1. The required accuracy range R2 is calculated to be a predetermined range around the landmark C2 at height H2, according to the required accuracy corresponding to position P2. The required accuracy range R3 is calculated to be a predetermined range around the landmark C3 at height H3, according to the required accuracy corresponding to position P3.

[0028] The superimposed image creation unit 26 creates a superimposed image showing the required accuracy ranges R1 to R3 and superimposes it on the camera image. The superimposed image is an image in which the required accuracy ranges R1 to R3 are clearly indicated so that they can be visually confirmed, and is created based on the display positions of the required accuracy ranges R1 to R3 calculated by the superimposition position calculation unit 25. For example, as shown in FIG. 4(b), the display positions of the required accuracy ranges R1 to R3 can be marked to clearly indicate them so that the worker can visually confirm them. An image synthesis processing method such as alpha blending can be used to superimpose the camera image and the superimposed image. Alpha blending is a method of semi-transparently combining two images using a coefficient (alpha value). By creating a superimposed image showing the required accuracy ranges R1 to R3 shown in FIG. 4(b) and superimposing it on the image captured by the camera sensor 13 shown in FIG. 4(a), the confirmation image shown in FIG. 4(c) can be created.

[0029] The superimposed image may be created by outputting the superimposed image to display device 14 and superimposing the superimposed image on a camera image already displayed on display device 14. For example, when a camera image such as that shown in FIG. 4(a) is already displayed on display device 14 and a superimposed image such as that shown in FIG. 4(b) is output to display device 14, a confirmation image such as that shown in FIG. 4(c) may be displayed on display device 14. Alternatively, a confirmation image in which the camera image and the superimposed image are superimposed in advance in ECU 20 may be created, and the confirmation image may be output to display device 14 for display. For example, a confirmation image such as that shown in FIG. 4(c) may be created in ECU 20 and output to display device 14, and the confirmation image may be displayed on display device 14.

[0030] 5 and 6 show examples of confirmation images displayed on the display screen of display device 14. Display screen 50 shown in FIG. 5 includes front display 51, back display 52, right display 53, and left display 54, which indicate the position of camera sensor 13, and confirmation image display 55, which displays a confirmation image. Front display 51 lights up when confirmation image display 55 corresponds to camera sensor 13F. Back display 52 lights up when confirmation image display 55 corresponds to camera sensor 13B. Right display 53 lights up when confirmation image display 55 corresponds to camera sensor 13R. Left display 54 lights up when confirmation image display 55 corresponds to camera sensor 13L. Each of displays 51 to 54 may simply indicate its correspondence with confirmation image display 55, or may also function as an input interface for switching confirmation image display 55 using a touch panel function.

[0031] 6, in addition to displays 61-64 and confirmation image display 65, an enlarged image display 66 may be provided that enlarges the area around confirmation target 2 in confirmation image display 65. Displays 61-64 and confirmation image display 65 are similar to displays 51-54 and confirmation image display 55 shown in FIG.

[0032] Fig. 7 shows a flowchart of a method for calibrating and verifying the calibration of camera sensor 13 mounted on vehicle 10. Specifically, this calibration and verification method involves calibrating camera sensor 13 and then verifying the calibration results using a verification image created by a verification image display device. The left side of the dashed line in Fig. 7 shows the process on the part of the on-board system mounted on vehicle 10, and the right side shows the process performed by an operator.

[0033] In step S101, the in-vehicle system executes a process of calibrating the camera sensor 13. There are no particular limitations on the method of calibrating the camera sensor 13, and various calibration methods can be used. Also, in this flowchart, the process related to camera calibration is performed by the system, but the process related to camera calibration may also be performed by an operator.

[0034] Steps S102 to S114 show the processing related to the calibration verification method. In step S102, the operator places the verification target 2. In this embodiment, the operator places the verification target 2 at an arbitrary position.

[0035] The processes shown in steps S103 to S108 are executed by the on-board system. In step S103, a process is executed to acquire position information of the confirmation target 2 from the on-board radar sensor 12 or the measuring instrument 1 installed outside the vehicle. For example, the relative positional relationship (for example, positional relationship in the front-rear and left-right directions) between the position of the camera sensor 13 of the vehicle 10 and the position where the confirmation target 2 is located is acquired as the position information.

[0036] In step S104, a process is executed to acquire a camera image from the camera sensor 13. For example, a camera image including a confirmation target 2 as shown in FIG.

[0037] In step S105, the display positions of the required accuracy ranges R1 to R3 are calculated. The display positions of the required accuracy ranges R1 to R3 are calculated based on the position of the confirmation target 2 and the required accuracy of the camera sensor 13 corresponding to the heights H2 and H3 of the marks C2 and C3. The display positions of the required accuracy ranges R1 to R3 are calculated based on the position information of the confirmation target 2 acquired in step S103. This prevents the display positions of the required accuracy ranges R1 to R3 from becoming inappropriate due to a shift in the position of the confirmation target 2.

[0038] In step S106, a superimposed image is created in which the display positions of the required accuracy ranges R1 to R3 calculated in step S105 are marked, thereby creating a superimposed image displaying the required accuracy ranges R1 to R3, for example, as shown in Figure 4(b).

[0039] In step S107, the superimposed image created in step S106 is superimposed on the camera image acquired in step S104. As a result, as shown in the subsequent step S108, a confirmation image is displayed on display device 14. By superimposing the superimposed image created in step S106 on the camera image that was being displayed on display device 14, a confirmation image such as that shown in FIG. 4(c) is displayed on display device 14. More specifically, the confirmation image is displayed on the display screen of display device 14 in a manner such as that shown in confirmation image displays 55 and 65 and enlarged image display 66 in FIGS. 5 and 6. Thereafter, the process proceeds to step S111.

[0040] Each process shown in steps S111 to S114 is performed by an operator. In step S111, the operator visually checks the confirmation image displayed on the display screen of display device 14 to confirm whether the accuracy of camera sensor 13 is within the required accuracy range. Specifically, for example, when the operator visually checks enlarged image display 66 shown in FIG. 6 and finds that the bottom of confirmation target 2, mark C2, and mark C3 are all located within the corresponding required accuracy ranges R1, R2, and R3, the operator determines that the accuracy of camera sensor 13 is within the required accuracy range, and proceeds to step S112, where the calibration of camera sensor 13 is completed.

[0041] If any one of the bottom of the confirmation target 2, the mark C2, and the mark C3 is located outside the corresponding required accuracy range R1, R2, or R3, it is determined that the accuracy of the camera sensor 13 is not within the required accuracy range, and the process proceeds to step S113. In step S113, it is determined that the camera sensor 13 is to be recalibrated, and in the following step S114, the calibration environment is reviewed and the process returns to step S101. As a result, the camera calibration process shown in step S101 is executed in the calibration environment reviewed in step S114, and the subsequent process related to the calibration confirmation method is executed.

[0042] As described above, the calibration confirmation method according to the first embodiment includes an image acquisition step (step S104) for acquiring a captured image of a confirmation target 2, with one or more markers C2 and C3 attached at a predetermined height, placed around a vehicle 10, captured by a camera sensor 13; a superimposition position calculation step (step S105) for calculating the display position of a required accuracy range R1 to R3 that represents the required accuracy of the camera corresponding to the position information of the confirmation target 2 and the heights H2 and H3 of the markers C2 and C3; a superimposition image creation step (steps S106 to S108) for creating a superimposed image in which the required accuracy range R1 to R3 is superimposed on the captured image and displaying it on a display device 14; and a confirmation step (step S111) for determining that the camera calibration result satisfies the required accuracy of the camera if the markers C2 and C3 of the confirmation target 2 are displayed within the required accuracy range. The ECU 20 can cause the display device 14 to display a confirmation image by executing a program including an image acquisition step (step S104), a superimposition position calculation step (step S105), and a superimposition image creation step (steps S106 to S108).

[0043] According to the calibration confirmation method of the first embodiment, as shown in step S102, the operator places the confirmation target 2, which has markers C2 and C3 attached at heights H2 and H3 corresponding to the required accuracy set according to the distance from the camera sensor 13, around the vehicle 10. As a result, as shown in step S104, the camera sensor 13 can capture an image of the confirmation target 2 in its placed state. Then, as shown in step S104, the superimposed image created in step S106 can be superimposed on the captured image obtained in step S104, and the superimposed image can be displayed on the display device 14 as a confirmation image as shown in step S108. According to the confirmation image displayed in step S108, if the bottom of the confirmation target 2 and the markers C2 and C3 are all displayed within the corresponding required accuracy ranges R1, R2, and R3, it can be determined that the calibration result of the camera sensor 13 satisfies the required accuracy of the camera. Therefore, the operator can easily confirm the calibration result by visually checking the confirmation image displayed on the display device 14.

[0044] Furthermore, even if the worker places the confirmation target 2 at an arbitrary position, in step S103, the position where the confirmation target 2 is placed can be measured by the radar sensor 12 or the measuring instrument 1 and acquired as position information of the confirmation target 2. Since the measured position of the confirmation target 2 measured with high accuracy using a measuring device can be used as the position information of the confirmation target 2, deviations in the position where the confirmation target 2 is placed can be suppressed.

[0045] As a result, the display positions of the required accuracy ranges R1, R2, and R3 calculated in the superposition position calculation process shown in step S105 can be prevented from being inappropriate positions, and misinterpretation of the calibration results when checking the calibration of the camera sensor 13 can be prevented.

[0046] (Second embodiment) Fig. 8 shows a flowchart of a method for calibrating and verifying the calibration of camera sensor 13 according to the second embodiment. The left side of the dashed line in Fig. 8 shows the process on the part of the on-board system installed in vehicle 10, and the right side shows the process performed by the worker. The second embodiment differs from the first embodiment in that, after calculating the designated drawing positions for drawing the required accuracy ranges R1 to R3 and creating a superimposed image, a confirmation target 2 is placed at the designated drawing position and the confirmation image is displayed.

[0047] In step S201, similar to step S101, the in-vehicle system executes a process of calibrating the camera sensor 13, and then the process proceeds to step S203.

[0048] In step S203, a designated drawing position for drawing the required accuracy ranges R1 to R3 is calculated based on the camera parameters of camera sensor 13. For example, a position where confirmation target 2 can be suitably placed can be calculated as the designated drawing position based on camera parameters such as the mounting position of camera sensor 13 relative to vehicle 10 and the direction of the optical axis of camera sensor 13.

[0049] In the following step S204, a process is executed to acquire a camera image from the camera sensor 13. The confirmation target 2 is not captured in the camera image acquired here.

[0050] In step S205, the display positions of the required accuracy ranges R1 to R3 are calculated. The display positions of the required accuracy ranges R1 to R3 are calculated based on the position of the confirmation target 2 and the required accuracy of the camera sensor 13 corresponding to the heights H2 and H3 of the marks C2 and C3, as in step S105. However, this differs from step S105 in that the designated drawing position calculated in step S203 is used as the position of the confirmation target 2.

[0051] In step S206, a superimposed image is created in which the display positions of the required accuracy ranges R1 to R3 calculated in step S205 are marked. In the following step S207, the superimposed image created in step S206 is superimposed on the camera image acquired in step S204. As a result, as shown in the following step S208, an instruction image is displayed on display device 14. Note that the instruction image refers to an image in which the superimposed image is superimposed on a camera image in which confirmation target 2 is not captured.

[0052] For example, first, as shown in FIG. 9(a), an image can be created that clearly indicates the designated drawing positions R1L, R1C, and R1R specified in step S203. Note that the grid shown in FIG. 9 indicates coordinates in the camera image. Next, at the designated drawing positions R1L, R1C, and R1R, the display positions of the required accuracy ranges R1 to R3 corresponding to heights of zero, H2, and H3, respectively, are calculated. Then, by clearly indicating the calculated required accuracy ranges R1 to R3 on the image, an indication image can be created as shown in FIG. 9(b). The dashed lines 2La, 2Ca, and 2Ra shown in FIG. 9(b) are indicators for placing the confirmation target 2. Then, proceed to step S209.

[0053] In step S209, the operator places the confirmation targets 2 at the designated drawing positions R1L, R1C, and R1R. While visually checking the instruction image, the operator positions the confirmation targets 2 so that they fit within the dashed lines 2La, 2Ca, and 2Ra, thereby accurately placing the confirmation targets 2 at the designated drawing positions R1L, R1C, and R1R. The confirmation targets 2L, 2C, and 2R shown in FIG. 9(c) illustrate the confirmation targets 2 placed at the designated drawing positions R1L, R1C, and R1R, respectively. One confirmation target 2 may be moved sequentially to obtain individual images of the confirmation targets 2 placed at the three designated drawing positions R1L, R1C, and R1R. Alternatively, three identical confirmation targets 2 may be prepared, and images of the confirmation targets 2 placed at each of the three designated drawing positions R1L, R1C, and R1R may be simultaneously obtained.

[0054] As a result of the operator placing the confirmation target 2 in step S208, as shown in step S210, an image in which the superimposed image is superimposed on the camera image capturing the confirmation target 2 is displayed on the display device 14. In other words, the confirmation image is displayed on the display device 14.

[0055] Thereafter, in step S211, the worker checks whether the accuracy of camera sensor 13 is within the required accuracy range by visually checking the confirmation image displayed on display device 14. The processes shown in steps S211 to S214 are the same as the processes shown in steps S111 to S114 in FIG. 7, and therefore description thereof will be omitted.

[0056] According to the calibration confirmation method of the second embodiment, as shown in step S203, a designated drawing position calculation step is executed to calculate designated drawing positions for drawing the required accuracy ranges R1 to R3 based on the camera parameters of the camera sensor 13. In a superimposition position calculation step shown in step S205, the designated drawing positions R1L, R1C, and R1R calculated in step S203 are used as position information for the confirmation target 2 to calculate the display position of the required accuracy ranges R1 to R3. In steps S206 to S208 of the superimposed image creation step, a superimposed image is superimposed on a captured image in which the confirmation target 2 is not captured, and an instruction image is displayed on the display device 14. Thereafter, in step S209, the operator places the confirmation target 2 at the designated drawing positions R1L, R1C, and R1R while visually checking the instruction image displayed on the display device 14. As shown in step S210, a confirmation image similar to that of the first embodiment can be displayed on the display device 14. Therefore, as in the first embodiment, the operator can easily check the calibration results by visually checking the confirmation image displayed on the display device 14. Furthermore, while visually checking the instruction image, the operator positions the confirmation target 2 so that it falls within the dashed lines 2La, 2Ca, and 2Ra, which are indicators for positioning the confirmation target 2, thereby preventing deviation from the position of the placed confirmation target 2. As a result, the required accuracy ranges R1 to R3 are prevented from being displayed in inappropriate positions, and misinterpretation of the calibration results when checking the calibration of the camera sensor 13 is prevented.

[0057] (Third embodiment) 10 shows a flowchart of a method for calibrating and verifying the calibration of the camera sensor 13 according to the third embodiment. The left side of the dashed line in FIG. 10 shows the process on the part of the on-board system installed in the vehicle 10, and the right side shows the process performed by the operator. The third embodiment differs from the first embodiment in that a measurement position, which is the position of the confirmation target 2, is calculated from the measurement data of the camera sensor 13, and the position information acquisition unit 21 acquires this calculated measurement position as the position information of the confirmation target 2.

[0058] In step S301, similar to step S101, the in-vehicle system performs a process of calibrating the camera sensor 13, and then the process proceeds to step S302. In step S302, the worker places the confirmation target 2. In this embodiment, the worker places the confirmation target 2 in an arbitrary position.

[0059] In step S303, similar to step S104, a process for acquiring a camera image from camera sensor 13 is executed. As a result, a captured image of confirmation target 2 placed at an arbitrary position can be acquired, as shown in FIG. 11(a). Then, the process proceeds to step S304. In step S304, a measurement position, which is the position of confirmation target 2 measured by camera sensor 13, is calculated based on the image information acquired from camera sensor 13, and this measurement position is acquired as position information of the confirmation target. Then, the process proceeds to step S305.

[0060] In step S305, the display positions of the required accuracy ranges R1 to R3 are calculated. The display positions of the required accuracy ranges R1 to R3 are calculated based on the position of the confirmation target 2 and the required accuracy of the camera sensor 13 corresponding to the heights of the markers C2 and C3, as in step S105. However, this differs from step S105 in that the position information of the confirmation target 2 calculated based on the captured image calculated in step S304 is used.

[0061] In step S306, a superimposed image is created in which the display positions of the required accuracy ranges R1 to R3 calculated in step S305 are marked. After step S306, the process proceeds to step S307. The processing shown in steps S307 to S314 is the same as the processing shown in steps S107 to S114 in FIG. 7, and therefore a description thereof will be omitted.

[0062] According to the calibration confirmation method of the third embodiment, as shown in steps S303 and S304, the measurement position of the confirmation target 2 measured by the camera sensor 13 is acquired as position information. In a superimposition position calculation step shown in step S305, the measurement position information input in step S304 is used as position information of the confirmation target 2 to calculate a display position within the required accuracy range. Thereafter, a superimposed image creation step shown in steps S306 to S308 allows a confirmation image similar to that of the first embodiment to be displayed on the display device 14. Therefore, as in the first embodiment, the operator can easily confirm the calibration results by visually checking the confirmation image displayed on the display device.

[0063] Furthermore, even if the operator places the confirmation target 2 in a position of their choice, in step S304 the position where the confirmation target 2 is placed can be measured by the camera sensor 13 and acquired as position information of the confirmation target 2. Because the position measured with high accuracy using the camera sensor 13 can be used as the position information of the confirmation target 2, deviations in the position where the confirmation target 2 is placed can be reduced. As a result, it is possible to prevent the display positions of the required accuracy ranges R1 to R3 from being inappropriate positions, and to reduce misinterpretation of the calibration results when checking the calibration of the camera sensor 13.

[0064] Since the position information of the confirmation target 2 can be acquired by the camera sensor 13, the vehicle 10 may be configured not to include an on-board sensor (such as the radar sensor 12) that can acquire the position information of the confirmation target 2 other than the camera sensor 13. On the other hand, the vehicle 10 may be configured to acquire the position information of the confirmation target 2 using both the radar sensor 12 and the camera sensor 13. By acquiring the position information of the confirmation target 2 using multiple measuring devices, the accuracy of the position information can be improved.

[0065] (Fourth embodiment) FIG. 12 shows a vehicle 30 equipped with an ECU 40 that functions as a calibration confirmation device according to the fourth embodiment. The vehicle 30 includes a camera sensor 33, an ECU 40, and a display device 34. The ECU 40 includes a position information acquisition unit 41, a required accuracy acquisition unit 42, a camera parameter acquisition unit 43, a camera image acquisition unit 44, a superimposition position calculation unit 45, and a superimposition image creation unit 46. The captured image acquired by the camera image acquisition unit 44 from the camera sensor 33 is output to the superimposition image creation unit 46 and the display device 34. The display device 34 includes an interface that allows an operator to input operation information. The operator can input position information of the confirmation target 2 to the display device 34, and the input position information of the confirmation target 2 is output from the display device 34 to the superimposition position calculation unit 45. The superimposition position calculation unit 45 is configured to calculate the display position of a required accuracy range, which represents the required accuracy of the camera corresponding to the height of the landmark, based on the position information of the confirmation target 2 output from the display device 34 and the required accuracy output from the required accuracy acquisition unit 22. Except for the matters described above, the configurations shown in FIG. 12 are the same as the configurations shown in FIG. 1, and therefore the explanations will be omitted by replacing the reference numbers in the 10s and 20s of FIG. 1 with the reference numbers in the 30s and 40s of FIG. 12, respectively.

[0066] 13 shows a flowchart of the method for calibrating and verifying the calibration of the camera sensor 13 according to the fourth embodiment. The left side of the dashed line in FIG. 13 shows the process on the side of the on-board system installed in the vehicle 10, and the right side shows the process by the worker.

[0067] In step S401, similar to step S101, the in-vehicle system performs a process of calibrating the camera sensor 13, and then the process proceeds to step S402. In step S402, the worker places the confirmation target 2. In this embodiment, the worker places the confirmation target 2 in an arbitrary position.

[0068] In step S403, similar to step S104, a process is executed to acquire a camera image from camera sensor 13. This allows acquisition of a captured image of confirmation target 2 placed at an arbitrary position. Then, the process proceeds to step S404.

[0069] In step S404, the worker inputs position information of the confirmation target 2 into the display device 34. As shown in FIG. 14, for example, the display device 34 is provided with a touch panel-type input unit 77 at the bottom of the display screen 70, which the worker can input information from. The input unit 77 is configured to allow the worker to input position information of the confirmation target 2 (more specifically, the front, back, left and right positions of the confirmation target 2) and the height of the mark on the confirmation target 2. Note that the displays 71 to 76 are the same as the displays 61 to 66 shown in FIG. 6. Then, the process proceeds to step S405.

[0070] In step S405, the display positions of the required accuracy ranges R1 to R3 are calculated. The display positions of the required accuracy ranges R1 to R3 are calculated based on the position of the confirmation target 2 and the required accuracy of the camera sensor 13 corresponding to the heights of the markers C2 and C3, as in step S105. However, this differs from step S105 in that the position information of the confirmation target 2 input by the operator via the display device 34 in step S404 is used.

[0071] In step S406, a superimposed image is created in which the display positions of the required accuracy ranges R1 to R3 calculated in step S405 are marked. After step S406, the process proceeds to step S407. The processing shown in steps S407 to S414 is the same as the processing shown in steps S107 to S114 in Fig. 7, and therefore description thereof will be omitted.

[0072] According to the calibration confirmation method of the fourth embodiment, as shown in step S404, the operator inputs position information of the confirmation target 2 from the display device 34. In the superimposition position calculation step shown in step S405, the display position within the required accuracy range is calculated using the position information of the confirmation target 2 input in step S404. Thereafter, in the superimposition image creation step shown in steps S406 to S408, a confirmation image similar to that of the first embodiment can be displayed on the display device 14. Therefore, as in the first embodiment, the operator can easily confirm the calibration results by visually checking the confirmation image displayed on the display device 14.

[0073] Furthermore, according to the calibration confirmation method of the fourth embodiment, the position information of the confirmation target 2 can be obtained by input from the operator, and therefore the method can be applied even if the vehicle 10 does not have an on-board sensor (such as a radar sensor 12) that can obtain the position information of the confirmation target 2.

[0074] (Fifth embodiment) The fifth embodiment differs from the first embodiment in that, as shown in FIG. 15, the positions of the confirmation target 2 and the positions of the landmarks are displayed on the display device 14 as a continuous position display W that is concentrically arranged around the camera sensor 13 when the vehicle 10 is viewed from above.

[0075] Specifically, as shown in FIG. 16(a), the display screen of display device 14 displays a continuous position display W1 indicating the position of confirmation target 2, a continuous position display W2 indicating the position of marker C2 on confirmation target 2, and a continuous position display W3 indicating the position of marker C3 on confirmation target 2. The continuous position displays W1-W3 are concentric circles, similar to the continuous position display W shown in FIG. 15. While visually viewing the display screen of display device 14, the operator can place confirmation target 2 at any position along the continuous position displays W1-W3, as shown at confirmation target position 2b in FIG. 16(b). As shown in FIG. 16(c), the position of confirmation target 2 may be sequentially moved from confirmation target position 2b to confirmation targets 2c and 2d, and images of the confirmation target 2 positioned at each of confirmation targets 2b, 2c, and 2d may be captured individually. Alternatively, three identical confirmation targets 2 may be prepared, and images of the confirmation targets 2 positioned at each of confirmation targets 2b, 2c, and 2d may be captured simultaneously.

[0076] 17 shows a display screen 80 of the display device 14 on which successive position displays W1 to W3 are displayed. A confirmation image display 85 and an enlarged image display 86 differ from the confirmation image display 65 and the enlarged image display 66 shown in FIG. 6 in that successive position displays W1 to W3 are displayed in the confirmation image display 85 and the enlarged image display 86. In the confirmation images displayed in the confirmation image display 85 and the enlarged image display 86, successive position displays W1 to W3 indicating the position of the confirmation target 2 are drawn in a substantially circular shape with the camera sensor 13 at the center. Note that the displays 81 to 84 are similar to the displays 61 to 64 shown in FIG. 6.

[0077] While the above embodiments have been described with reference to an example in which an operator visually checks the confirmation image to confirm the calibration results, the ECUs 20 and 40 may be configured to determine whether the bottom of the confirmation target 2, the mark C2, and the mark C3 are all displayed within the corresponding required accuracy ranges R1, R2, and R3, thereby determining whether the calibration results of the camera sensor 13 satisfy the required accuracy of the camera. The calibration confirmation program executed by the ECUs 20 and 40 may include a confirmation step for determining that the calibration results of the camera sensor 13 satisfy the required accuracy of the camera when the marks C2 and C3 of the confirmation target 2 are displayed within the required accuracy ranges R1 to R3, and the ECUs 20 and 40 may be provided with a confirmation unit for determining that the calibration results of the camera sensor 13 satisfy the required accuracy of the camera. Furthermore, while the above embodiments have been described with reference to an example in which an operator places the confirmation target 2, the ECUs 20 and 40 may place the confirmation target 2 by instructing a device or the like that moves the confirmation target 2.

[0078] According to each of the above embodiments, the following effects can be obtained.

[0079] The calibration confirmation method for confirming the calibration result of the camera sensor 13 mounted on the vehicle 10 includes image acquisition steps (S103, S204, S303, S403), superimposition position calculation steps (S105, S205, S305, S405), superimposed image creation steps (S106 to S108, S206 to S208, S210, S306 to S308, S406 to S408), and confirmation steps (S111, S211, S311, S411).

[0080] The image acquisition step acquires a captured image of a state in which a confirmation target 2, with one or more markers C2, C3 attached at a predetermined height, is placed around the vehicle 10, captured by the camera sensor 13. The superimposition position calculation step calculates the display position of the required accuracy range R1 to R3, which indicates the required accuracy of the camera sensor 13, corresponding to the heights H2, H3 of the markers C2, C3, based on the position information of the confirmation target 2 and the required accuracy of the camera sensor 13. The superimposed image creation step creates a superimposed image indicating the display position of the required accuracy range R1 to R3, and displays the confirmation image, in which the superimposed image is superimposed on the captured image, on the display devices 14, 34. The confirmation step determines that the calibration result of the camera sensor 13 satisfies the required accuracy of the camera, if the markers C2, C3 of the confirmation target 2 are displayed within the required accuracy range R1 to R3 in the confirmation image.

[0081] According to the above calibration confirmation method, a confirmation target 2, with one or more markers C2, C3 attached at a predetermined height, is photographed by the camera sensor 13, and a required accuracy range R1 to R3 is superimposed on the photographed image. The superimposed image is then displayed on the display device 14, 34 as a confirmation image. According to this confirmation image, if the markers C2, C3 on the confirmation target 2 are displayed within the required accuracy range R1 to R3, it can be determined that the calibration result of the camera sensor 13 satisfies the required accuracy of the camera sensor 13. Therefore, the operator can easily confirm the calibration result by visually checking the confirmation image displayed on the display device 14, 34. Furthermore, the display position of the required accuracy range R1 to R3 is calculated based on the position information of the confirmation target 2, which prevents the required accuracy range from being displayed in an inappropriate position due to a positional shift of the confirmation target 2. This prevents erroneous recognition of the calibration result when confirming the calibration of the camera sensor 13.

[0082] The above calibration confirmation method may further include a position information acquisition step (S103, S304, S404) for acquiring position information of the confirmation target 2. In this case, the superimposition position calculation step can be configured to calculate the display position of the required accuracy ranges R1 to R3 based on the position information of the confirmation target 2 acquired in the position information acquisition step.

[0083] The position information acquisition step may be configured to acquire position information of the confirmation target 2 measured by a predetermined measurement device (for example, measuring instrument 1, radar sensor 12, camera sensor 13). Since the measured position of the confirmation target 2 measured with high accuracy using a measurement device can be used as the position information of the confirmation target 2, it is possible to suppress deviation of the position where the confirmation target 2 is placed. When the camera sensor 13 mounted on the vehicle 10 is used as the measurement device, the calibration confirmation method can be applied even to a vehicle that does not have an on-board sensor (radar sensor 12, etc.) other than the camera sensor 13 that can acquire position information of the confirmation target 2.

[0084] The position information acquisition step may be configured to acquire, as position information of the confirmation target, the position at which the confirmation target 2 is to be placed, input via the display device 34. Since the position information of the confirmation target 2 can be acquired by input by the worker, this method is applicable even when the vehicle 10 does not have an on-board sensor (such as the radar sensor 12) that can acquire the position information of the confirmation target 2.

[0085] The above calibration confirmation method may further include a designated drawing position calculation step (S203) for calculating a designated drawing position for drawing the required accuracy ranges R1 to R3 based on the camera parameters of the camera sensor 13. In this case, the superimposition position calculation step can be configured to calculate the display position of the required accuracy ranges R1 to R3 using the designated drawing position calculated in the designated drawing position calculation step as position information of the confirmation target 2.

[0086] In the above calibration confirmation method, the confirmation image may depict the position of confirmation target 2 in a substantially circular shape centered on camera sensor 13. The worker can place confirmation target 2 at any position along continuous position displays W1 to W3 while visually checking the display screen of display device 14. Because the worker can visually check the image in which the superimposed image is superimposed on the camera image and place confirmation target 2 in an appropriate position, deviation from the position of confirmation target 2 can be reduced.

[0087] The above calibration confirmation method can be realized by causing the ECU 20, 40 to execute the following calibration confirmation program. The calibration confirmation program includes an image acquisition step, a superimposition position calculation step, and a superimposition image creation step, and may further include a position information acquisition step and a drawing specification position calculation step.

[0088] The above calibration confirmation method can be realized by the ECU 20, 40 functioning as a calibration confirmation device. The ECU 20, 40 includes a camera image acquisition unit 24, 44, a superimposition position calculation unit 25, 45, and a superimposition image creation unit 26, 46, and may further include a position information acquisition unit 21, 41, a required accuracy acquisition unit 22, 42, and a camera parameter acquisition unit 23, 43.

[0089] The camera image acquisition unit 24, 44 acquires a captured image taken by the camera sensor 13, 33 of a confirmation target 2, with one or more markers C2, C3 attached at a predetermined height, arranged around the vehicle 10. The superimposition position calculation unit 25, 45 calculates the display position of a required accuracy range R1 to R3 that indicates the required accuracy of the camera sensor 13, 33 corresponding to the heights H2, H3 of the markers C2, C3, based on the position information of the confirmation target 2 and the required accuracy of the camera sensor 13, 33. The superimposed image creation unit 26, 46 creates a superimposed image that indicates the display position of the required accuracy range R1 to R3, and displays a confirmation image on the display device 14, 34, with the superimposed image superimposed on the captured image.

[0090] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0091] The following describes characteristic configurations extracted from the above-described embodiments. [Configuration 1] A calibration confirmation method for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), comprising: an image acquisition step (S103, S204, S303, S403) for acquiring a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is arranged around the vehicle, the confirmation target (2) being photographed by the camera (13, 33); a superimposition position calculation step (S105, S205, S305, S405) for calculating a display position of a required accuracy range representing the required accuracy of the camera corresponding to the height of the landmark based on the position information of the confirmation target and the required accuracy of the camera; a superimposed image creating step (S106 to S108, S206 to S208, S210, S306 to S308, S406 to S408) of creating a superimposed image indicating the display position of the required accuracy range and displaying a confirmation image in which the superimposed image is superimposed on the captured image on a display device (14, 34); A calibration confirmation method including a confirmation step (S111, S211, S311, S411) of determining that the calibration result of the camera satisfies the required accuracy of the camera if the mark of the confirmation target is displayed within the required accuracy range in the confirmation image. [Configuration 2] Further includes a position information acquisition step (S103, S304, S404) of acquiring position information of the confirmation target, The calibration verification method according to configuration 1, wherein the superimposition position calculation step calculates the display position of the required accuracy range based on the position information of the verification target acquired in the position information acquisition step. [Configuration 3] The calibration verification method according to configuration 2, wherein the position information acquisition step acquires position information of the verification target measured by a predetermined measurement device (1, 12, 13). [Configuration 4] 4. The calibration verification method according to claim 3, wherein the measuring device is a camera (13) mounted on the vehicle. [Configuration 5] 3. The calibration verification method according to configuration 2, wherein the position information acquisition step acquires, as position information of the verification target, a position at which the verification target is to be placed, input via the display device. [Configuration 6] The method further includes a drawing designation position calculation step (S203) of calculating a drawing designation position for drawing the required accuracy range based on the camera parameters of the camera, The calibration verification method according to any one of configurations 1 to 5, wherein the superposition position calculation step calculates the display position within the required accuracy range using the designated drawing position calculated in the designated drawing position calculation step as position information of the verification target. [Configuration 7] 7. The calibration verification method according to any one of configurations 1 to 6, wherein the verification image depicts the position of the verification target in a substantially circular shape with the camera at the center. [Configuration 8] A calibration confirmation program for displaying on a display device (14) a confirmation image for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), the program comprising: an image acquisition step (S103, S204, S303, S403) for acquiring a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is arranged around the vehicle, the confirmation target (2) being photographed by the camera (13, 33); a superimposition position calculation step (S105, S205, S305, S405) for calculating a display position of a required accuracy range representing the required accuracy of the camera corresponding to the height of the landmark based on the position information of the confirmation target and the required accuracy of the camera; and a superimposed image creation step (S106 to S108, S206 to S208, S210, S306 to S308, S406 to S408) for creating a superimposed image indicating the display position of the required accuracy range and displaying a confirmation image in which the superimposed image is superimposed on the captured image on a display device (14, 34). [Configuration 9] A calibration confirmation device (20, 40) that displays a confirmation image on a display device (14, 34) for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), an image acquisition unit (24, 44) that acquires a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is placed around the vehicle, the image being photographed by the camera; a superimposition position calculation unit (25, 45) that calculates a display position of a required accuracy range that indicates the required accuracy of the camera corresponding to the height of the landmark based on the position information of the confirmation target and the required accuracy of the camera; A calibration confirmation device comprising: a superimposed image creation unit (26, 46) that creates a superimposed image indicating the display position of the required accuracy range and displays a confirmation image on a display device in which the superimposed image is superimposed on the captured image. [Explanation of symbols]

[0092] 2... confirmation target, 13, 33... camera sensor, 14, 34... display device, 20, 40... ECU, 24, 44... camera image acquisition unit, 25, 45... superimposition position calculation unit, 26, 46... superimposition image creation unit

Claims

1. A calibration confirmation method for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), comprising: an image acquisition step (S104, S204, S303, S403) of acquiring a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is arranged around the vehicle, the confirmation target (2) being photographed by the camera (13, 33); a superimposition position calculation step (S105, S205, S305, S405) of calculating a display position of a required accuracy range representing the required accuracy of the camera corresponding to the height of the landmark as a display position on the image captured by the camera, based on position information of the confirmation target indicating the relative positional relationship of the confirmation target with respect to the camera and the required accuracy of the camera determined according to the distance to a position on the ground that is an extension of a straight line connecting the camera and the landmark; a superimposed image creating step (S106 to S108, S206 to S208, S210, S306 to S308, S406 to S408) of creating a superimposed image indicating the display position of the required accuracy range and displaying a confirmation image in which the superimposed image is superimposed on the captured image on a display device (14, 34); A calibration confirmation method including a confirmation step (S111, S211, S311, S411) of determining that the calibration result of the camera satisfies the required accuracy of the camera if the mark of the confirmation target is displayed within the required accuracy range in the confirmation image.

2. The method further includes a position information acquisition step (S103, S304, S404) of acquiring position information of the confirmation target, 2. The calibration verification method according to claim 1, wherein the superimposition position calculation step calculates the display position of the required accuracy range based on the position information of the verification target acquired in the position information acquisition step.

3. 3. The calibration verification method according to claim 2, wherein the position information acquisition step acquires position information of the verification target measured by a predetermined measuring device (1, 12, 13).

4. 4. The calibration verification method according to claim 3, wherein the measuring device is a camera (13) mounted on the vehicle.

5. 3. The calibration verification method according to claim 2, wherein the position information acquisition step acquires, as the position information of the verification target, a position at which the verification target is to be placed, input via the display device.

6. The method further includes a designated drawing position calculation step (S203) of calculating a designated drawing position for drawing the required accuracy range based on the camera parameters of the camera, 2. The calibration verification method according to claim 1, wherein the superimposition position calculation step calculates the display position within the required accuracy range using the designated drawing position calculated in the designated drawing position calculation step as position information of the verification target.

7. 7. The calibration verification method according to claim 1, wherein the verification image depicts the position of the verification target in a substantially circular shape with the camera at its center.

8. A calibration confirmation program for displaying on a display device (14) a confirmation image for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), the program comprising: an image acquisition step (S104, S204, S303, S403) of acquiring a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is arranged around the vehicle, the confirmation target (2) being photographed by the camera (13, 33); a superimposition position calculation step (S105, S205, S305, S405) of calculating a display position of a required accuracy range representing the required accuracy of the camera corresponding to the height of the landmark as a display position on the image captured by the camera, based on position information of the confirmation target indicating the relative positional relationship of the confirmation target with respect to the camera and the required accuracy of the camera determined according to the distance to a position on the ground that is an extension of a straight line connecting the camera and the landmark; and a superimposed image creation step (S106 to S108, S206 to S208, S210, S306 to S308, S406 to S408) for creating a superimposed image indicating the display position of the required accuracy range and displaying a confirmation image in which the superimposed image is superimposed on the captured image on a display device (14, 34).

9. A calibration confirmation device (20, 40) that displays a confirmation image on a display device (14, 34) for confirming a calibration result of a camera (13, 33) mounted on a vehicle (10, 30), an image acquisition unit (24, 44) for acquiring a photographed image of a state in which a confirmation target (2) having one or more marks at a predetermined height is arranged around the vehicle, the image being photographed by the camera; a superimposition position calculation unit (25, 45) that calculates, as a display position on the image captured by the camera, a display position of a required accuracy range that indicates the required accuracy of the camera corresponding to the height of the landmark, based on position information of the confirmation target that indicates the relative positional relationship of the confirmation target with respect to the camera and the required accuracy of the camera that is determined according to the distance to a position on the ground that is an extension of a straight line connecting the camera and the landmark; A calibration confirmation device comprising: a superimposed image creation unit (26, 46) that creates a superimposed image indicating the display position of the required accuracy range and displays a confirmation image on a display device in which the superimposed image is superimposed on the captured image.

Images