In-vehicle camera calibration system, in-vehicle camera calibration method, and in-vehicle camera calibration program

Abstract

To provide an on-vehicle camera calibration system capable of correcting positional displacement of a photometric / colorimetric area of an on-vehicle camera, an on-vehicle camera calibration method, and an on-vehicle camera calibration program.SOLUTION: Regarding a pair of left and right markers 120 and 120R for optical axis adjustments installed in such a manner that a line segment connecting both the markers is in parallel with a horizon line in a state where an on-vehicle camera is assembled to a vehicle, in which a photometric / colorimetric area 110 is previously provided for adjusting exposure and a color tone of a picked-up image into a predetermined range around a photometric / colorimetric area H axis 114 passing a photometric / colorimetric area center point 112 matched to an optical axis in an imaging range and being theoretically in parallel with a horizon line 100 in the imaging range, a camera ECU matches the photometric / colorimetric area center point 112 to a segmentation center point 122 in an image for calibration which is picked up in such a manner that an intermediate point of the line segment is substantially matched to a center of the picked-up image of the on-vehicle camera, and matches the photometric / colorimetric area H axis 114 to a segmentation H axis 124 in the image for calibration.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an in-vehicle camera calibration system, an in-vehicle camera calibration method, and an in-vehicle camera calibration program for calibrating an error (assembly variation) in the assembly position of an in-vehicle camera to a vehicle.

Background Art

[0002] Patent Document 1 describes a technique for correcting the distortion of the optical axis of a cut-out image displayed on a display based on an optical axis adjustment marker in a calibration process for assembly variations generated in the process of assembling an in-vehicle camera to a vehicle.

[0003] FIG. 6(A) shows the working ranges at the camera factory and the vehicle assembly factory, and FIG. 6(B) is an example of a through image when the in-vehicle camera images infinity at the camera factory. In the upper right and upper left of the through image shown in FIG. 6(B), the garnish ridge line 204 of the vehicle is shown, and in the lower part of the through image, the bumper ridge line 202 of the vehicle is reflected. A photometric and colorimetric area 210 is provided at the center of the through image. Each of the upper side and the lower side of the photometric and colorimetric area 210 is set to be parallel to the actual horizon 100.

[0004] FIG. 6(C) is an image of the optical axis adjustment markers 120L and 120R installed on the in-vehicle camera assembled to the vehicle such that the line segment actually connecting the optical axis adjustment marker 220L and the optical axis adjustment marker 220R is parallel to the actual horizon. On the image of FIG. 6(C), the photometric and colorimetric area 210 is inclined counterclockwise with respect to the optical axis adjustment markers 220L and 220R, indicating that assembly variations have occurred.

[0005] FIG. 6(D) is an explanatory diagram showing an example of the cut-out range 232 of the image after optical axis adjustment. The cut-out range 232 is calibrated such that the upper side or the lower side of the cut-out range 232 is parallel to the actual horizon based on the positions of the optical axis adjustment markers 220L and 220R on the image.

Prior Art Documents

Patent Document

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, in the technology described in Patent Document 1, since the position of the photometric and colorimetric area 210 provided around the optical axis of the in-vehicle camera is not calibrated in order to adjust the exposure and color tone of the image acquired by the in-vehicle camera, there is room for improvement in adjusting the exposure and color tone of the image acquired by the in-vehicle camera.

[0008] The present invention has been made in consideration of the above facts, and an object thereof is to obtain an in-vehicle camera calibration system, an in-vehicle camera calibration method, and an in-vehicle camera calibration program that can correct the positional deviation of the photometric and colorimetric area of the in-vehicle camera.

Means for Solving the Problems

[0009] In order to solve the above problems, the in-vehicle camera calibration system according to claim 1 includes a processor, and the processor sets a reference point and a reference axis that are reference for the image based on an image including a marker arranged so as to be at a specific position with respect to the horizon imaged by the in-vehicle camera, and sets a temporary coordinate point and a temporary coordinate axis that are preset for a photometric and colorimetric area for adjusting at least one of the exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis.

[0010] According to the in-vehicle camera calibration system according to claim 1, by setting a reference point and a reference axis that are reference for the image, and adjusting a temporary coordinate point and a temporary coordinate axis that are preset for a photometric and colorimetric area for adjusting at least one of the exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis, the positional deviation of the photometric and colorimetric area of the in-vehicle camera can be corrected.

[0011] In the in-vehicle camera calibration system according to claim 2, the processor sets a cut-out range indicating the range of an image displayed on a display device of a vehicle as a rectangular range that includes the adjusted photometric and colorimetric area centered on the center point of the reference axis and has sides parallel and perpendicular to the reference axis.

[0012] According to the in-vehicle camera calibration system according to claim 2, a cut-out range indicating the range of an image displayed on a display device of a vehicle can be set as a rectangular range that has sides parallel and perpendicular to the reference axis in the image and is centered on the center point of the reference axis in the image.

[0013] In the in-vehicle camera calibration system according to claim 3, when the upper end of the photometric and colorimetric area before adjustment in the image exists above the horizon in the image, the processor calibrates the position of the photometric and colorimetric area.

[0014] According to the in-vehicle camera calibration system according to claim 3, as long as the upper end of the photometric and colorimetric area before position calibration exists above the horizon in the used image, the position of the photometric and colorimetric area can be calibrated.

[0015] In order to solve the above problems, the in-vehicle camera calibration method according to claim 4 sets a reference point and a reference axis serving as references for the image based on an image including a marker arranged at a specific position with respect to the horizon imaged by the in-vehicle camera, and adjusts a temporarily set coordinate point and a temporarily set coordinate axis for a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis.

[0016] According to the in-vehicle camera calibration method according to claim 4, by setting a reference point and a reference axis serving as references for the image and adjusting a temporarily set coordinate point and a temporarily set coordinate axis for a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis, it is possible to correct the displacement of the photometric and colorimetric area of the in-vehicle camera.

[0017] To solve the above problems, the in-vehicle camera calibration program according to claim 5 causes a processor to set a reference point and a reference axis for the image based on an image including a marker arranged at a specific position with respect to the horizon imaged by the in-vehicle camera, and to adjust a temporarily set coordinate point and a temporarily set coordinate axis for a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so that they have a predetermined relationship with the reference point and the reference axis.

[0018] According to the in-vehicle camera calibration program according to claim 5, by setting a reference point and a reference axis for the image and adjusting a temporarily set coordinate point and a temporarily set coordinate axis for a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so that they have a predetermined relationship with the reference point and the reference axis, it is possible to correct the positional deviation of the photometric and colorimetric area of the in-vehicle camera.

Effects of the Invention

[0019] The present invention has an effect that an in-vehicle camera calibration system, an in-vehicle camera calibration method, and an in-vehicle camera calibration program capable of correcting the positional deviation of the photometric and colorimetric area of the in-vehicle camera can be obtained.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiment for Carrying out the Invention

[0021] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. The in-vehicle camera calibration system 10 shown in FIG. 1 includes one in-vehicle camera 12, a camera ECU (Electronic Control Unit) 14, and a display 16, and the in-vehicle camera 12 and the display 16 are connected to the camera ECU 14. The in-vehicle camera 12 has a wide-angle imaging range and is attached to the vehicle so as to be able to image, for example, the front of the vehicle. The in-vehicle camera 12 is an example of an imaging unit.

[0022] The display 16 is provided at a position spaced from the center of the vehicle's instrument panel upward.

[0023] The camera ECU 14 includes a CPU (Central Processing Unit) 22, a memory 24 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), a non-volatile storage unit 26 such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and a communication unit 28. The CPU 22, the memory 24, the storage unit 26, and the communication unit 28 are communicably connected to each other via an internal bus 30.

[0024] The storage unit 26 of the camera ECU 14 stores a display program 32 and a calibration program 34. The CPU 22 of the camera ECU 14 executes the display program 32 read from the storage unit 26 and expanded in the memory 24, thereby appropriately performing image processing on the image data acquired by the in-vehicle camera 12 and displaying it on the display. Also, the CPU 32 executes the calibration program 34 expanded in the memory 24, thereby functioning as an optical axis adjustment unit 204 that calibrates the position of the photometric and colorimetric area and a cutout range setting unit 206 that sets the cutout range, as shown in FIG. 2, and performs the calibration process of the in-vehicle camera 12 described later. Note that the camera ECU 14 is an example of the in-vehicle camera calibration system 10.

[0025] The display program 32 causes the display 16 to display the captured image captured by the in-vehicle camera 12. As described above, since the imaging range of the in-vehicle camera 12 is wide-angle, the acquired image data may include distortion aberration peculiar to a wide-angle lens. In such a case, an image in which the distortion aberration is corrected is displayed on the display 16 by the image processing by the display program 32.

[0026] FIG. 3(A) is a flowchart showing an example of the setting process of the in-vehicle camera 12 at the camera factory. In step S100, a default photometric and colorimetric area common to in-vehicle cameras as products is set.

[0027] FIG. 4(A) shows the working ranges in the camera factory and the vehicle assembly factory, and FIG. 4(B) is an example of a through-image showing the imaging range of the in-vehicle camera 12 when the in-vehicle camera 12 images infinity in the camera factory. The imaging range of an imaging device having a photographing lens, not limited to the in-vehicle camera 12, exhibits a substantially circular shape (image circle), but in this embodiment, for convenience, the imaging range is represented as a rectangular through-image.

[0028] In the upper right and upper left of the through-image shown in FIG. 4(B), the garnish ridge line 104 of the vehicle is reflected, and in the lower part of the through-image, the bumper ridge line 102 of the vehicle is reflected. At the center of the through-image that coincides with the optical axis of the in-vehicle camera 12, there is a photometric and colorimetric area center point 112, and a photometric and colorimetric area 110 on the rectangle is provided around the photometric and colorimetric area H-axis 114 passing through the photometric and colorimetric area center point 112. The photometric and colorimetric area H-axis 114 is theoretically parallel to the horizon 100. However, when the horizon exhibits a convex-upward or convex-downward curved shape in the through-image due to aberrations such as distortion aberration of the lens of the in-vehicle camera 12, the tangent line 108 at the vertex 106 of the curve is made parallel to the photometric and colorimetric area H-axis 114.

[0029] The photometric and colorimetric area 110 is set in a rectangular shape with a predetermined size having sides parallel and perpendicular to the photometric and colorimetric area H-axis 114, centered on the photometric and colorimetric area center point 112. In step S100, the set photometric and colorimetric area 110 and the photometric and colorimetric area center point 112 are stored in the storage unit of the in-vehicle camera 12.

[0030] In step S102, the in-vehicle camera 12 for which the setting is completed is shipped, and the processing in the camera factory is terminated.

[0031] FIG. 3(B) is a flowchart showing an example of the assembly of the in-vehicle camera 12 to the vehicle and the calibration process of the in-vehicle camera 12 in the vehicle assembly factory. In step S200, the in-vehicle camera 12 is assembled to the vehicle. When assembling in step S200, there may be an assembly variation, which is an error in the assembly position of the in-vehicle camera 12 to the vehicle.

[0032] FIG. 4(C) is an explanatory diagram showing an example of assembly variations that occur during vehicle assembly. FIG. 4(C) shows the in-vehicle camera 12 in a state of being assembled to the vehicle. The calibration image is captured such that the line segment actually connecting the optical axis adjustment marker 120L and the optical axis adjustment marker 120R is parallel to the actual horizon, and the midpoint of the line segment substantially coincides with the center of the image captured by the in-vehicle camera 12. On the image of FIG. 4(C), the photometric and colorimetric area H-axis 114 is inclined counterclockwise with respect to the cutout H-axis 124 that connects the optical axis adjustment marker 120L and the optical axis adjustment marker 120R and passes through the cutout center point 122, indicating that assembly variations have occurred.

[0033] In step S202, the cutout range 132 of the through-image of the in-vehicle camera 12 is determined by recognizing the optical axis adjustment marker. The cutout range 132 indicates the range of the cutout image displayed on the display 16. FIG. 4(D) is an explanatory diagram showing an example of the cutout range 132 of the image after optical axis adjustment. The cutout range 132 is set to a rectangular shape of a predetermined size centered on the cutout center point 122 and having sides parallel and perpendicular to the cutout H-axis 124.

[0034] In step S204, the coordinates of the cutout center point 122 on the image and the coordinates of the cutout H-axis 124 on the image are stored in the memory 24 of the camera ECU 14. If direct writing is possible to the storage unit of the in-vehicle camera 12, the coordinates of the cutout center point 122 on the image and the coordinates of the cutout H-axis 124 on the image may be stored in the storage unit of the in-vehicle camera 12.

[0035] In step S206, adjustment is made so that the cutout center point 122 and the photometric and colorimetric area center point 112 coincide or have a unique positional relationship in the same coordinate system. Specifically, as indicated by the arrow 126, the photometric and colorimetric area center point 112 is moved to the cutout center point 122 so that the cutout center point 122 and the photometric and colorimetric area center point 112 coincide or have a unique positional relationship in the same coordinate system.

[0036] In step S208, with the center point 112 of the adjusted photometric and colorimetric area as the center, the H-axis 114 of the photometric and colorimetric area is adjusted to be parallel to the cutout H-axis 124. Specifically, as shown by the arrow 128, the H-axis 114 of the photometric and colorimetric area is moved close to the cutout H-axis 124 so that the H-axis 114 of the photometric and colorimetric area is adjusted to be parallel to or coincide with the cutout H-axis 124, and the cutout range 132 is set to have a size within a predetermined range that includes the photometric and colorimetric area 130 after position correction.

[0037] In step S210, the coordinates of the adjusted photometric and colorimetric area 130 on the image and the coordinates of the set cutout range 132 on the image are stored in the memory 24 of the camera ECU 14, and the process ends. If it is possible to directly write to the storage unit of the in-vehicle camera 12, the coordinates of the photometric and colorimetric area 130 on the image and the coordinates of the set cutout range 132 on the image may be stored in the storage unit of the in-vehicle camera 12.

[0038] FIG. 5(A) is an explanatory diagram of the maximum assembly variation that can be calibrated in the in-vehicle camera calibration system 10 according to the present embodiment. In the present embodiment, if the upper side of the photometric and colorimetric area 110 before calibration is above the horizon 100, the photometric and colorimetric area 110 can be calibrated to the photometric and colorimetric area 130. When the upper side (upper end) of the photometric and colorimetric area 110 before calibration is above the horizon 100, the distance L between the upper side of the photometric and colorimetric area 110 and the horizon 100 is a positive value. In the present embodiment, the distance L when the upper side of the photometric and colorimetric area 110 is above the horizon 100 is set as a positive value, and the distance L when the upper side of the photometric and colorimetric area 110 is below the horizon 100 is set as a negative value.

[0039] FIG. 5(B) is a schematic diagram showing the photometric and colorimetric area 210 when only the cutout range 232 is calibrated according to the markers 220L and 220R. In the case shown in FIG. 5(B), since the assembly variation of the photometric and colorimetric area 210 remains as it is, the photometric and colorimetric area 210 is displaced with respect to the calibrated cutout range 232, so there may be difficulties in adjusting the exposure and color tone of the in-vehicle camera 12.

[0040] As described above, in the present embodiment, by calibrating the photometric and colorimetric area 130 together with the cut-out range 132, it is possible to correct the displacement of the photometric and colorimetric area of the in-vehicle camera as well as the distortion of the optical axis of the cut-out image.

[0041] In recent years, in order to reduce costs, it has become common to implement the in-vehicle camera 12 common to multiple vehicle models. Furthermore, brackets and the like used when assembling the in-vehicle camera 12 to the vehicle also tend to be made common among multiple vehicle models.

[0042] When not only the in-vehicle camera 12 but also the brackets are made common among multiple vehicle models, the assembly variation of the in-vehicle camera 12 is more likely to occur than when using dedicated brackets for each vehicle model. However, according to the in-vehicle camera calibration system 10 according to the present embodiment, even when implementing the in-vehicle camera 12 and brackets common to multiple vehicle models, it is possible to appropriately calibrate the generated assembly variation.

[0043] In the present embodiment, the H-axis that is horizontal with respect to the horizon is used as the reference axis, but it is not limited to this. The cut-out range 132 may be set and the photometric and colorimetric area 130 may be calibrated using the V-axis that is perpendicular to the horizon as the reference axis.

[0044] Note that the in-vehicle camera calibration system 10 according to the above embodiment is also applicable to vehicles capable of autonomous driving.

Explanation of Reference Numerals

[0045] 10 In-vehicle camera calibration system 12 In-vehicle camera 14 Camera ECU 16 Display 22 CPU 24 Memory 26 Storage unit 28 Communication unit 32 Display program 34 Calibration program 100 Horizon 110 Photometric and Colorimetric Area 112 Center Point of Photometric and Colorimetric Area 114 H Axis of Photometric and Colorimetric Area 120 Marker for Optical Axis Adjustment 120L Marker for Optical Axis Adjustment 120R Marker for Optical Axis Adjustment 122 Cut-Out Center Point 124 Cut-Out H Axis 130 Photometric and Colorimetric Area 132 Cut-Out Range 204 Optical Axis Adjustment Unit 206 Cut-Out Range Setting Unit L Distance

Claims

1. Comprising a processor, the processor sets a reference point and a reference axis of the image based on an image including a marker arranged to be at a specific position with respect to the horizon imaged by an in-vehicle camera, and is configured to adjust a preset temporary coordinate point and a temporary coordinate axis with respect to a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis. An in-vehicle camera calibration system.

2. The processor sets a cut-out range indicating the range of the image displayed on the display device of the vehicle as a rectangular range including the adjusted photometric and colorimetric area centered on the center point of the reference axis and having sides parallel and perpendicular to the reference axis. The in-vehicle camera calibration system according to claim 1.

3. The processor calibrates the position of the photometric and colorimetric area when the upper end of the photometric and colorimetric area before adjustment in the image exists above the horizon in the image. The in-vehicle camera calibration system according to claim 1 or 2.

4. Based on an image including a marker arranged to be at a specific position with respect to the horizon imaged by an in-vehicle camera, a reference point and a reference axis serving as a reference for the image are set, An in-vehicle camera calibration method for adjusting a preset temporary coordinate point and a temporary coordinate axis with respect to a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis.

5. A processor, based on an image including a marker arranged to be at a specific position with respect to the horizon imaged by an in-vehicle camera, sets a reference point and a reference axis serving as a reference for the image, and an in-vehicle camera calibration program that operates to adjust a preset temporary coordinate point and a temporary coordinate axis with respect to a photometric and colorimetric area for adjusting at least one of exposure and color tone in the in-vehicle camera so as to have a predetermined relationship with the reference point and the reference axis.

Images