Camera calibration method and target display

Abstract

When calibrating a camera that views a target image from an oblique angle, this makes it easier to recognize the feature points of the target image within the camera image that represent the target image. [Solution] The target image 30 is formed such that the aspect ratio of the target image representing the target image 30 in the camera image is closer to 1 than the aspect ratio of the target image 30. Therefore, feature points such as intersections included in the target image are less likely to be distorted in the camera image. As a result, it is possible to make it easier to recognize the feature points of the target image when performing camera calibration.

Description

Technical Field

[0001] The present disclosure relates to a camera calibration method and a target display body.

Background Art

[0002] Patent Document 1 describes a calibration method for an in-vehicle camera. The in-vehicle camera of Patent Document 1 is disposed on the vehicle body such that the optical axis of the camera intersects the ground at a desired position a predetermined distance away from the vehicle body. The calibration target is drawn on the ground. Accordingly, the in-vehicle camera views the target drawn on the ground from an obliquely upper side.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, the in-vehicle camera of Patent Document 1 needs to view the target drawn on the ground from an oblique direction. In this way, when the camera needs to view the actual image of the target from an oblique direction during camera calibration, feature points such as intersections included in the captured target image, which is the target displayed in the camera image, may be distorted and become difficult to recognize.

[0005] This phenomenon becomes more pronounced, for example, as the camera resolution decreases, and as the angle between the camera's optical axis and the display surface (e.g., the ground) on which the target's actual image is displayed decreases. For example, in the case of the in-vehicle camera described in Patent Document 1, the lower the mounting position of the in-vehicle camera on the vehicle, the smaller the angle between the camera's optical axis and the ground (the display surface) becomes, making it easier for feature points such as intersections to be distorted in the camera image. The inventors have found the above to be true as a result of their detailed investigation.

[0006] In view of the above points, this disclosure aims to provide a camera calibration method and a target display unit that make it easier to recognize feature points of the target image when calibrating a camera that views the target image from an oblique angle. [Means for solving the problem]

[0007] To achieve the above objective, one aspect of the camera calibration method of this disclosure is: A calibration method for performing calibration of cameras (201, 202, 203, 204), The process involves capturing (S201) a target image (30, 31) displayed on a display surface (802) tilted with respect to the camera's optical axis (201a, 202a, 203a, 204a) using the camera, and This includes generating calibration data (Dc) used to calibrate the camera based on target imaging data obtained by capturing images with the camera (S202~S206), In a camera image (Pc) based on target acquisition data, the target image (Pct) representing the actual target image is formed such that the maximum value of the maximum vertical dimension (Dvp) (Wvp) and the maximum horizontal dimension (Dhp) of the target image (Pct) is defined as the first image dimension (W1p), and the minimum value is defined as the second image dimension (W2p). Furthermore, the target image is formed such that the ratio (Rpc) of the first image dimension to the second image dimension is closer to 1 than the ratio (Rr) of the first image dimension to the second image dimension. The vertical orientation is the direction aligned with the display surface, and is the vertical orientation within the camera image. The horizontal orientation is the direction aligned with the display surface, which corresponds to the horizontal direction within the camera image.

[0008] This method prevents feature points, such as intersections, contained in the target image from being lost within the camera image. As a result, it becomes easier to recognize the feature points of the target image when performing camera calibration.

[0009] Furthermore, a target representation from another perspective of this disclosure is: A target display unit used for calibrating cameras (201-204), The camera displays the actual target images (30, 31) captured by the camera, and the display surface (802) is tilted with respect to the camera's optical axis (201a, 202a, 203a, 204a). When the maximum vertical dimension (Wvp) and the maximum horizontal dimension (Dhp) of the target image (Pct) representing the target real image within the camera image (Pc) captured by the camera are defined as the first image dimension (W1p) and the minimum as the second image dimension (W2p), and the maximum vertical dimension (Wvr) and the maximum horizontal dimension (Dhr) of the target image (W1r) and the minimum as the second image dimension (W2r) of the target real image are defined as the first real image dimension (W1r) and the minimum as the second real image dimension (W2r), the target real image is formed such that the ratio (Rpc) of the first image dimension to the second image dimension is closer to 1 than the ratio (Rr) of the first real image dimension to the second real image dimension. The vertical orientation is the direction aligned with the display surface, and is the vertical orientation within the camera image. The horizontal orientation is the direction aligned with the display surface, which corresponds to the horizontal direction within the camera image.

[0010] In this way, similar to the camera calibration method described above, feature points such as intersections included in the target image are less likely to be distorted within the camera image. As a result, it becomes easier to recognize the feature points of the target image when performing camera calibration.

[0011] In addition, each element in the application documents may be given a reference numeral in parentheses. In this case, the reference numeral merely indicates one example of the correspondence between the element and the specific configuration described in the embodiments described later. Therefore, this disclosure is not limited in any way by the inclusion of such reference numerals. [Brief explanation of the drawing]

[0012] [Figure 1] In the first embodiment, this is a schematic plan view showing the location on the vehicle's manufacturing line where the calibration of the multiple cameras on the vehicle is performed, and its surrounding area. [Figure 2]This is a plan view showing an excerpt of the area in Figure 1 where multiple cameras are calibrated; in other words, it is an excerpt of section II of Figure 1. [Figure 3] This diagram schematically shows an excerpt of portion IIIa from the view in direction III of Figure 2, illustrating the positional relationship between the first camera among multiple cameras and the target image located directly in front of the first camera. [Figure 4] This is a block diagram showing a plurality of cameras and a control device to which they are electrically connected in the first embodiment. [Figure 5] This is a flowchart showing the procedure for the calibration process in the vehicle manufacturing process in the first embodiment. [Figure 6] This flowchart shows the control process performed by the control device in Figure 4 at step S103 of the flowchart in Figure 5. [Figure 7] In the first embodiment, this figure shows a target image representing a real target image located in front of the first camera, among the camera images. [Figure 8] In the first embodiment, this figure shows a target image located in front of the first camera, among the multiple target images shown in Figure 2. [Figure 9] This figure shows a target image of a first comparative example to be compared with the first embodiment, and corresponds to Figure 8. [Figure 10] In the first comparative example, the figure shows a target image representing the actual target image of Figure 9 within the camera image, and corresponds to Figure 7. [Figure 11] In the second comparative example, this is a plan view showing an excerpt of the area where the calibration of multiple cameras is performed, that is, an excerpt of the part corresponding to part II of Figure 1, and is the same as Figure 2. [Figure 12] This is a view in the direction of arrow XII in Figure 11 in the second comparative example. [Figure 13]In the second embodiment, this is a plan view extracting and showing the vehicle arranged at the calibration vehicle position on the vehicle production line, and three actual target images arranged on the floor surface on the front side of the vehicle. [Figure 14] In the second embodiment, this is a diagram showing a target captured image representing the actual target image positioned in front of the first camera among the camera images, and corresponds to Figure 7. [Mode for Carrying Out the Invention]

[0013] Hereinafter, each embodiment will be described with reference to the drawings. Note that, in the following respective embodiments, parts that are identical or equivalent to each other are denoted by the same reference numerals in the drawings.

[0014] (First Embodiment) As shown in Figure 1 and Figure 2, in the present embodiment, in the calibration step, a calibration method for calibrating a plurality of cameras 201, 202, 203, 204 (see Figure 4 described later) included in the automobile 10 is implemented. This calibration step is one step in the manufacturing processes of the automobile 10. In the calibration step, as shown in Figure 3, the calibration of the plurality of cameras 201, 202, 203, 204 is performed, for example, in a factory, with the automobile 10 placed stationary on the floor surface 80. As a matter of course, the floor surface 80 in the factory is the upper surface of the factory floor 79, so it faces the upper side in the vertical direction and extends along the horizontal plane.

[0015] In the description of the present embodiment, the automobile 10 may be simply referred to as the vehicle 10. In addition, in Figure 1 and Figure 2, the vehicle front-rear direction D1 that is the front-rear direction of the vehicle 10 and the vehicle left-right direction D2 that is the left-right direction of the vehicle 10 are indicated by double-headed arrows, and in Figure 3, the vehicle front-rear direction D1 and the vehicle up-down direction D3 that is the up-down direction of the vehicle 10 are indicated by double-headed arrows. These directions D1, D2, D3 are directions intersecting each other, strictly speaking, directions perpendicular to each other. The vehicle up-down direction D3 is also the vertical direction in the factory.

[0016] Furthermore, the front side of vehicle 10 is also referred to as the vehicle front, the rear side of vehicle 10 is also referred to as the vehicle rear, the left side of vehicle 10 is also referred to as the vehicle left, and the right side of vehicle 10 is also referred to as the vehicle right. Additionally, the upper side of vehicle 10 is also referred to as the vehicle upper, and the lower side of vehicle 10 is also referred to as the vehicle lower.

[0017] As shown in Figures 2 and 4, the automobile 10 is equipped with first to fourth cameras 201, 202, 203, and 204, and a control device 12 that controls the first to fourth cameras 201, 202, 203, and 204. Note that the first to fourth cameras 201, 202, 203, and 204 may be abbreviated as first to fourth cameras 201 to 204, or cameras 201 to 204.

[0018] The first to fourth cameras 201 to 204 are each electrically connected to the control device 12. Each of the first to fourth cameras 201 to 204 can capture, for example, moving images and still images at a predetermined frame rate. The first to fourth cameras 201 to 204 output video signals, which are electrical signals indicating the captured camera image Pc, to the control device 12. In other words, the control device 12 acquires the camera image Pc from each of the first to fourth cameras 201 to 204 as captured data, for example, digital data. Furthermore, since the first to fourth cameras 201 to 204 are installed on the vehicle 10, they are located above the floor surface 80 inside the factory.

[0019] The first camera 201 is mounted on the vehicle 10 facing the front of the vehicle and captures images of the external conditions in front of the vehicle. Therefore, the optical axis 201a of the first camera 201, i.e., the first camera optical axis 201a, extends from the first camera 201 on the vehicle 10 toward the front of the vehicle.

[0020] The second camera 202 is mounted on the vehicle 10 facing the rear of the vehicle and captures images of the area outside the vehicle at the rear. Therefore, the optical axis 202a of the second camera 202, i.e., the second camera optical axis 202a, extends from the second camera 202 on the vehicle 10 toward the rear of the vehicle.

[0021] The third camera 203 is mounted on the vehicle 10 facing the left side of the vehicle and captures images of the exterior on the left side of the vehicle. Therefore, the optical axis 203a of the third camera 203, i.e., the optical axis 203a of the third camera 203, extends from the third camera 203 on the vehicle 10 to the left side of the vehicle.

[0022] The fourth camera 204 is mounted on the vehicle 10 facing the right side of the vehicle and captures images of the exterior on the right side of the vehicle. Therefore, the optical axis 204a of the fourth camera 204, i.e., the optical axis 204a of the fourth camera 204, extends from the fourth camera 204 on the vehicle 10 to the right side of the vehicle.

[0023] More specifically, as shown in Figure 3, the first camera 201 is positioned diagonally downward rather than horizontally. Similarly, the second to fourth cameras 202 to 204 are also positioned diagonally downward. In other words, the first camera optical axis 201a extends diagonally downward from the first camera 201, the second camera optical axis 202a extends diagonally downward from the second camera 202, the third camera optical axis 203a extends diagonally downward from the third camera 203, and the fourth camera optical axis 204a extends diagonally downward from the fourth camera 204. Consequently, the floor surface 80 extends along a plane (specifically, a horizontal plane) that is inclined with respect to each of the first to fourth camera optical axes 201a, 202a, 203a, and 204a.

[0024] The control device 12 is a so-called electronic control device and has the configuration of a microcomputer equipped with a CPU, RAM, ROM, and non-volatile rewritable memory (not shown). The control device 12 reads and executes a computer program stored in the ROM or non-volatile rewritable memory, which are non-transitional physical recording media. When this computer program is executed, the method corresponding to the computer program is performed. That is, the control device 12 performs various control processes, such as the control process shown in Figure 6, which will be described later, according to the computer program.

[0025] For example, the control device 12 generates video information based on video signals obtained from the first to fourth cameras 201 to 204, and outputs a signal indicating the generated video information to other electronic control devices in the vehicle 10. For example, the video information output by the control device 12 is used for automatic brake control, which automatically activates the brakes of the vehicle 10, and for display on a display device installed in the passenger compartment. The control device 12 also performs electronic calibration of each of the cameras 201 to 204, which will be described later.

[0026] Next, the manufacturing process of the vehicle 10, including the calibration process described above, will be explained. As shown in Figure 1, the vehicle 10 is moved along the vehicle 10 manufacturing line 82 in the factory as the manufacturing process progresses.

[0027] Furthermore, as shown in Figure 2, the calibration process uses multiple target images 30, i.e., target real images 30, displayed on the factory floor 80. Targets are also called markers. Specifically, a total of eight target real images 30 are used in this embodiment.

[0028] The eight target images 30 in Figure 2 may be drawn on the floor surface 80, or drawn on a thin sheet or paper attached to the floor surface 80, but in this embodiment, they will be described as being drawn on the floor surface 80. Furthermore, the target images 30 and the target images Pct obtained by photographing the target images 30 with cameras 201-204, described later, include solid black areas, but these solid black areas are shown as areas with dot-like hatching.

[0029] As described above, the target image 30 is drawn on the floor surface 80, so the portion of the floor surface 80 on which the target image 30 is drawn becomes the display surface 802 on which the target image 30 captured by cameras 201 to 204 is displayed. The portion of the factory floor 79 on which the display surface 802 of the target image 30 is formed functions as a target display body used for the calibration of cameras 201 to 204. Since the target image 30 is a figure drawn on the floor surface 80, it has no thickness, but in Figure 3, for the sake of easier understanding of the illustration, the target image 30 located in front of the first camera 201 is shown with a dashed line.

[0030] Furthermore, if the target image 30 is displayed not on the floor surface 80, but on, for example, the sheet or paper attached to the floor surface 80, then that sheet or paper functions as the target display body.

[0031] Figure 5 is a flowchart showing the procedure for the calibration process in the manufacturing process of vehicle 10, and the calibration process proceeds according to Figure 5.

[0032] Specifically, once the pre-calibration process is completed, in step S101, the vehicle 10 is moved as shown by arrow A1 in Figure 1. That is, on the floor surface 80, the vehicle 10 is moved from the vehicle position where the pre-calibration process was performed to a predetermined calibration vehicle position for performing the calibration process.

[0033] The calibration vehicle position is a predetermined fixed position. Therefore, the relative positional relationship between the vehicle 10 positioned at the calibration vehicle position on the floor surface 80 and the multiple target real images 30 provided on the floor surface 80 is kept constant. At this time, as shown in Figure 3, the portion of the floor surface 80 on which the vehicle 10 is positioned and placed as described above becomes the mounting surface 803 of the vehicle 10.

[0034] Then, as shown in Figure 2, when the vehicle 10 is positioned at the calibration vehicle position, the multiple target real images 30 are arranged on the floor surface 80 so as to surround the vehicle 10. Specifically, three target real images 30 are positioned on the front side of the vehicle and three on the rear side of the vehicle, one is positioned in front of the third camera 203 on the left side of the vehicle, and one is positioned in front of the fourth camera 204 on the right side of the vehicle.

[0035] Therefore, the first camera 201 is capable of capturing a total of three target images 30: one target image 30 located directly in front of the first camera 201, and two target images 30 located on either side of it, diagonally in front of the first camera 201. The target image 30 located directly in front of the first camera 201 is positioned in a directional view along the vehicle's vertical direction D3 (for example, the plan view shown as Figure 2) so as to be aligned with the first camera 201 in the axial direction of the first camera optical axis 201a.

[0036] The same applies to the second camera 202. That is, the second camera 202 is also capable of capturing three target images 30. The target images 30 located in front of the second camera 202 are arranged in the plan view above so as to be aligned with the second camera 202 in the direction of the second camera optical axis 202a.

[0037] Furthermore, the third and fourth cameras 203 and 204 are each capable of capturing a target image 30 located directly in front of their respective cameras. The target images 30 located directly in front of the third and fourth cameras 203 and 204 are arranged in the plan view described above so as to be aligned with the optical axis of the camera. When the target image 30 located directly in front of the first camera 201 is described separately from the other target images 30, its reference numeral is changed and it is sometimes simply referred to as the front target image 31. After step S101 in Figure 5, the process proceeds to step S102.

[0038] In step S102, the control unit 12 is set to calibration mode for calibrating each camera 201 to 204. This makes the control unit 12 capable of executing the computer program for calibrating each camera 201 to 204. After step S102, the process proceeds to step S103.

[0039] In step S103, the control device 12 performs calibration of each camera 201 to 204 according to the computer program. Specifically, the control device 12 executes the control process shown in the flowchart of Figure 6. This control process in Figure 6 is executed for each of the four cameras 201 to 204, so it is executed a total of four times in step S103.

[0040] Note that the control process shown in Figure 6, which is performed on the first camera 201, will be explained below, but the explanation of the control process shown in Figure 6, which is performed on the other cameras 202 to 204, will be omitted. This is because the control process shown in Figure 6, which is performed on each of the cameras 201 to 204, is the same.

[0041] In the control process for the first camera 201 shown in Figure 6, first, in step S201, the control device 12 causes the first camera 201 to capture an image. At this time, the first camera 201 captures three target images 30 located on the front side of the vehicle at once, so the camera image Pc captured by the first camera 201 contains three target images Pct representing the target images 30. The control device 12 then acquires the camera image Pc captured by the first camera 201 as capture data from the first camera 201. In this embodiment, the capture data acquired in step S201 represents the camera image Pc including the target images Pct, and is therefore sometimes referred to as target capture data.

[0042] Figure 7 shows the target image Pct representing the target real image 30 (i.e., the front target real image 31) located in front of the first camera 201, from the camera image Pc. The target images Pct representing the two target real images 30 located diagonally in front of the first camera 201, from the camera image Pc, are the same as the target images Pct in Figure 7. After step S201 in Figure 6, proceed to step S202.

[0043] In step S202, the control device 12 determines a target search range for searching for the target image Pct within the camera image Pc, which is a two-dimensional image based on the target acquisition data. The target search range is determined so that all target images Pct included in the camera image Pc fall within that range. In the subsequent step S203, the control device 12 detects all target images Pct included in the camera image Pc within the target search range. For example, the control device 12 detects the target images Pct using template matching. After step S203 in Figure 6, the process proceeds to step S204.

[0044] In step S204, as shown in Figure 7, the control device 12 extracts multiple feature points Ch, such as intersections and corners, from the target image Pct within the camera image Pc. This extraction of feature points Ch is performed for all target image Pct detected in step S203. Following step S204 in Figure 6, the process proceeds to step S205.

[0045] In step S205, the control device 12 first calculates the two-dimensional coordinate position X2 of the feature point Ch extracted in step S204 in the camera image Pc. This calculation of the two-dimensional coordinate position X2 of the feature point Ch is performed for all feature points Ch extracted in step S204.

[0046] Next, the control device 12 uses known parameters to transform the two-dimensional coordinate positions X2 of all feature points Ch to their transformed positions X3, which are the coordinate positions in the world coordinate system. This world coordinate system is a three-dimensional coordinate system in real space. After step S205 in Figure 6, the process proceeds to step S206.

[0047] In step S206, the control device 12 estimates parameters for calibration data Dc, which correct the transformed position X3 of all feature points Ch in the world coordinate system, so that the transformed position X3 of each feature point Ch becomes the ideal position Xp of that feature point Ch, through an optimization process. The ideal position Xp of a feature point Ch is the coordinate position in which the feature point Ch is originally located in the world coordinate system.

[0048] The calibration data Dc generated by the estimation in step S206 is stored in the control device 12 and used to electronically calibrate the first camera 201 after the calibration process is completed. In other words, for example, after the calibration process is completed, if the control device 12 acquires shooting data from the first camera 201 showing the camera image Pc captured by the first camera 201, it corrects the acquired shooting data with the calibration data Dc. The control device 12 then outputs the video information based on the corrected shooting data to other electronic control devices connected to the control device 12.

[0049] Furthermore, steps S202 to S206 in Figure 6 above correspond to generating the calibration data Dc based on the target imaging data obtained in step S201.

[0050] Steps S201 to S206 in Figure 6 described above have been performed for all of the first to fourth cameras 202 to 204 and are complete. Once this is finished, step S103 in Figure 5 is completed, and the process in Figure 5 proceeds from step S103 to step S104.

[0051] In step S104, the calibration mode of the control device 12, which was set in step S102, is deactivated. Then, the vehicle 10 is moved as shown by arrow A2 in Figure 1. That is, the vehicle 10 is moved from the predetermined calibration vehicle position described above to a vehicle position for performing the post-calibration process.

[0052] Next, the target image 30 used in this embodiment will be described. Figure 8 shows the front target image 31, which is one of the multiple target images 30 shown in Figure 2.

[0053] As shown in Figures 2 and 8, the multiple target images 30, including the front target image 31, are each two-dimensional figures. More specifically, each of the multiple target images 30 has a rectangular shape 30a that forms the outline of the target image 30, and two straight lines 30b and 30c that are located inside the rectangular shape 30a and intersect each other.

[0054] Furthermore, the first straight line 30b, one of the two straight lines 30b and 30c, extends to connect a pair of opposing sides of the quadrilateral shape 30a. The second straight line 30c, the other of the two straight lines 30b and 30c, extends to connect another pair of opposing sides of the quadrilateral shape 30a.

[0055] As a result, the inside of the quadrilateral shape 30a is divided into four regions 30e to 30h by the first and second lines 30b and 30c. Of these four regions 30e to 30h, the pair of solid black regions 30e and 30f that are separated by the intersection point 30i of the first and second lines 30b and 30c are filled in black, while the other pair of solid white regions 30g and 30h that are separated by the intersection point 30i are filled in white.

[0056] Furthermore, each of the multiple target real images 30 has a pair of obtuse first vertex angles G1r and another pair of acute second vertex angles G2r, separated by the intersection point 30i, through the intersecting first and second lines 30b and 30c.

[0057] Furthermore, as shown in Figures 7 and 8, each of the multiple target real images 30 is formed in relation to the target image Pct as follows. That is, as shown in the following equation F1, each of the multiple target real images 30 is formed such that the aspect ratio Rpc of the target image Pct representing the target real image 30 is closer to 1 than the aspect ratio Rr of the target real image 30. Rr-1>Rpc-1 ···(F1)

[0058] Here, in the above equation F1, the aspect ratio Rpc of the target image Pct is the ratio of the first image dimension W1p of the target image Pct to the second image dimension W2p of the target image Pct, as shown in the following equation F2. Rpc=W1p / W2p ···(F2) However, the first image dimension W1p in the above formula F2 is the maximum value of the maximum vertical dimension Dvp Wvp and the maximum horizontal dimension Dhp Whp of the target image Pct. The second image dimension W2p is the minimum value of the maximum vertical dimension Dvp Wvp and the maximum horizontal dimension Dhp Whp of the target image Pct. For this reason, for example, the aspect ratio Rpc in the above formula F2 will be 1 or greater. Also, if the maximum dimensions Wvp and Whp of the target image Pct are the same, then the first image dimension W1p and the second image dimension W2p will also be the same, and the aspect ratio Rpc of the target image Pct will be "Rpc=1".

[0059] Furthermore, in the above formula F1, the aspect ratio Rr of the target real image 30 is the ratio of the first real image dimension W1r of the target real image 30 to the second real image dimension W2r, as shown in the following formula F3. Rr = W1r / W2r ... (F3) However, the first real image dimension W1r in the above formula F3 is the maximum value of the maximum dimension Wvr in the vertical direction Dvr of the target real image 30 and the maximum dimension Whr in the horizontal direction Dhr of the target real image 30. The second real image dimension W2r is the minimum value of the maximum dimension Wvr in the vertical direction Dvr of the target real image 30 and the maximum dimension Whr in the horizontal direction Dhr of the target real image 30. The vertical direction Dvr of the target real image 30 is the direction along the floor surface 80 and corresponds to the vertical direction Dvp in the camera image Pc, and the horizontal direction Dhr is the direction along the floor surface 80 and corresponds to the horizontal direction Dhp in the camera image Pc. For example, from the above definition, the aspect ratio Rr in the above formula F3 is 1 or greater. Furthermore, if the maximum dimensions Wvr and Whr of the target image 30 are the same, then the first image dimension W1r and the second image dimension W2r will also be the same, and the aspect ratio Rr of the target image 30 will be "Rr=1". In this embodiment, as shown in Figure 8, the relationship between the maximum dimension Wvr in the vertical direction Dvr and the maximum dimension Whr in the horizontal direction Dhr of the target image 30 is "Wvr>Whr", so the first and second image dimensions W1r and W2r are "W1r=Wvr" and "W2r=Whr", respectively.

[0060] Furthermore, between the target real image 30 and the target image Pct representing the target real image 30, the angular difference ΔGp between the first shooting angle G1p and the second shooting angle G2p shown in Figure 7 is smaller than the angular difference ΔGr between the first vertical angle G1r and the second vertical angle G2r shown in Figure 8. In short, the relationship between the angular differences ΔGp and ΔGr is "ΔGp < ΔGr".

[0061] The first shooting angle G1p is the angle corresponding to the first vertical angle G1r in the camera image Pc, or in other words, the angle that shows the first vertical angle G1r in the camera image Pc. The second shooting angle G2p is the angle corresponding to the second vertical angle G2r in the camera image Pc, or in other words, the angle that shows the second vertical angle G2r in the camera image Pc. Furthermore, the angle differences ΔGp and ΔGr can be expressed by the following equations F4 and F5, respectively. ΔGp=│G1p-G2p│ ···(F4) ΔGr=│G1r-G2r│ ···(F5)

[0062] For example, as shown in Figure 7, within the camera image Pc containing the target image Pct, the first imaging line L1p representing the first straight line 30b and the second imaging line L2p representing the second straight line 30c are orthogonal to each target image Pct. In order for the target image Pct within the camera image Pc to be in this state, the first vertical angle G1r and the second vertical angle G2r in Figure 8 are different from each other in all target images 30. As a result, the angle difference ΔGp between the first imaging angle G1p and the second imaging angle G2p in Figure 7 is zero or nearly zero, so the above relationship "ΔGp < ΔGr" holds true.

[0063] For example, in this embodiment, the maximum dimension Wvr in the vertical direction Dvr of the target real image 30 is larger than the maximum dimension Whr in the horizontal direction Dhr. As a result, in the target image Pct representing the target real image 30, each region corresponding to the four regions 30e to 30h of the target real image 30, and the shape corresponding to the rectangular shape 30a that forms the outer shape of the target real image 30, are all square.

[0064] To explain the effects of this embodiment, let's consider a target image 90 of a first comparative example to be compared with this embodiment. The target image 90 of the first comparative example is a figure displayed on the floor surface 80, similar to the target image 30 of this embodiment, but as shown in Figure 9, it has a square-shaped outline and two straight lines arranged inside it that are perpendicular to each other. Furthermore, the target image 90 of the first comparative example has a pair of solid black areas and a pair of solid white areas. In short, the target image 90 of the first comparative example has the same or substantially the same shape as the target image Pct in Figure 7. In other respects, the target image 90 of the first comparative example is the same as the target image 30 of this embodiment.

[0065] Therefore, in the first comparative example, if the target real image 90 is captured by, for example, the first camera 201 that views the target real image 90 at an angle, the target image Pcta representing the target real image 90 in the camera image Pc will be as shown in Figure 10. In other words, in that case, the target image Pcta will have a shape that is compressed in the vertical direction Dvp. As a result, it becomes difficult to recognize the feature points of the target image Pcta when performing camera calibration.

[0066] In contrast, according to this embodiment, as shown in Figures 7 and 8, multiple target images 30 are formed such that the aspect ratio Rpc of the target image Pct representing the target image 30 is closer to 1 than the aspect ratio Rr of the target image 30. Therefore, compared to the first comparative example described above, for example, feature points Ch such as intersections included in the target image Pct are less likely to be distorted within the camera image Pc. As a result, it is possible to make it easier to recognize the feature points Ch of the target image Pct when performing calibration of cameras 201 to 204.

[0067] (1) Furthermore, according to this embodiment, as shown in Figures 7 and 8, the target real image 30 includes a first straight line 30b and a second straight line 30c that intersects the first straight line 30b. The angle difference ΔGr between the first vertical angle G1r and the second vertical angle G2r formed by the first and second straight lines is less than the angle difference ΔGp between the first shooting angle G1p and the second shooting angle G2p in the camera image Pc. The first shooting angle G1p is the angle corresponding to the first vertical angle G1r in the camera image Pc, and the second shooting angle G2p is the angle corresponding to the second vertical angle G2r in the camera image Pc.

[0068] Therefore, compared to the first comparative example described above, for example, it is possible to make it easier to recognize the intersection point of the first imaging line L1p representing the first straight line 30b and the second imaging line L2p representing the second straight line 30c in the camera image Pc as one of the feature points Ch of the target image Pct.

[0069] (2) Furthermore, according to this embodiment, in the target real image 30, the first vertical angle G1r and the second vertical angle G2r are different from each other so that the first imaging line L1p and the second imaging line L2p are orthogonal within the camera image Pc.

[0070] Therefore, compared to, for example, the case where the first imaging line L1p and the second imaging line L2p intersect without being orthogonal in the camera image Pc, it is possible to make the intersection point of the first imaging line L1p and the second imaging line L2p easier to recognize as one of the feature points Ch of the target image Pct.

[0071] (3) Furthermore, according to this embodiment, as shown in Figure 3, the display surface 802 of the target real image 30 extends upward along the horizontal plane, and the first to fourth cameras 201 to 204 are provided on the vehicle 10 and are located above the display surface 802 and are positioned diagonally downward. Then, in step S201 of Figure 6, with the vehicle 10 positioned on the mounting surface 803 that extends along the horizontal plane, multiple target real images 30 are captured by the first to fourth cameras 201 to 204.

[0072] Therefore, since the target image 30 is formed planarly along the floor surface 80, which includes the mounting surface 803 of the vehicle 10, it is possible to install the target image 30 in a way that does not obstruct the movement of the vehicle 10 and the worker.

[0073] For example, as shown in Figures 11 and 12, in the second comparative example, the actual image of the target is displayed on the display surface 94a facing the vehicle 10 on a target board 94 positioned upright on the floor surface 80. In this embodiment, the following can be said. That is, in the second comparative example, after moving the vehicle 10 from the vehicle position where the previous process was performed to a predetermined calibration vehicle position for performing the calibration process, it is necessary to place the target board 94 around the vehicle 10. Then, when moving the vehicle 10 from the calibration vehicle position after the calibration process is completed, it is necessary to remove the target board 94 that was placed around the vehicle 10. In contrast, in this embodiment, it is possible to eliminate the trouble that occurs in the second comparative example.

[0074] (Second Embodiment) Next, a second embodiment will be described. In this embodiment, the differences from the first embodiment described above will be mainly explained. Furthermore, parts that are the same as or equivalent to the above embodiment will be omitted or simplified in their description. The same applies to the descriptions of the embodiments described later.

[0075] As shown in Figure 13, in this embodiment, the target image 30 differs from that of the first embodiment. Specifically, the target image 30 in this embodiment has a first rectangular shape 30j that forms the outer shape of the target image 30, and a second rectangular shape 30k that is arranged inside the first rectangular shape 30j.

[0076] Furthermore, the area inside the first rectangular shape 30j and outside the second rectangular shape 30k, that is, the area between the first rectangular shape 30j and the second rectangular shape 30k, is a solid white area 30m. In contrast, the area inside the second rectangular shape 30k is a solid black area 30n.

[0077] In this embodiment as well, the eight target images 30 are arranged in the same configuration as in the first embodiment, surrounding the vehicle 10 positioned at the calibration vehicle location. Figure 13 shows an excerpt of the eight target images 30, specifically those positioned on the front side of the vehicle.

[0078] Furthermore, as shown in Figures 13 and 14, the relationship of equation F1 above holds true in this embodiment as well as in the first embodiment. That is, in this embodiment as well, multiple target images 30 are formed such that the aspect ratio Rpc of the target image Pct representing the target image 30 is closer to 1 than the aspect ratio Rr of the target image 30.

[0079] Furthermore, as shown in Figure 14, both the figure S1p corresponding to the first rectangular shape 30j of the target real image 30 and the figure S2p corresponding to the second rectangular shape 30k of the target image Pct are square in shape.

[0080] Figure 14 shows the target image Pct representing the target real image 30 (i.e., the front target real image 31) located in front of the first camera 201 (see Figure 3) within the camera image Pc. Furthermore, the target images Pct representing the two target real images 30 located diagonally in front of the first camera 201 within the camera image Pc are similar to the target images Pct in Figure 14.

[0081] Except as described above, this embodiment is the same as the first embodiment. In this embodiment, the effects obtained from the configuration common to the first embodiment can be obtained in the same way as in the first embodiment.

[0082] (Other embodiments) (1) In each of the embodiments described above, for example as shown in Figure 8, the target image 30 includes solid black areas 30e and 30f in part of the target image 30, but this is just an example. For example, the target image 30 may not include solid black areas 30e and 30f and may consist of multiple straight lines or curves drawn on a white background.

[0083] (2) In each of the embodiments described above, for example as shown in Figure 3, the display surface 802 of the target real image 30 is a plane along the horizontal plane, but this is just one example. For example, it is also conceivable that the display surface 802 is inclined or upright with respect to the horizontal plane.

[0084] (3) In each of the embodiments described above, the processing of each step shown in the flowchart of Figure 6 is implemented by a computer program, but it may also be implemented by hardware.

[0085] (4) This disclosure is not limited to the embodiments described above and can be implemented in various modified forms. Furthermore, it goes without saying that, in each of the above embodiments, the elements constituting the embodiment are not necessarily essential, except in cases where they are explicitly stated to be particularly essential or where they are clearly considered essential in principle.

[0086] Furthermore, in each of the above embodiments, when numerical values ​​such as the number, numerical values, quantities, or ranges of the components of the embodiment are mentioned, the embodiments are not limited to those specific numbers unless explicitly stated as particularly essential or when they are clearly limited to a specific number in principle. Also, in each of the above embodiments, when the material, shape, positional relationship, etc. of the components are mentioned, the embodiments are not limited to those material, shape, positional relationship, etc. unless explicitly stated or when they are clearly limited to a specific material, shape, positional relationship, etc. in principle.

[0087] Furthermore, the control device 12 and its method described herein may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by a computer program. Alternatively, the control device 12 and its method described herein may be implemented by a dedicated computer provided by configuring a processor by one or more dedicated hardware logic circuits. Alternatively, the control device 12 and its method described herein may be implemented by one or more dedicated computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. The computer program may also be stored as instructions executed by the computer on a computer-readable non-transitional tangible recording medium. [Explanation of symbols]

[0088] 201, 202, 203, 204 Camera Optical axis of cameras 201a, 202a, 203a, and 204a 30 Target Actual Images 802 Display surface PC camera image PCT Target Image Lateral view of the Dhp target image Vertical direction of DVP target image Dhr Target Image Horizontal Correspondence Vertical correspondence of DVR target real image

Claims

1. A calibration method for calibrating cameras (201, 202, 203, 204), S201: The camera captures the target real images (30, 31) displayed on a display surface (802) that is tilted with respect to the optical axis (201a, 202a, 203a, 204a) of the camera. This includes generating calibration data (Dc) used to calibrate the camera based on target imaging data obtained by imaging with the camera (S202 to S206), In the camera image (Pc) based on the target imaging data, the maximum value of the maximum vertical dimension (Wvp) and the maximum horizontal dimension (Whp) of the target imaging image (Pct) representing the target image is defined as the first image dimension (W1p) and the minimum value as the second image dimension (W2p). In the camera image (Pc) based on the target imaging data, the maximum value of the maximum vertical dimension (Dvp) and the maximum horizontal dimension (Whr) of the target imaging image is defined as the first image dimension (W1r) and the minimum value as the second image dimension (W2r). The target image is formed such that the ratio (Rpc) of the first image dimension to the second image dimension is closer to 1 than the ratio (Rr) of the first image dimension to the second image dimension. The aforementioned vertical correspondence direction is the direction along the display surface, and within the camera image, it is the direction that corresponds to the vertical direction. A camera calibration method wherein the aforementioned horizontal direction is a direction along the display surface and is the horizontal direction within the camera image.

2. The target image includes a first straight line (30b) and a second straight line (30c) that intersects the first straight line. The camera calibration method according to claim 1, wherein the angle difference (ΔGp) between the first shooting angle (G1p) corresponding to the first vertex angle in the camera image and the second shooting angle (G2p) corresponding to the second vertex angle in the camera image is smaller than the angle difference (ΔGr) between the first obtuse vertex angle (G1r) formed by the first line and the second line and the second acute vertex angle (G2r) formed by the first line and the second line.

3. The target image includes a first straight line (30b) and a second straight line (30c) that intersects the first straight line. The camera calibration method according to claim 1, wherein in the camera image, a first shooting line (L1p) representing the first straight line and a second shooting line (L2p) representing the second straight line are orthogonal, and in the target image, a pair of first vertex angles (G1r) and another pair of second vertex angles (G2r) formed by the first straight line and the second straight line are different from each other.

4. The display surface extends upward along the horizontal plane, The camera is installed on the vehicle (10), is located above the display surface and is positioned diagonally downward, A camera calibration method according to any one of claims 1 to 3, wherein the vehicle is positioned on a mounting surface (803) that extends along the horizontal plane, and the target real image is captured by the camera.

5. A target display unit used for calibrating cameras (201-204), The camera displays the target images (30, 31) captured by the camera, and the display surface (802) is tilted with respect to the optical axis (201a, 202a, 203a, 204a) of the camera. In the camera image (Pc) captured by the camera, the maximum value of the maximum vertical dimension (Wvp) and the maximum horizontal dimension (Whp) of the target image (Pct) representing the target image is defined as the first image dimension (W1p) and the minimum value as the second image dimension (W2p). In the camera image (Pc) captured by the camera, the maximum value of the maximum horizontal dimension (Dhp) of the target image (Pct) is defined as the first image dimension (W1r) and the minimum value as the second image dimension (W2r). The target image is formed such that the ratio (Rpc) of the first image dimension to the second image dimension is closer to 1 than the ratio (Rr) of the first image dimension to the second image dimension. The aforementioned vertical correspondence direction is the direction along the display surface, and within the camera image, it is the direction that corresponds to the vertical direction. The aforementioned horizontal direction is a direction along the display surface, which becomes the horizontal direction within the camera image, in the target display body.

6. The target image includes a first straight line (30b) and a second straight line (30c) that intersects the first straight line. The target display body according to claim 5, wherein the angle difference (ΔGp) between the first shooting angle (G1p) corresponding to the first vertical angle in the camera image and the second shooting angle (G2p) corresponding to the second vertical angle in the camera image is smaller than the angle difference (ΔGr) between the first obtuse vertical angle (G1r) formed by the first line and the second line and the second acute vertical angle (G2r) formed by the first line and the second line.

7. The target image includes a first straight line (30b) and a second straight line (30c) that intersects the first straight line. The target display body according to claim 5, wherein in the camera image, a first photographic line (L1p) representing the first straight line and a second photographic line (L2p) representing the second straight line are orthogonal, and in the target image, a pair of first vertically opposite angles (G1r) and another pair of second vertically opposite angles (G2r) formed by the first straight line and the second straight line are different from each other.

Images