Vehicle position / attitude estimation method, vehicle position / attitude estimation program, and vehicle position / attitude estimation device

Abstract

To provide a technique capable of calculating a vehicle position and a vehicle attitude without requiring a sensor that can perform range-finding other than an image sensor.SOLUTION: A vehicle position / attitude estimation method of estimating a position and an attitude of a vehicle from an image 10 obtained by imaging a vehicle 20 comprises: an information acquisition step of acquiring two-dimensional information of the vehicle captured in the image and a vehicle size including the width, height and length of the vehicle; and a vehicle position / attitude estimation step of estimating the position and the attitude of the vehicle on the basis of such a relation that each apex of a three-dimensional bounding box in a camera coordinate system is limited to two dimensional information when being projected to an image coordinate system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle position / attitude estimation method, a vehicle position / attitude estimation program, and a vehicle position / attitude estimation device for estimating the position and attitude of a vehicle. [Background technology]

[0002] Non-Patent Document 1 describes a method for estimating a 3D bounding box (3DBBox) surrounding a vehicle from an image of the vehicle. In this method, a 2D bounding box (2DBBox), vehicle posture R, and vehicle size D are estimated using a deep neural network (DNN). Then, the 3DBBox is estimated by calculating the vehicle position T under the geometric constraint that each vertex of the 3DBBox fits within the 2DBBox when projected in the camera coordinate system. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Arsalan Mousavian et al., “3D Bounding Box Estimation Using Deep Learning and Geometry”, CVPR2017, 2017 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology of Non-Patent Document 1, it is necessary to estimate the vehicle attitude R and vehicle size D using a DNN, and then calculate the vehicle position based on the estimated vehicle attitude R and vehicle size D. To train a DNN that estimates the vehicle attitude R and vehicle size D, true values ​​containing this information are required. To generate accurate true values ​​of information related to the vehicle attitude R and vehicle size D for DNN training, a sensor capable of measuring distances other than an image sensor (such as LiDAR) is required, and therefore the cost of generating the true values ​​is high.

[0005] In view of the above, an object of the present invention is to provide a technology capable of calculating a vehicle position and a vehicle attitude without requiring a sensor capable of measuring distance other than an image sensor. [Means for solving the problem]

[0006] The present invention provides a vehicle position / orientation estimation method for estimating the position and orientation of a vehicle from an image of the vehicle captured by an imaging device. This vehicle position / orientation estimation method includes an acquisition step of acquiring two-dimensional information about the vehicle captured in the image and a vehicle size including the width, height, and length of the vehicle, and a calculation step of calculating the position and orientation of the vehicle based on a relationship in which each vertex of a three-dimensional bounding box in a camera coordinate system is constrained by the two-dimensional information when projected onto an image coordinate system. and,

[0007] According to the above vehicle position / orientation estimation method, when each vertex of a three-dimensional bounding box in the camera coordinate system is projected onto the image coordinate system, a simultaneous equation or determinant can be obtained in which parameters related to the vehicle position and vehicle orientation are unknown variables based on relationships constrained by two-dimensional information. Then, the vehicle position and vehicle orientation can be estimated by solving this simultaneous equation or determinant using the acquired vehicle size. According to the vehicle position / orientation estimation method of the present invention, unlike Non-Patent Document 1, it is not necessary to estimate the vehicle orientation R in advance using a DNN, and therefore the vehicle position and vehicle orientation can be calculated without requiring a sensor capable of measuring distance other than an image sensor.

[0008] The present invention also provides a vehicle position / attitude estimation program for estimating the position and attitude of a vehicle from an image of the vehicle. This vehicle position / attitude estimation program causes a computer to execute an information acquisition step of acquiring two-dimensional information of the vehicle captured in the image and a vehicle size including the width, height, and length of the vehicle, and a vehicle position / attitude estimation step of estimating the position and attitude of the vehicle based on a relationship in which each vertex of the three-dimensional bounding box in the camera coordinate system is constrained by the two-dimensional information when projected onto an image coordinate system.

[0009] The present invention also provides a vehicle position / attitude estimation device that acquires a photographed image of a vehicle and estimates the position and attitude of the vehicle, comprising: an image acquisition unit that acquires a photographed image of the vehicle, an information acquisition unit that acquires two-dimensional information of the vehicle photographed in the image and a vehicle size including the width, height, and length of the vehicle, and a vehicle position / attitude estimation unit that estimates the position and attitude of the vehicle based on a relationship in which each vertex of the three-dimensional bounding box in the camera coordinate system is constrained by the two-dimensional information when projected onto an image coordinate system. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing a state in which an extended 2DBBox used in a vehicle position estimation method according to an embodiment is applied to a camera image of a vehicle. [Figure 2] A diagram showing the application of 2DBBox to a camera image of a vehicle. [Figure 3] A diagram showing the application of 3DBBox to a camera image of a vehicle. [Figure 4] 1 is a schematic diagram of an in-vehicle system including a vehicle position estimation device according to an embodiment; [Figure 5] 1A and 1B are diagrams illustrating a vehicle posture and a bounding box. [Figure 6] FIG. 2 is a diagram illustrating a vehicle size. [Figure 7]3 is a flowchart of a vehicle position / estimation method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] In the vehicle position / posture estimation method according to the embodiment, as shown in FIG. 1, a bounding box 30 is applied to an image 10 in which a vehicle 20 is captured, and the vehicle position and posture of the vehicle 20 are estimated.

[0012] The bounding box 30 is an extended 2DBBox and includes a rectangular bounding box 33 that encloses the top, bottom, left, and right edges of the vehicle 20, position information in the left-right direction (horizontal direction in FIG. 1) of a corner (point P5 in FIG. 1) of the face closest to the viewpoint, either the front or back of the vehicle 20, and position information in the up-down direction (vertical direction in FIG. 1) of the bottom edge (point P7 in FIG. 1) of the face farthest from the viewpoint, either the front or back of the vehicle 20. Here, the horizontal direction is the direction of the line segment connecting points P1, P5, and P2, and is the left-right direction of the bounding box 33. The vertical direction is the direction of the line segment connecting points P4, P7, and P2, and is the up-down direction of the bounding box 33. In this specification, the horizontal direction may be referred to as the x-direction, and the vertical direction may be referred to as the y-direction.

[0013] 2 shows a two-dimensional bounding box 40 applied to an image 10. The bounding box 40 is a 2DBBox that surrounds a vehicle 20 in a rectangular shape. The bounding box 33 shown in FIG. 1 is a rectangular 2DBBox similar to the bounding box 40 shown in FIG. 2.

[0014] 3 shows a state in which a three-dimensional bounding box 50 is applied to image 10. Bounding box 50 is a 3DBBox that surrounds vehicle 20 with a rectangular parallelepiped. In bounding box 50, which is a rectangular parallelepiped, edge line 51 is the edge line between the front surface and the left surface of vehicle 20, and edge line 52 is the edge line between the bottom surface and the left surface of vehicle 20.

[0015] Line segment 31 shown in FIG. 1 connects points P5 and P6 and is located horizontally at the same position as edge line 51. Line segment 31 is parallel to the line segment connecting points P1 and P3 and the line segment connecting points P2 and P4. Line segment 32 connects points P5 and P7 and is similar to edge line 52. In other words, bounding box 30 shown in FIG. 1 is an extended 2DBBox obtained by adding edges 51 and 52 of bounding box 50, which is a 3DBBox, to bounding box 40, which is a 2DBBox. The extended 2DBBox includes position information for bounding box 40, which is a 2DBBox, and edges 51 and 52.

[0016] FIG. 4 illustrates an example of an in-vehicle system capable of implementing the vehicle position / attitude estimation method. The in-vehicle system is mounted on the vehicle 20 and includes an imaging device 60 and an ECU 70. The imaging device 60 may be a monocular camera such as a CCD camera, a CMOS image sensor, or a near-infrared camera, or may be a stereo camera. The ECU 70 acquires an image 10 of the vehicle 20 from the imaging device 60 and estimates the vehicle position T and vehicle attitude R of the vehicle 20 by applying a bounding box 30 to the image 10 of the vehicle 20. Note that the image 10 may be acquired from a device other than the imaging device 60 mounted on the vehicle 20. For example, the image 10 of the vehicle 20 may be acquired from an imaging device installed outside the vehicle 20 via a communication device.

[0017] The ECU 70 includes an image acquisition unit 71, a DNN estimation unit 72, a Box projection unit 73, and a vehicle position / attitude estimation unit 74. The ECU 70 is a computer (more specifically, a microcomputer) equipped with a CPU, a ROM, a RAM, an operation unit, a storage device, an input / output interface, etc. The CPU realizes these functions by executing programs stored in the ROM and RAM. As a result, the ECU 70 functions as a vehicle position / attitude estimation device that acquires an image 10 captured of the vehicle 20 and estimates the position and attitude of the vehicle 20.

[0018] The image acquisition unit 71 acquires images 10 of the vehicle 20 from the imaging device 60 or the like. The acquired images 10 are stored in a storage unit (ROM, RAM, etc.) of the ECU 70 in chronological order.

[0019] The DNN estimation unit 72 uses a deep neural network (DNN) to estimate various parameters such as the position information of each point constituting the extended 2DBBox and the vehicle size D (width W, height H, length L). The position information of each point constituting the extended 2DBBox is the y coordinate of the top end of the vehicle 20 (top), the y coordinate of the bottom end (bottom), the x coordinate of the left end (left), the x coordinate of the right end (right), the x coordinate (devide_x) of the corner of the front or back of the vehicle 20 that is closest to the viewpoint, and the y coordinate (tail_y) of the bottom end of the front or back of the vehicle 20 that is farthest from the viewpoint. 1, top is the y coordinate of points P3, P4, and P6, bottom is the y coordinate of points P1, P2, and P5, left is the x coordinate of points P1 and P3, right is the x coordinate of points P2, P4, and P7, devide_x is the x coordinate of points P5 and P6, and tail_y is the y coordinate of point P7. The DNN estimation unit 72 estimates the bounding box 30 by estimating the drawing positions of the bounding box 33 and the line segments 31 and 32 from the pixel information of the image 10.

[0020] It should be noted that a preset value can be used for the vehicle size D instead of the value estimated by the DNN estimation unit 72. Furthermore, since the vehicle size D remains unchanged even if the position or posture of the vehicle 20 changes, it is also possible to use an estimated value previously estimated by the DNN estimation unit 72.

[0021] The box projection unit 73 projects a bounding box onto the image 10 of the vehicle 20, which is acquired by the image acquisition unit 71 and stored in the storage unit of the ECU 70. This allows the bounding boxes 30, 40, and 50 to be projected onto the image 10 of the vehicle 20, for example, as shown in FIGS.

[0022] The vehicle position / attitude estimation unit 74 estimates the position and attitude of the vehicle 20 based on the relationship that each vertex of the 3DBBox in the camera coordinate system is constrained by two-dimensional information when projected onto the image coordinate system. This constraint can be expressed by the following formula (1). In the following formula (1), the image coordinate xpcs on the left side is xpcs=(x,y), and the camera coordinate Xccs on the right side is Xccs=(X,Y,Z), and the right side corresponds to the camera coordinate converted into the image coordinate system. Note that examples of two-dimensional information include information represented by a 2DBBox, an extended 2DBBox, etc. In this embodiment, a case where information represented by an extended 2DBBox is used as two-dimensional information will be described as an example.

[0023]

number

[0024] In the above equation (1), s is the scale factor of the projective transformation. R is the vehicle attitude, and is expressed as R=(p, r, θ) using pitch angle p, roll angle r, and yaw angle θ. Note that p=r=0. T is the vehicle position, and is expressed as T=(Tx, Ty, Tz). K is the internal parameter matrix of the imaging device 60, and can be expressed by the following equation (2). In the following equation (2), fx and fy are the x and y components of the focal length of the camera, and cx and cy are the x and y components of the principal point of the camera.

[0025]

number

[0026] By transforming the above formula (1), the following formula (3) can be obtained. By using the above formula (2), xpcs=(x, y), and Xccs=(X, Y, Z) in the following formula (3), the following formula (4) can be obtained.

[0027]

number

[0028] By rearranging the above equation (4) with respect to the four unknown variables, yaw angle θ and vehicle position T = (Tx, Ty, Tz), the following equations (5) and (6) can be obtained. Note that A represents an unknown variable, A = (θ, Tx, Ty, Tz). The yaw angle θ is treated as two variables, sin θ and cos θ.

[0029]

number

[0030] As described above, by combining one vertex of the 3DBBox in the camera coordinate system with one point of the extended 2DBBox in the image coordinate system, two simultaneous equations can be obtained with sinθ, cosθ, Tx, Ty, and Tz as unknown variables. For example, since there are five unknown variables, if there are three pairs of vertices of the 3DBBox and points of the extended 2DBBox, six simultaneous equations can be obtained, and sinθ, cosθ, Tx, Ty, and Tz can be calculated linearly. The simultaneous equations can be solved using, for example, the Moore-Penrose inverse matrix.

[0031] Fig. 5 is a diagram showing the relationship between points 1 to 4 of the extended 2DBox in the image coordinate system and the vehicle posture R. As shown in Fig. 5, regardless of whether the surface of the vehicle 20 captured in the image 10 is the front (F), back (B), left (L), right (R), front / left (LF), back / left (LB), front / right (RF), or back / right (RB), three or more sets of points can be obtained by applying the extended 2DBox.

[0032] 6 is a diagram showing the relationship between each vertex of the 3DBBox in the camera coordinate system and the vehicle size D (width W, height H, length L). As shown in Fig. 6, the center of the 3DBBox is set as the origin O, and the x-axis is set to the right of the vehicle 20, the y-axis is set downward, and the z-axis is set forward. In this case, the coordinates of the front upper left edge flt, front lower left edge flb, front upper right edge frt, front lower right edge frb, rear upper left edge rlt, rear lower left edge rlb, rear upper right edge rrt, and rear lower right edge rrb of the vehicle 20 can be expressed as follows: flt = (-W / 2, -H / 2, L / 2), flb = (-W / 2, H / 2, L / 2), frt = (W / 2, -H / 2, L / 2), frb = (W / 2, H / 2, L / 2), rlt = (-W / 2, -H / 2, -L / 2), rlb = (-W / 2, H / 2, -L / 2), rrt = (W / 2, -H / 2, -L / 2), and rrb = (W / 2, H / 2, -L / 2). That is, the camera coordinates Xccs = (X, Y, Z) can be calculated from the vehicle size D.

[0033] The simultaneous equations shown in equations (5) and (6) above can also be solved using a nonlinear method described below. By selecting M / 2 combinations of one vertex of the 3DBBox in the camera coordinate system and one point of the extended 2DBBox in the image coordinate system according to the target type, M simultaneous equations can be obtained. These M simultaneous equations are designated as f1(A) to fM(A). For N unknown variables A = (θ, Tx, Ty, Tz) = (a1, a2, ..., aN), the variation ΔA = (Δa1, Δa2, ..., ΔaN) is defined as a linear Taylor expansion, and the following equation (7) can be obtained. Furthermore, the following equation (8) is a determinant of equation (7).

[0034]

number

[0035] Furthermore, from the above formulas (5) and (6), the following formulas (9) and (10) can be obtained.

[0036]

number

[0037] The above equation (8) can be expressed as the following equation (11) using the Jacobian determinant J. When M>N, the following equation (11) can be solved using the Moore-Penrose inverse matrix as shown in the following equation (12).

[0038]

number

[0039] After calculating ΔA from the above equation (12), A=A-ΔA, and the procedure explained using the above equations (7) to (12) is repeated until ΔA converges. That is, the procedure is repeated until the absolute value of ΔA, abs(ΔA), becomes smaller than a predetermined value ε (until abs(ΔA)<ε). Note that the initial value of A can be a value calculated by a linear solution.

[0040] According to the ECU 70, as shown in the above equation (1), each vertex of the 3DBBox in the camera coordinate system is constrained to an extended 2DBBox when projected onto the image coordinate system, and based on this relationship, a determinant or simultaneous equations can be obtained with parameters related to the vehicle position T and vehicle attitude R as unknown variables, as shown in the above equations (4) to (6). Then, by solving this determinant or simultaneous equation using the acquired vehicle size D, the unknown variables of the vehicle position T and vehicle attitude R can be estimated. Unlike the technology described in Non-Patent Document 1, there is no need to estimate the vehicle attitude R in advance using a DNN. Therefore, as shown in FIG. 4, the vehicle position T and vehicle attitude R of the vehicle 20 can be calculated without requiring any distance measuring sensors other than an image sensor such as the imaging device 60.

[0041] Fig. 7 shows a flowchart of the vehicle position / attitude estimation process performed by the ECU 70. The ECU 70 executes the vehicle position / attitude estimation program stored in the storage unit to perform the processes shown in Fig. 7. The processes shown in this flowchart are executed continuously at predetermined intervals.

[0042] First, in step S101, an image 10 captured by the vehicle 20 is acquired from the imaging device 60. The acquired image 10 is stored as image information in the ECU 70. Thereafter, the process proceeds to step S102.

[0043] In step S102, the extended 2DBBox and the vehicle size D are estimated by the DNN. For example, the coordinates of points P1 to P7 constituting the bounding box 30 in the image 10 shown in Fig. 1 and the vehicle size D (width W, height H, length L) are estimated. Note that if the vehicle size D is set in advance, there is no need to estimate the vehicle size D by the DNN in step S102.

[0044] In step S103, the extended 2DBBox estimated in step S102 is projected onto the image 10 acquired in step S101. For example, a bounding box 30 is projected onto the image 10 as shown in FIG.

[0045] In step S104, the vehicle position T and vehicle attitude R are estimated for the vehicle 20. Based on the relationship shown in the above equation (1) that each vertex of the 3DBBox in the camera coordinate system is constrained to an extended 2DBBox when projected onto the image coordinate system, the vehicle position T and vehicle attitude R of the vehicle 20 are estimated by solving the determinant shown in the above equation (4) or the simultaneous equations shown in the above equations (5) and (6).

[0046] As described above, according to the vehicle position / orientation estimation method of this embodiment, in step S102, the extended 2DBBox and vehicle size D are estimated using a DNN, but the vehicle orientation R is not estimated using a DNN. Instead, in step S104, the vehicle position T and vehicle orientation R of the vehicle 20 are estimated. In step S104, the vehicle position T and vehicle orientation R can be estimated by solving a determinant or simultaneous equations obtained based on the relationship that is constrained by the extended 2DBBox when each vertex of the 3DBBox in the camera coordinate system is projected onto the image coordinate system. According to the vehicle position / orientation estimation method of this embodiment, unlike Non-Patent Document 1, it is not necessary to estimate the vehicle orientation R in advance using a DNN, and therefore the vehicle position T and vehicle orientation R can be calculated without requiring any distance-measuring sensor other than an image sensor.

[0047] Furthermore, by using an extended 2DBBox as two-dimensional information that constrains each vertex of the 3DBBox in the camera coordinate system, the devide_x and tail_y coordinates can be used in addition to the top, bottom, left, and right coordinates of the vehicle 20, which makes it possible to more effectively prevent the estimated results of the vehicle position T and vehicle attitude R from becoming inconsistent compared to when a 2DBBox is used as two-dimensional information.

[0048] According to the above embodiment, the following effects can be obtained.

[0049] A vehicle position / attitude estimation method for estimating the position and attitude of a vehicle 20 from an image 10 captured of the vehicle 20 includes an information acquisition step (e.g., step S102) and a vehicle position / attitude estimation step (e.g., step S104). The information acquisition step acquires two-dimensional information of the vehicle 20 captured in the image 10 and a vehicle size D: D = (W, H, L) including the width W, height H, and length L of the vehicle 20. Note that the information acquisition step may acquire the vehicle size D by estimation, or may acquire it by being set in advance by input, etc. The vehicle position / attitude estimation step (step S104) estimates the vehicle position T: T = (Tx, Ty, Tz) and the vehicle attitude R: R = (p, r, θ) based on the relationship that each vertex of a three-dimensional bounding box (50) in the camera coordinate system is constrained by two-dimensional information when projected onto the image coordinate system.

[0050] According to the vehicle position / orientation estimation method described above, when each vertex of the 3DBBox in the camera coordinate system is projected onto the image coordinate system, a determinant or simultaneous equations with parameters related to the vehicle position T and vehicle orientation R as unknown variables can be obtained in the vehicle position / orientation estimation step, as shown in the above equations (4) to (6), based on the relationship constrained by two-dimensional information. Then, by solving this determinant or simultaneous equations using the acquired vehicle size D, the unknown variables of the vehicle position T and vehicle orientation R can be estimated. Therefore, unlike the technology described in Non-Patent Document 1, it is not necessary to estimate the vehicle orientation R in advance using a DNN, and therefore the vehicle position T and vehicle orientation R of the vehicle 20 can be estimated without requiring any distance-measuring sensor other than an image sensor.

[0051] In the above vehicle position / attitude estimation method, the two-dimensional information may be an extended two-dimensional bounding box (extended 2DBBox). Here, the extended 2DBBox includes a rectangular 2DBBox (bounding box 33) that surrounds the top, bottom, left, and right edges of the vehicle 20, horizontal position information of a corner (P5) of the face closest to the viewpoint, either the front or back of the vehicle 20, and vertical position information of the bottom edge (P7) of the face farthest from the viewpoint, either the front or back of the vehicle 20. When the extended 2DBBox is used as the two-dimensional information, it is possible to more effectively prevent inconsistencies in the estimation results of the vehicle position T and vehicle attitude R compared to when the 2DBBox is used as the two-dimensional information.

[0052] In the above vehicle position / attitude estimation method, the information acquisition step may be configured to acquire a preset vehicle size D as the vehicle size D. Alternatively, as shown in step S102, the information acquisition step may be configured to acquire a vehicle size D estimated by a DNN.

[0053] The vehicle position / attitude estimation method described above can be realized by causing a computer such as the ECU 70 to execute a vehicle position / attitude estimation program that estimates the position and attitude of the vehicle 20 from an image 10 captured of the vehicle 20. This vehicle position / attitude estimation program executes an information acquisition step (e.g., S102) that acquires two-dimensional information of the vehicle 20 captured in the image 10 and a vehicle size D including the width W, height H, and length L of the vehicle 20, and a vehicle position / attitude estimation step (e.g., S104) that estimates the position and attitude of the vehicle 20 based on a relationship in which each vertex of a three-dimensional bounding box (50) in the camera coordinate system is constrained by the two-dimensional information when projected onto the image coordinate system.

[0054] The above vehicle position / attitude estimation method can be realized by a vehicle position / attitude estimation device (ECU 70) that acquires an image 10 of the vehicle 20 and estimates the position and attitude of the vehicle 20. This vehicle position / attitude estimation device includes an image acquisition unit 71 that acquires the image 10 of the vehicle 20, an information acquisition unit (e.g., a DNN estimation unit 72) that acquires two-dimensional information of the vehicle 20 captured in the image 10 and a vehicle size D including the width W, height H, and length L of the vehicle 20, and a vehicle position / attitude estimation unit 74 that estimates the position and attitude of the vehicle 20 based on a relationship in which each vertex of a three-dimensional bounding box (bounding box 50) in the camera coordinate system is constrained by the two-dimensional information when projected onto the image coordinate system.

[0055] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium. [Explanation of symbols]

[0056] 10...image, 20...vehicle, 30, 40, 50...bounding box, 70...ECU, 71...image acquisition unit, 72...DNN estimation unit, 74...vehicle position / posture estimation unit

Claims

1. A vehicle position / attitude estimation method for estimating a position and attitude of a vehicle (20) from an image (10) of the vehicle taken by a camera (60), comprising: an image coordinate system is defined by x and y directions, which are horizontal and vertical directions in the image, and a camera coordinate system is defined by x, y and z directions, which are the horizontal and vertical directions of the image plus a front-to-back direction, By computer, an information acquisition step (S102) of acquiring two-dimensional information of the vehicle captured in the image and a vehicle size including a width, a height, and a length of the vehicle; a vehicle position / posture estimation step (S104) for estimating the position and posture of the vehicle, which are represented by coordinate positions in the x-direction, y-direction, and z-direction in the camera coordinate system, based on a relationship constrained by the two-dimensional information (30) when each vertex of the three-dimensional bounding box (50) in the camera coordinate system is projected onto the image coordinate system; The two-dimensional information is First position information is information on the positions of the four corners of a rectangular two-dimensional bounding box (33) that surrounds the top, bottom, left, and right edges of the vehicle in the image; Second position information is information on a position corresponding to a corner (P5, P6) of the front or rear surface of the vehicle that is closest to the camera viewpoint and is midway in the left-right direction of the two-dimensional bounding box; and third position information, which is information on a position corresponding to a bottom end (P7) of the front or rear surface of the vehicle that is farthest from the camera viewpoint and is midway in the up-down direction of the two-dimensional bounding box; In the information acquisition step, the first to third position information in the extended two-dimensional bounding box is acquired; a vehicle position / orientation estimation method, in which the vehicle position / orientation estimation step uses the width, height, and length of the vehicle, and estimates the position and orientation of the vehicle based on a relationship in which each vertex of the three-dimensional bounding box, when projected onto the image coordinate system, is constrained by the first to third position information in the extended two-dimensional bounding box.

2. The vehicle position / attitude estimation method according to claim 1 , wherein the information acquisition step acquires, as the vehicle size, a predetermined vehicle size or a vehicle size estimated by a deep neural network.

3. A vehicle position / attitude estimation program that estimates the position and attitude of a vehicle (20) from an image (10) of the vehicle (20) captured by a camera (60), comprising: an image coordinate system is defined by x and y directions, which are horizontal and vertical directions in the image, and a camera coordinate system is defined by x, y and z directions, which are the horizontal and vertical directions of the image plus a front-to-back direction, On the computer, an information acquisition step (S102) of acquiring two-dimensional information of the vehicle captured in the image and a vehicle size including a width, a height, and a length of the vehicle; a vehicle position / posture estimation step (S104) for estimating the position and posture of the vehicle, which are represented by coordinate positions in the x-direction, y-direction, and z-direction in the camera coordinate system, based on a relationship constrained by the two-dimensional information when each vertex of the three-dimensional bounding box (50) in the camera coordinate system is projected onto the image coordinate system; The two-dimensional information is First position information is information on the positions of the four corners of a rectangular two-dimensional bounding box (33) that surrounds the top, bottom, left, and right edges of the vehicle in the image; Second position information is information on a position corresponding to a corner (P5, P6) of the front or rear surface of the vehicle that is closest to the camera viewpoint and is midway in the left-right direction of the two-dimensional bounding box; and third position information, which is information on a position corresponding to a bottom end (P7) of the front or rear surface of the vehicle that is farthest from the camera viewpoint and is midway in the up-down direction of the two-dimensional bounding box; In the information acquisition step, the first to third position information in the extended two-dimensional bounding box is acquired; a vehicle position / posture estimation program, wherein in the vehicle position / posture estimation step, the width, height, and length of the vehicle are used, and the position and posture of each vertex of the three-dimensional bounding box is estimated based on a relationship in which, when projected onto the image coordinate system, the vertices are constrained by the first to third position information in the extended two-dimensional bounding box.

4. A vehicle position / attitude estimation device (70) that acquires an image (10) of a vehicle (20) captured by a camera (60) and estimates the position and attitude of the vehicle, an image coordinate system is defined by x and y directions, which are horizontal and vertical directions in the image, and a camera coordinate system is defined by x, y and z directions, which are the horizontal and vertical directions of the image plus a front-to-back direction, an image acquisition unit (71) that acquires an image of the vehicle; an information acquisition unit (72) that acquires two-dimensional information of the vehicle captured in the image and a vehicle size including the width, height, and length of the vehicle; a vehicle position / posture estimation unit (74) that estimates the position of the vehicle, which is represented by coordinate positions in the x-direction, y-direction, and z-direction in the camera coordinate system, and the posture of the vehicle, based on a relationship that is constrained by the two-dimensional information when each vertex of the three-dimensional bounding box (50) in the camera coordinate system is projected onto the image coordinate system; The two-dimensional information is First position information is information on the positions of the four corners of a rectangular two-dimensional bounding box (33) that surrounds the top, bottom, left, and right edges of the vehicle in the image; Second position information is information on a position corresponding to a corner (P5, P6) of the front or rear surface of the vehicle that is closest to the camera viewpoint and is midway in the left-right direction of the two-dimensional bounding box; and third position information, which is information on a position corresponding to a bottom end (P7) of the front or rear surface of the vehicle that is farthest from the camera viewpoint and is midway in the up-down direction of the two-dimensional bounding box; the information acquisition unit acquires the first to third position information in the extended two-dimensional bounding box; The vehicle position / posture estimation unit estimates the position and posture of the vehicle using the width, height, and length of the vehicle, and based on a relationship in which each vertex of the three-dimensional bounding box is constrained by the first to third position information in the extended two-dimensional bounding box when projected onto the image coordinate system.

Images