Inertial measurement unit evaluation method and system

Abstract

An inertial measurement unit evaluation method is used to evaluate the accuracy of an inertial measurement unit included in a self-driving car. The self-driving car further includes a left camera and a right camera, both of which face the front direction of the self-driving car. The inertial measurement unit evaluation method includes an image capture step, a vanishing line calculation step, an image parameter determination step, a unit parameter acquisition step, and a parameter comparison step. The vanishing line calculation step includes calculating a vanishing line equation for the vanishing line, where the vanishing line is the line connecting the first vanishing point and the second vanishing point. The image parameter determination step includes determining the image parameter set of the self-driving car based on the vanishing line equation. The unit parameter acquisition step includes obtaining the unit parameter set of the self-driving car through the inertial measurement unit. In this way, the accuracy of the inertial measurement unit can be effectively evaluated. The present application also relates to a related inertial measurement unit evaluation system.

Description

Technical Field

[0001] The present invention relates to an inertial measurement unit evaluation method and system thereof, and more particularly to an inertial measurement unit evaluation method and system thereof applied to a self-driving car. Background Art

[0002] When self-driving cars use high-precision maps for positioning, multipath effects or non-line-of-sight propagation (NLOS) in their Global Navigation Satellite System (GNSS) can cause errors that are difficult to correct. In densely obstructed urban areas, where tall buildings and structures can block satellite signals over large areas, the magnitude and direction of these errors are affected by the geometric relationship between obstacles and the signal. The resulting path delay error can reach tens of meters, necessitating the use of an inertial measurement unit (IMU) for inertial navigation assistance.

[0003] An inertial measurement unit (IMU) typically consists of a gyroscope and an accelerometer, with accuracy being the most important performance metric. IMU errors include both gyroscope errors and drift errors. Drift errors are caused by external noise, vibration, and temperature gradients, causing actual errors to far exceed the specified error. Therefore, IMUs must be calibrated before use. Current calibration methods require the use of complex and modified mathematical operations as filters, or the use of expensive, high-precision IMUs.

[0004] Therefore, there is a great need in the market for a method and system that can improve the accuracy of an inertial measurement unit while effectively reducing computational complexity and cost. Summary of the Invention

[0005] The present invention provides an inertial measurement unit (IMU) evaluation method and system. The method utilizes dual cameras on a self-driving car and includes a vanishing line calculation step, an image parameter determination step, an IMU parameter acquisition step, and a parameter comparison step to assess whether to perform further computationally intensive calibration on the IMU.

[0006] According to one embodiment of the present invention, an inertial measurement unit (IMU) evaluation method is provided for evaluating the accuracy of an IMU included in a self-driving car. The self-driving car further includes a left camera and a right camera, both facing forward of the self-driving car. The IMU evaluation method includes an image capture step, a vanishing line calculation step, an image parameter determination step, a unit parameter acquisition step, and a parameter comparison step. The image capture step includes capturing a left image and a right image at a time point using the left camera and the right camera, respectively. The left image includes a first left lane line, a first right lane line, and a first vanishing point, which is the intersection of the first left lane line and the first right lane line; the right image includes a second left lane line, a second right lane line, and a second vanishing point, which is the intersection of the second left lane line and the second right lane line. The vanishing line calculation step includes calculating a vanishing line equation for the vanishing line, where the vanishing line is a line connecting the first vanishing point and the second vanishing point. The image parameter determination step includes determining a set of image parameters for the self-driving car based on the vanishing line equation. The unit parameter acquisition step includes acquiring the set of unit parameters for the self-driving car using the IMU. The parameter comparison step includes comparing the image parameter set and the unit parameter set and generating a comparison result, thereby effectively evaluating the accuracy of the inertial measurement unit.

[0007] According to another embodiment of the present invention, an inertial measurement unit evaluation system is provided, which is disposed in a self-driving car. The inertial measurement unit evaluation system includes an inertial measurement unit, a left camera, a right camera, a storage medium, and a processor. The left camera faces the front direction of the self-driving car. The right camera faces the front direction. The storage medium provides an inertial measurement unit evaluation program. The processor is communicatively coupled to the inertial measurement unit, the left camera, the right camera, and the storage medium. Based on the inertial measurement unit evaluation program, the processor is configured to capture a left image and a right image at a time point using the left camera and the right camera, respectively. The left image includes a first left lane line, a first right lane line, and a first vanishing point, the first vanishing point being the intersection of the first left lane line and the first right lane line. The right image includes a second left lane line, a second right lane line, and a second vanishing point, the second vanishing point being the intersection of the second left lane line and the second right lane line. The processor, based on the IMU evaluation program, is further configured to calculate a vanishing line, which is a line connecting the first vanishing point and the second vanishing point. Based on the vanishing line, an image parameter set of the self-driving vehicle is determined. A unit parameter set of the self-driving vehicle is obtained via the IMU. The image parameter set and the unit parameter set are compared to generate a comparison result. Thus, the IMU evaluation system can assess whether to perform further computationally intensive calibration on the IMU. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 FIG2 is a flow chart illustrating an inertial measurement unit evaluation method according to a first embodiment of the present invention;

[0009] Figure 2AFIG2 is a block diagram illustrating an inertial measurement unit evaluation system according to a second embodiment of the present invention;

[0010] Figure 2B 、 Figure 2C and Figure 2D Draw separately Figure 2A Schematic diagrams of the self-driving car set up by the inertial measurement unit evaluation system, including top, side, and front views;

[0011] Figure 3A 、 Figure 3B 、 Figure 3C 、 Figure 3D 、 Figure 3E and Figure 3F Draw separately Figure 1 Schematic diagram of images corresponding to the sub-steps of the image capture step;

[0012] Figure 4A and Figure 4B Draw separately Figure 1 Reference map for image coordinate system transformation in the vanishing line calculation step;

[0013] Figure 5A and Figure 5B Schematic diagrams respectively illustrate the vanishing points of the left image and the right image of the reference line used to determine the image roll tilt angle in the inertial measurement unit evaluation method of the first embodiment;

[0014] Figure 5C A schematic diagram illustrating a reference line for determining an image roll tilt angle in an inertial measurement unit evaluation method according to a first embodiment;

[0015] Figure 5D and Figure 5E Schematic diagrams respectively illustrating the vanishing points of the left image and the right image for determining the vanishing line of the image roll tilt angle in the inertial measurement unit evaluation method of the first embodiment;

[0016] Figure 5F A schematic diagram illustrating a vanishing line for determining an image roll tilt angle in an inertial measurement unit evaluation method according to a first embodiment;

[0017] Figure 5G A schematic diagram illustrating an image rollover tilt angle in an inertial measurement unit estimation method according to a first embodiment is shown;

[0018] Figure 6A and Figure 6B Schematic diagrams respectively illustrate the vanishing points of the left image and the right image of the reference line used to determine the vertical tilt angle of the image in the inertial measurement unit evaluation method of the first embodiment;

[0019] Figure 6CA schematic diagram illustrating a reference line for determining an image's vertical tilt angle in the inertial measurement unit evaluation method according to the first embodiment;

[0020] Figure 6D and Figure 6E Schematic diagrams respectively illustrate the vanishing points of the left image and the right image used to determine the vanishing lines of the vertical tilt angle of the image in the inertial measurement unit evaluation method of the first embodiment;

[0021] Figure 6F A schematic diagram illustrating a vanishing line for determining the vertical tilt angle of an image in the inertial measurement unit evaluation method according to the first embodiment;

[0022] Figure 6G A schematic diagram illustrating parameters used to determine the vertical tilt angle of an image in the inertial measurement unit evaluation method of the first embodiment is shown;

[0023] Figure 6H A schematic diagram illustrating the vertical tilt angle of an image in the inertial measurement unit evaluation method according to the first embodiment is shown;

[0024] Figure 7A and Figure 7B Schematic diagrams respectively illustrate the vanishing points of the left image and the right image of the reference line used to determine the left and right tilt angles of the image in the inertial measurement unit evaluation method of the first embodiment;

[0025] Figure 7C A schematic diagram illustrating a reference line for determining the left and right tilt angles of an image in the inertial measurement unit evaluation method of the first embodiment;

[0026] Figure 7D and Figure 7E Schematic diagrams respectively illustrating the vanishing points of the left image and the right image used to determine the left and right tilt angles of the image in the inertial measurement unit evaluation method of the first embodiment; and

[0027] Figure 7F FIG. 1 is a schematic diagram illustrating the left-right tilt angle of an image in the inertial measurement unit evaluation method according to the first embodiment.

[0028] The description of the accompanying drawings is as follows:

[0029] 100: Inertial Measurement Unit Evaluation Methodology

[0030] 110: Reference line definition steps

[0031] 120: Image Capture Steps

[0032] 140: Vanishing Line Calculation Steps

[0033] 160: Image Parameter Determination Steps

[0034] 170: Unit parameter acquisition steps

[0035] 180: Parameter comparison step

[0036] 190: Calibration Requirement Steps

[0037] 200: Inertial Measurement Unit Evaluation System

[0038] 210: Left camera

[0039] 220: Right camera

[0040] 230: Inertial Measurement Unit

[0041] 280: Storage Media

[0042] 282: Inertial Measurement Unit Evaluation Procedure

[0043] 290: Processor

[0044] 351, 361, 371, 451, 461, 471: Left image

[0045] 352, 362, 372, 452, 462, 472: Right image

[0046] 353, 363, 373, 453, 463, 473: First left lane

[0047] 354, 364, 374, 454, 464, 474: Second left lane

[0048] 355, 365, 375, 455, 465, 475: First right lane line

[0049] 356, 366, 376, 456, 466, 476: Second right lane line

[0050] 357, 367, 377, 457, 467, 477: the first vanishing point

[0051] 358, 358t, 368, 368t, 378, 378t, 458, 458t, 468, 468t, 478: Second Vanishing Point

[0052] 359, 369, 379: Reference lines

[0053] 459, 469: Vanishing Line

[0054] 511, 521, 531, 541, 551, 561: Images

[0055] 548: Triangle frame

[0056] 563: Left lane line

[0057] 565: Right lane line

[0058] 567: Vanishing Point

[0059] 670: Imaging surface

[0060] 800: Self-driving car

[0061] 870: Longitudinal center plane

[0062] 880: front

[0063] A, B, C: Vertex

[0064] b: distance between the center lines of the two cameras

[0065] d6: Difference

[0066] d7: absolute value

[0067] f: focal length

[0068] O1, O2: Optical Center

[0069] Oc, O: origin

[0070] P: point to be measured

[0071] P': image point to be measured

[0072] P1, P2: imaging points

[0073] s0: depth distance

[0074] s6, s7: disappearing distance

[0075] X: Horizontal direction

[0076] Xc, Yc, Zc: direction

[0077] Xp, Yp, Xp1, Yp1, Xp2, Yp2, Xcp, Ycp, Zcp: coordinate values

[0078] Y: vertical direction

[0079] Z: Normal direction

[0080] Φ: Image rollover angle

[0081] ω: image up and down tilt angle

[0082] τ: image left and right tilt angle DETAILED DESCRIPTION

[0083] Figure 1 FIG. 1 is a flow chart of an inertial measurement unit evaluation method 100 according to a first embodiment of the present invention. Figure 2AFIG. 1 is a block diagram of an inertial measurement unit evaluation system 200 according to a second embodiment of the present invention. Figure 2B 、 Figure 2C and Figure 2D Draw separately Figure 2A Schematic diagrams of the top, side and front views of the self-driving car 800 in which the inertial measurement unit evaluation system 200 is installed. Figures 1 to 2D The inertial measurement unit evaluation method 100 of the first embodiment is described using an inertial measurement unit evaluation system 200 according to a second embodiment of the present invention. The inertial measurement unit evaluation method 100 is used to evaluate the accuracy of an inertial measurement unit 230 included in a self-driving car 800. The self-driving car 800 further includes a left camera 210 and a right camera 220. Both the left camera 210 and the right camera 220 face forward of the self-driving car 800. For example, the left camera 210 and the right camera 220 may be mounted on the front windshield 880 of the self-driving car 800 so as to face forward. The inertial measurement unit evaluation method 100 includes an image capture step 120, a vanishing line calculation step 140, an image parameter determination step 160, a unit parameter acquisition step 170, and a parameter comparison step 180. Furthermore, the self-driving car 800 may be an autonomous driving vehicle, or may be a fully self-driving, semi-self-driving, or assisted driving vehicle, such as an autonomous car or an unmanned guided vehicle (AGV).

[0084] Figure 5D and Figure 5E Schematic diagrams illustrating a first vanishing point 457 and a second vanishing point 458 of a left image 451 and a right image 452 for determining a vanishing line 459 of an image roll tilt angle Φ in the inertial measurement unit evaluation method 100 according to the first embodiment are respectively shown. Figure 5F A schematic diagram illustrating a vanishing line 459 for determining the image roll tilt angle Φ in the inertial measurement unit evaluation method 100 of the first embodiment is shown. Figures 5D to 5FThe image capturing step 120 includes capturing a left image 451 and a right image 452 at a time point by the left camera 210 and the right camera 220, respectively. The images may be image frames. The left image 451 includes a first left lane line 453, a first right lane line 455, and a first vanishing point 457. The first vanishing point 457 is the intersection of the first left lane line 453 and the first right lane line 455. The right image 452 includes a second left lane line 454, a second right lane line 456, and a second vanishing point 458. The second vanishing point 458 is the intersection of the second left lane line 454 and the second right lane line 456. The vanishing line calculating step 140 includes calculating a vanishing line equation for a vanishing line 459. The vanishing line 459 is a line connecting the first vanishing point 457 and the second vanishing point 458, or the second vanishing point 458t after conversion to the image coordinate system, and is as shown in FIG. Figure 5F shown.

[0085] Please refer to Figure 1 Image parameter determination step 160 includes determining an image parameter set for self-driving car 800 based on a vanishing line equation. Unit parameter acquisition step 170 includes acquiring a unit parameter set (i.e., an inertial measurement unit parameter set) for self-driving car 800 via inertial measurement unit 230. Parameter comparison step 180 includes comparing the image parameter set and the unit parameter set and generating a comparison result. In response to the increasing demand for image processing in vehicles, the number of cameras has shifted from a single to multiple cameras. Furthermore, the present invention utilizes lane recognition technology, which does not place an excessive burden on the hardware and computing system of self-driving car 800, facilitating system integration and effectively evaluating the accuracy of inertial measurement unit 230.

[0086] Figure 3A 、 Figure 3B 、 Figure 3C 、 Figure 3D 、 Figure 3E and Figure 3F Draw separately Figure 1 Schematic diagrams of images 511, 521, 531, 541, 551 and 561 corresponding to the lane line recognition sub-step of the image capture step 120. Figures 3A to 3FThe lane line recognition sub-step in the image capture step 120 uses the Hough Transform feature detection method. First, the left camera 210 or the right camera 220 captures the raw data (YUV) image 511. Then, grayscale conversion is performed and the image coordinate system is defined, as shown in image 521. Edge processing is performed using the Canny algorithm, as shown in image 531. Next, a region of interest (ROI) is set, such as the triangle box 548 in image 541. It is formed by connecting the bottom line of image 541 with two diagonal lines. The inside of triangle box 548 is the recognition area, and the outside of triangle box 548 is the non-recognition area. The outside of triangle box 548, such as image 551, is filled with a specific color block (e.g., black). Next, lane markings are identified using the Hough transform feature detection method, such as left lane marking 563 and right lane marking 565 in image 561. Multiple coordinate values for left lane marking 563 and right lane marking 565 are obtained. The coordinate values of any two points on the lines can be used to determine the slope and equation of the line, and finally, the vanishing point 567 at the intersection of left lane marking 563 and right lane marking 565 is determined. For example, the equation of the line for left lane marking 563 is Y = a1 × X + b1, and the equation of the line for right lane marking 565 is Y = a2 × X + b2, where a1, a2, b1, and b2 are coefficients. Vanishing point 567 is represented as (K1, K2).

[0087] Furthermore, the vanishing point is the intersection point where all parallel lines in the camera coordinate system of three-dimensional space intersect. The two lanes are parallel in the camera coordinate system of three-dimensional space, but in the image coordinate system of two-dimensional space, the lines of the two lanes will eventually intersect, and the intersection point is the vanishing point. For example Figure 5D The coordinates of the first vanishing point 457 can be expressed as (K1, K2). Furthermore, the union of all vanishing points constitutes a vanishing line.

[0088] In vanishing line calculation step 140, the image coordinate system of left image 451 and the image coordinate system of right image 452 are converted into the other image coordinate system. The horizontal coordinate of the other image coordinate system is represented as X and the vertical coordinate is represented as Y. The vanishing line equation is Y = a × X + b, where a and b are coefficients. This coordinate system conversion and subsequent calculation steps help reduce the amount of computation required and accurately determine the three-axis angles (i.e., the three-axis tilt angles or the three-axis rotation angles) of self-driving car 800 using dual images.

[0089] Figure 4A and Figure 4B Draw separately Figure 1 Refer to the reference diagram for the image coordinate system transformation in the vanishing line calculation step 140. Figure 4A,Furthermore, the camera coordinate system (Xc, Yc, Zc) is converted to the image coordinate system (X, Y), which is converted from three-dimensional space to two-dimensional space, which belongs to the perspective projection relationship, such as Figure 4A As shown, the camera coordinate system is the coordinate system of the directions (axes) Xc, Yc, Zc and the origin Oc, and the image coordinate system is the coordinate system of the horizontal direction X, the vertical direction Y and the origin O. Figure 4A The measured point P, focal length f and vertices A, B and C are shown. The coordinate value of the measured point P in the camera coordinate system is P(Xcp, Ycp, Zcp) and the coordinate value of the image measured point P' converted to the image coordinate system is P'(Xp, Yp). Figure 4A There are triangular geometric relationships ΔABOc~ΔOCOc, ΔPBOc~ΔP'COc, so there are further line segment geometric relationships of the following equations (1), (2), and (3), as well as the conversion relationship from the camera coordinate system to the image coordinate system of equation (4):

[0090]

[0091]

[0092] as well as

[0093]

[0094] Please refer to Figure 4B , which illustrates a schematic diagram of dual cameras (eg, a left camera 210 and a right camera 220) expressed in an image coordinate system. Figure 4B The diagram shows the optical centers O1 and O2 of the dual cameras, the image point P to be measured (i.e., the coordinate value in the image coordinate system is P'(Xp, Yp)), the imaging point P1 (coordinate value (Xp1, Yp1)) and P2 (coordinate value (Xp2, Yp2)) on the imaging surface 670 of the dual camera's photoreceptor, the focal length f, the centerline (Baseline) distance b of the dual cameras, and the depth distance s0. Figure 4B The following line segment geometric relationship is given by equation (5), the coordinate relationship of the imaging points P1 and P2 in the horizontal direction X is given by equation (6), the coordinate relationship of the imaging points P1 and P2 in the vertical direction Y is given by equation (7), and the depth distance s0 is given by equation (8):

[0095]

[0096] Xp2 = ((Xpb) / Xp) × Xp1 = M × Xp1 (6);

[0097] Y p 2 = Y p 1 Formula (7): and

[0098] s 0 = f × b / (X p 1 - X p 2) Formula (8).

[0099] Figure 5A and Figure 5B Schematic diagrams illustrating a first vanishing point 357 and a second vanishing point 358 of a left image 351 and a right image 352 of a reference line 359 for determining an image roll tilt angle Φ in the inertial measurement unit evaluation method 100 according to the first embodiment are respectively shown. Figure 5C A schematic diagram illustrating a reference line 359 for determining the image rollover tilt angle Φ in the inertial measurement unit evaluation method 100 of the first embodiment is shown. Figures 5A to 5C The inertial measurement unit evaluation method 100 may further include a reference line definition step 110. When the self-driving car 800 is in a non-tilted state at another time point (e.g., determined by the inertial measurement unit 230), the other time point is earlier than the aforementioned time point. The left image 351 and the right image 352 are captured by the left camera 210 and the right camera 220 at the other time point, respectively. The left image 351 includes a first left lane line 353, a first right lane line 355, and a first vanishing point 357, and the right image 352 includes a second left lane line 354, a second right lane line 356, and a second vanishing point 358. Then, the second vanishing point 358 in the image coordinate system of the right image 352 is converted to the second vanishing point 358t in the image coordinate system of the left image 351 (e.g., Figure 5C As shown, a vanishing line is formed by connecting first vanishing point 357 and second vanishing point 358t in left image 351. This line has a vanishing line equation and is defined as reference line 359 and its reference line equation. The reference line equation is Y=K5, where K5 is a constant. Using this reference line in a non-tilted state and subsequent calculation steps helps reduce computational complexity and accurately determine the three-axis angles of self-driving car 800 using dual images.

[0100] Furthermore, in the camera coordinate system, when the camera is horizontal and not tilted, the slope is equal to 0. Because the slope = 0, a single vanishing point coordinate value can be used to obtain the vanishing line equation, which is expressed as Y = K0 (or other constant symbols), and can be used as the original straight line equation for the three axes of the vehicle body in the camera coordinate system. Conversely, when the camera is tilted (the slope is not equal to 0), the intersection of the two lane lines in the two-dimensional image coordinate system is the vanishing point. Although a single vanishing point coordinate value can be obtained, the vanishing line equation cannot be calculated because the coordinate value of another point on the vanishing line is required to calculate the vanishing line equation.

[0101] Figure 5G A schematic diagram illustrating the image rollover tilt angle Φ in the inertial measurement unit evaluation method 100 of the first embodiment is shown. Figure 1 、 Figures 5C to 5GIn the image capturing step 120 after the reference line defining step 110 , the left image 451 and the right image 452 are captured by the left camera 210 and the right camera 220 at a time point, respectively.

[0102] In the vanishing line calculation step 140, the second vanishing point 458 in the image coordinate system of the right image 452 is converted to the second vanishing point 458t in the image coordinate system of the left image 451 (e.g., Figure 5F As shown), the first vanishing point 457 and the second vanishing point 458t in the left image 451 are connected to form a vanishing line 459, which has a vanishing line equation Y=a5×X+b5, where a5 and b5 are coefficients.

[0103] In the image parameter determination step 160, the image parameter set may include an image roll tilt angle Φ, such as Figure 5G As shown, the image rollover angle Φ is Figure 5F The vanishing line 459 and Figure 5C The angle between the vanishing line equation Y = a5 × X + b5 and the reference line equation Y = K5 is the image rollover tilt angle Φ, which is the angle at which self-driving car 800 tilts clockwise or counterclockwise relative to the normal direction Z. In the unit parameter acquisition step 170, the unit parameter set includes the unit rollover tilt angle, which corresponds to the image rollover tilt angle Φ. In the parameter comparison step 180, the image rollover tilt angle Φ is compared with the unit rollover tilt angle. This allows the image rollover tilt angle Φ to be accurately calculated for comparison with the unit rollover tilt angle.

[0104] Figure 6A and Figure 6B Schematic diagrams illustrating a first vanishing point 367 and a second vanishing point 368 of a left image 361 and a right image 362 of a reference line 369 for determining an image vertical tilt angle ω in the inertial measurement unit evaluation method 100 according to the first embodiment are respectively shown. Figure 6C FIG. 3 is a schematic diagram illustrating a reference line 369 for determining an image vertical tilt angle ω in the inertial measurement unit evaluation method 100 according to the first embodiment. Figure 6D and Figure 6E Schematic diagrams illustrating a first vanishing point 467 and a second vanishing point 468 of a left image 461 and a right image 462 for determining a vanishing line 469 of an image vertical tilt angle ω in the inertial measurement unit evaluation method 100 according to the first embodiment are respectively shown. Figure 6F FIG. 4 is a schematic diagram illustrating a vanishing line 469 for determining an image vertical tilt angle ω in the inertial measurement unit evaluation method 100 according to the first embodiment. Figure 6G FIG. 1 is a schematic diagram illustrating parameters for determining the vertical tilt angle ω of an image in the inertial measurement unit evaluation method 100 according to the first embodiment. Figure 6HA schematic diagram illustrating the image vertical tilt angle ω in the inertial measurement unit evaluation method 100 of the first embodiment is shown. Figure 1 、 Figures 6A to 6H In the reference line definition step 110, when the self-driving car 800 is in a non-tilted state, the left image 361 and the right image 362 are captured by the left camera 210 and the right camera 220, respectively. The left image 361 includes a first left lane line 363, a first right lane line 365, and a first vanishing point 367, and the right image 362 includes a second left lane line 364, a second right lane line 366, and a second vanishing point 368. Then, the second vanishing point 368 in the image coordinate system of the right image 362 is converted to the second vanishing point 368t in the image coordinate system of the left image 361 (as shown in FIG. Figure 6C As shown), the first vanishing point 367 and the second vanishing point 368t in the left image 361 are connected to form a reference line 369, which has a reference line equation Y=K62, where K62 is a constant.

[0105] In the image capture step 120 after the reference line definition step 110, the left image 461 and the right image 462 are captured by the left camera 210 and the right camera 220, respectively. The left image 461 includes a first left lane line 463, a first right lane line 465, and a first vanishing point 467, and the right image 462 includes a second left lane line 464, a second right lane line 466, and a second vanishing point 468. In the vanishing line calculation step 140, the second vanishing point 468 in the image coordinate system of the right image 462 is converted to the second vanishing point 468t in the image coordinate system of the left image 461 (e.g., Figure 6F As shown), the first vanishing point 467 and the second vanishing point 468t in the left image 461 are connected to form a vanishing line 469, which has a vanishing line equation Y=K63, where K63 is a constant.

[0106] In the vanishing line calculation step 140, the vanishing distance s6 between the vanishing line 469 and the normal direction Z of one of the left camera 210 and the right camera 220 can be further calculated (e.g., Figure 6H ), which is the distance between one of the left camera 210 and the right camera 220 and the first vanishing point 467, and the horizontal direction X, the vertical direction Y and the normal direction Z are perpendicular to each other.

[0107] In the image parameter determination step 160, the image parameter set may include an image vertical tilt angle ω, such as Figure 6HAs shown, the image vertical tilt angle ω is derived based on the difference d6 between vanishing line 469 and reference line 369 in the longitudinal direction Y and the vanishing distance s6, i.e., d6 = K63 - K62. The image vertical tilt angle ω, representing the upward or downward angle of the self-driving car 800, can be calculated using tan(ω) = d6 / s6. In the unit parameter acquisition step 170, the unit parameter set includes the unit vertical tilt angle, which corresponds to the image vertical tilt angle ω. In the parameter comparison step 180, the image vertical tilt angle ω is compared with the unit vertical tilt angle. This allows the image vertical tilt angle ω to be accurately calculated for comparison with the unit vertical tilt angle.

[0108] Figure 7A and Figure 7B Schematic diagrams illustrating a first vanishing point 377 and a second vanishing point 378 of a left image 371 and a right image 372 of a reference line 379 for determining a left-right tilt angle τ of an image in the inertial measurement unit evaluation method 100 of the first embodiment are respectively shown. Figure 7C FIG. 3 is a schematic diagram illustrating a reference line 379 for determining the left-right tilt angle τ of an image in the inertial measurement unit evaluation method 100 according to the first embodiment. Figure 7D and Figure 7E Schematic diagrams illustrating a first vanishing point 477 and a second vanishing point 478 of a left image 471 and a right image 472 for determining the left-right tilt angle τ of the image in the inertial measurement unit evaluation method 100 of the first embodiment are respectively shown. Figure 7F A schematic diagram illustrating the left and right tilt angle τ of an image in the inertial measurement unit evaluation method 100 of the first embodiment is shown. Figure 1 、 7A to 7F In the reference line definition step 110, when the self-driving car 800 is in a non-tilted state, the left image 371 and the right image 372 are captured by the left camera 210 and the right camera 220, respectively. The left image 371 includes a first left lane line 373, a first right lane line 375, and a first vanishing point 377, and the right image 372 includes a second left lane line 374, a second right lane line 376, and a second vanishing point 378. Then, the second vanishing point 378 in the image coordinate system of the right image 372 is converted to the second vanishing point 378t in the image coordinate system of the left image 371 (as shown in FIG. Figure 7C As shown in FIG3 , the coordinate value of the first vanishing point 377 in the left image 371 is (Xp1, Yp1), and the coordinate value of the second vanishing point 378t after conversion is (Xp2, Yp2). The first vanishing point 377 and the second vanishing point 378t in the left image 371 are connected to form a reference line 379, which has a reference line equation Y=K7, where K7 is a constant. It should be noted that in practice, it can be defined as Figure 5C 、 Figure 6C 、 Figure 7C Any reference line in the figure is a reference line in other figures, that is, the reference lines in the figures and their reference line equations are the same.

[0109] In the image capture step 120 following the reference line definition step 110, a left image 471 and a right image 472 are captured by the left camera 210 and the right camera 220, respectively. The left image 471 includes a first left lane line 473, a first right lane line 475, and a first vanishing point 477, and the right image 472 includes a second left lane line 474, a second right lane line 476, and a second vanishing point 478. In the vanishing line calculation step 140, the second vanishing point 478 in the image coordinate system of the right image 472 is converted to a second vanishing point (not shown) in the image coordinate system of the left image 471. The coordinate value of the first vanishing point 477 in the left image 471 is (Xq1, Yq1), and the coordinate value of the converted second vanishing point in the left image 471 is (Xq2, Yq2). The vanishing distance s7 (e.g., the vanishing distance s7) between the vanishing line and the normal direction Z parallel to one of the left camera 210 and the right camera 220 is calculated. Figure 7F shown).

[0110] In the image parameter determination step 160, the image parameter set may include the left and right tilt angles τ of the image, such as Figure 7F As shown, the image left-right tilt angle τ is derived based on the absolute value d7 of the difference in the horizontal direction X between the second vanishing point after transformation in the left image 471 and the second vanishing point 378t after transformation in the left image 371 used to define the reference line 379, i.e., d7 = |Xp2 - Xq2|. The image left-right tilt angle τ, representing the angle at which the self-driving car 800 tilts left or right, can be calculated using tan(τ) = d7 / s7. In the unit parameter acquisition step 170, the unit parameter set includes a unit left-right tilt angle, which corresponds to the image left-right tilt angle τ. In the parameter comparison step 180, the image left-right tilt angle τ and the unit left-right tilt angle are compared. This allows the image left-right tilt angle τ to be accurately calculated for comparison with the unit left-right tilt angle.

[0111] In practice, at a given point in time, two or more of the three-axis angles (i.e., image roll tilt angle Φ, image vertical tilt angle ω, and image horizontal tilt angle τ) calculated by the inertial measurement unit evaluation method 100 for the self-driving car 800 may be non-zero. For example, the clockwise image roll tilt angle Φ and the upward image vertical tilt angle ω may coexist, or the upward image vertical tilt angle ω and the leftward image horizontal tilt angle τ may coexist, or the counterclockwise image roll tilt angle Φ, the downward image vertical tilt angle ω, and the rightward image horizontal tilt angle τ may coexist, without limitation. Furthermore, once the vanishing line equation Y = a5 × X + b5 associated with the image roll tilt angle Φ is determined, the image roll tilt angle Φ can be determined, and the image vertical tilt angle ω and the image horizontal tilt angle τ can be further calculated using the vanishing line equation Y = a5 × X + b5 as a reference.

[0112] Please refer to Figure 1 The IMU evaluation method 100 may further include a calibration request step 190 , in which, when the comparison result satisfies a threshold condition, a calibration of the IMU 230 is requested, i.e., a signal is generated and transmitted requesting calibration of the IMU 230. In this manner, the IMU evaluation method 100 may be used to assess whether to further perform computationally intensive calibration (e.g., using a Kalman filter mathematical operation) on the IMU 230. If the comparison result does not satisfy the threshold condition, i.e., if the IMU evaluation method 100 assesses that the unit parameter set measured by the IMU 230 is within an acceptable accuracy range, the computationally intensive calibration may be omitted, while also ensuring the accuracy of the calibration of the IMU 230.

[0113] Please refer to Figures 2A to 2D The inertial measurement unit evaluation system 200 is disposed in the self-driving car 800 . The inertial measurement unit evaluation system 200 includes an inertial measurement unit 230 , a left camera 210 , a right camera 220 , a storage medium 280 , and a processor 290 .

[0114] Left camera 210 faces forward of self-driving car 800, while right camera 220 faces forward. Storage medium 280 provides an inertial measurement unit evaluation program 282. Processor 290 is communicatively coupled to inertial measurement unit 230, left camera 210, right camera 220, and storage medium 280. Based on inertial measurement unit evaluation program 282, processor 290 uses left camera 210 and right camera 220 to capture a left image 451 and a right image 452 at a point in time. Left image 451 includes a first left lane marking 453, a first right lane marking 455, and a first vanishing point 457, which is the intersection of first left lane marking 453 and first right lane marking 455. Right image 452 includes a second left lane marking 454, a second right lane marking 456, and a second vanishing point 458, which is the intersection of second left lane marking 454 and second right lane marking 456. Based on the IMU evaluation program 282, the processor 290 further calculates a vanishing line 459, which is a line connecting the first vanishing point 457 and the second vanishing point 458, or the second vanishing point 458t after conversion to the image coordinate system. Based on the vanishing line, the processor 290 determines an image parameter set for the self-driving vehicle 800. The processor 290 obtains a unit parameter set for the self-driving vehicle 800 through the IMU 230, compares the image parameter set with the unit parameter set, and generates a comparison result. This allows the IMU evaluation system 200 to assess whether to perform further computationally intensive calibration on the IMU 230.

[0115] Please refer to Figure 2B and Figure 2D, the left camera 210 and the right camera 220 can be symmetrical about the virtual longitudinal center plane 870 of the self-driving car 800. This helps reduce the computational complexity required by the inertial measurement unit evaluation process 282. Furthermore, the left and right cameras according to the present invention can be dual cameras on the same device, or each can be an independent single camera device.

[0116] For other details of the inertial measurement unit evaluation system 200 of the second embodiment, reference may be made to the inertial measurement unit evaluation method 100 of the first embodiment, and will not be described in detail here.

[0117] Although the present invention has been disclosed above in terms of embodiments, this is not intended to limit the present invention. Anyone skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for evaluating the accuracy of an inertial measurement unit (IMU) included in a self-driving car, wherein the self-driving car further includes a left camera and a right camera, wherein the left camera and the right camera both face forward of the self-driving car, wherein: The inertial measurement unit evaluation method includes: an image capturing step of capturing a left image and a right image at a time point using the left camera and the right camera, respectively, wherein the left image includes a first left lane line, a first right lane line, and a first vanishing point, the first vanishing point being the intersection of the first left lane line and the first right lane line; and the right image includes a second left lane line, a second right lane line, and a second vanishing point, the second vanishing point being the intersection of the second left lane line and the second right lane line; a vanishing line calculation step of calculating a vanishing line equation of a vanishing line, the vanishing line being a line connecting the first vanishing point and the second vanishing point; an image parameter determination step, determining an image parameter set of the self-driving car according to the vanishing line equation; a unit parameter acquisition step, obtaining a unit parameter set of the self-driving car through the inertial measurement unit; and A parameter comparison step compares the image parameter set and the unit parameter set and generates a comparison result.

2. The inertial measurement unit evaluation method according to claim 1, wherein: In the vanishing line calculation step, one of the image coordinate systems of the left image and the right image is converted into another image coordinate system. A horizontal coordinate of the other image coordinate system is represented by X and a vertical coordinate is represented by Y. The vanishing line equation is Y=a×X+b, where a and b are coefficients.

3. The inertial measurement unit evaluation method according to claim 2, wherein: Also includes: A reference line definition step, when the self-driving car is in a non-tilted state at another time point, the another time point being earlier than the time point, calculating another vanishing line equation of another vanishing line using another left image and another right image captured by the left camera and the right camera at the another time point, and defining the another vanishing line and the another vanishing line equation as a reference line and a reference line equation, the reference line equation being Y=K0, where K0 is a constant.

4. The inertial measurement unit evaluation method according to claim 3, wherein: In the image parameter determination step, the image parameter set includes an image roll tilt angle, which is the angle between the vanishing line and the reference line; Wherein, in the unit parameter acquisition step, the unit parameter group includes a unit rollover tilt angle, which corresponds to the rollover tilt angle of the image; In the parameter comparison step, the image rollover tilt angle and the unit rollover tilt angle are compared.

5. The inertial measurement unit evaluation method according to claim 3, wherein: In the vanishing line calculation step, a vanishing distance of the vanishing line in a normal direction parallel to one of the left camera and the right camera is further calculated, and the horizontal direction, the vertical direction, and the normal direction are perpendicular to each other; Wherein, in the image parameter determination step, the image parameter set includes an image vertical tilt angle, which is obtained based on the difference between the vanishing line and the reference line in the vertical direction and the vanishing distance; Wherein, in the unit parameter acquisition step, the unit parameter group includes a unit up-down tilt angle, which corresponds to the up-down tilt angle of the image; In the parameter comparison step, the image up-down tilt angle and the unit up-down tilt angle are compared.

6. The inertial measurement unit evaluation method according to claim 3, wherein: In the vanishing line calculation step, a vanishing distance of the vanishing line in a normal direction parallel to one of the left camera and the right camera is further calculated, and the horizontal direction, the vertical direction, and the normal direction are perpendicular to each other; wherein, in the image parameter determination step, the image parameter set includes an image left-right tilt angle, which is obtained based on an absolute value of a difference in the horizontal direction between the first vanishing point and another first vanishing point used to define the reference line, or based on an absolute value of a difference in the horizontal direction between the second vanishing point and another second vanishing point used to define the reference line; Wherein, in the unit parameter acquisition step, the unit parameter group includes a unit left-right tilt angle, which corresponds to the left-right tilt angle of the image; In the parameter comparison step, the left-right tilt angle of the image and the left-right tilt angle of the unit are compared.

7. The inertial measurement unit evaluation method according to claim 1, wherein: Also includes: A calibration requesting step requests calibration of the inertial measurement unit when the comparison result satisfies a threshold condition.

8. An inertial measurement unit evaluation system, characterized in that Set up in a self-driving car, the inertial measurement unit evaluation system includes: an inertial measurement unit; a left camera facing a front direction of the self-driving car; a right camera, facing the front direction; a storage medium providing an inertial measurement unit evaluation program; as well as a processor communicatively coupled to the inertial measurement unit, the left camera, the right camera, and the storage medium; The processor is configured to: Capturing a left image and a right image by the left camera and the right camera, respectively, at a time point, wherein the left image includes a first left lane line, a first right lane line, and a first vanishing point, the first vanishing point being the intersection of the first left lane line and the first right lane line; and the right image includes a second left lane line, a second right lane line, and a second vanishing point, the second vanishing point being the intersection of the second left lane line and the second right lane line; Calculating a vanishing line, which is a line connecting the first vanishing point and the second vanishing point; determining an image parameter set of the self-driving car according to the vanishing line; Obtaining a unit parameter set of the self-driving car through the inertial measurement unit; and The image parameter set and the unit parameter set are compared to generate a comparison result.

9. The inertial measurement unit evaluation system of claim 8, wherein: The left camera and the right camera are symmetrical to a longitudinal center plane of the self-driving car.

10. The inertial measurement unit evaluation system of claim 8, wherein: The processor is further configured to: When the comparison result satisfies a threshold condition, a calibration of the inertial measurement unit is requested to be performed.

Images