System for calibrating external parameters of a camera in an autonomous transportation vehicle

Abstract

A system for determining calibrated camera extrinsic parameters of an autonomous transportation vehicle includes a camera mounted to the autonomous transportation vehicle that collects image data including a plurality of image frames. The system also includes one or more autonomous driving controllers in electronic communication with the camera, the one or more autonomous driving controllers executing instructions to determine a transportation vehicle pose estimate based on a location and movement of the autonomous transportation vehicle by a localization algorithm. The one or more autonomous driving controllers determine the calibrated camera extrinsic parameters based on three-dimensional coordinates of particular feature points of interest corresponding to two consecutive image frames, the particular feature points of interest corresponding to the two consecutive image frames, and a camera pose corresponding to the two consecutive image frames.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a system for calibrating camera extrinsic parameters of a particular camera in an autonomous transportation vehicle based on a transportation vehicle pose estimate determined by a localization algorithm, initial extrinsic parameters of the particular camera, and by tracking feature points between two consecutive image frames collected by the particular camera. BACKGROUND

[0002] Autonomous transportation vehicles can employ various sensing technologies to collect data related to its surrounding environment, such as but not limited to radar, laser, global positioning system (GPS), and cameras. For example, cameras are mounted along the front, back, and sides of the autonomous transportation vehicle for observing the surrounding environment. The cameras generate readings represented in a respective camera coordinate system, where the readings provided by the cameras are converted into readings represented in a world coordinate system for alignment with the autonomous transportation vehicle. Cameras include their own unique intrinsic and extrinsic parameters. Intrinsic parameters are internal and fixed for a particular camera, indicating the optical center and focal length of the particular camera. Extrinsic parameters define the position and orientation of the camera coordinate system relative to the position of the world coordinate system and are used to convert readings represented in the camera coordinate system to the world coordinate system.

[0003] Extrinsic parameters are estimated dynamically, which can result in a high computational load that requires a large amount of system resources. In one method to reduce the computational load, extrinsic parameters can be determined during offline manufacturing calibration performed at the final assembly plant. While offline manufacturing calibration is accurate, this solution can not be widely used since offline manufacturing calibration is expensive and can be inconvenient for users. Furthermore, existing online calibration methods for estimating camera extrinsic parameters can be repetitive and unnecessarily waste computational resources since existing methods use the same inertial and GPS sensor measurements as the localization module to estimate the transportation vehicle pose.

[0004] Accordingly, while current autonomous transportation vehicle systems achieve their intended purpose, there is a need for an improved method for estimating camera extrinsic parameters.

[0005] SUMMARY

[0006] According to aspects, a system for determining calibrated camera extrinsic parameters of an autonomous transportation vehicle is disclosed. The system includes a camera mounted to the autonomous transportation vehicle and one or more autonomous driving controllers in electronic communication with the camera, the camera collecting image data including a plurality of image frames. The one or more autonomous driving controllers execute instructions to determine a transportation vehicle pose estimate based on a position and movement of the autonomous transportation vehicle by a localization algorithm. The one or more autonomous driving controllers execute instructions to determine a camera pose of the camera based on the transportation vehicle pose estimate and initial camera extrinsic parameters. The one or more autonomous driving controllers execute instructions to identify a particular feature of interest between two consecutive image frames as part of the plurality of image frames. The one or more autonomous driving controllers execute instructions to perform a triangulation algorithm to determine three-dimensional coordinates of the particular feature of interest corresponding to the two consecutive image frames. The one or more autonomous driving controllers execute instructions to perform a non-linear optimization algorithm to determine the calibrated camera extrinsic parameters based on the three-dimensional coordinates of the particular feature of interest corresponding to the two consecutive image frames, the particular feature of interest corresponding to the two consecutive image frames, and the camera pose corresponding to the two consecutive image frames.

[0007] In an aspect, identifying the particular feature of interest between the two consecutive image frames further includes performing a feature detection algorithm to detect the particular feature of interest in the two consecutive image frames, tracking the feature detected in the immediately preceding image frame with the current image frame, and selecting the particular feature of interest between the two consecutive image frames.

[0008] In another aspect, the one or more autonomous driving controllers execute instructions to determine a motion change between the particular feature of interest between the two consecutive image frames.

[0009] In yet another aspect, the one or more autonomous driving controllers execute instructions to compare the motion change between the particular feature of interest between the two consecutive image frames to a threshold amount of motion change. The threshold amount of motion change is a predefined tuning parameter that selects two image frames as part of a sliding window. The one or more autonomous driving controllers execute instructions to determine that the motion change between the particular feature of interest between the two consecutive image frames is equal to or greater than the threshold amount of motion change.

[0010] In an aspect, performing the non-linear optimization algorithm includes minimizing re-projection errors with respect to the calibrated camera extrinsic parameters and the three-dimensional coordinates of the particular feature of interest while solving a cost function.

[0011] In another aspect, the one or more autonomous driving controllers execute instructions to save results of solving the nonlinear optimization algorithm to a memory, clear a sliding window containing consecutive image frames, re-determine calibrated camera extrinsic parameters, and after obtaining a threshold number of calibrated camera extrinsic parameters, perform one or more filtering and averaging operations.

[0012] In yet another aspect, the camera pose is determined by transforming the initial camera extrinsic parameters from the transportation vehicle body coordinate system to the world coordinate system.

[0013] In an aspect, the initial camera extrinsic parameters are saved in a memory of the one or more autonomous driving controllers.

[0014] In another aspect, the initial camera extrinsic parameters represent an initial approximation of actual extrinsic parameters of the camera.

[0015] In yet another aspect, the initial camera extrinsic parameters are represented based on the transportation vehicle body coordinate system, and the initial camera extrinsic parameters include an initial translation vector and an initial rotation matrix.

[0016] In an aspect, the particular feature points of interest corresponding to two consecutive image frames are represented based on a two-dimensional image plane coordinate system.

[0017] In another aspect, the three-dimensional coordinates of the particular feature points of interest are represented based on a world coordinate system.

[0018] In yet another aspect, the plurality of image frames are provided to the one or more autonomous driving controllers based on a sliding window buffering technique.

[0019] In an aspect, the transportation vehicle pose estimate includes a transportation vehicle translation vector and a transportation vehicle rotation matrix represented based on a world coordinate system.

[0020] In one aspect, a method for determining calibrated camera extrinsic parameters of an autonomous transportation vehicle is disclosed. The method includes determining, by one or more autonomous driving controllers, a transportation vehicle pose estimate based on a position and movement of the autonomous transportation vehicle by a localization algorithm. The method includes determining a camera pose of a camera mounted to the autonomous transportation vehicle based on the transportation vehicle pose estimate and initial camera extrinsic parameters, wherein the camera collects image data comprising a plurality of image frames. The method further includes identifying a particular feature point of interest between two consecutive image frames as part of the plurality of image frames. The method further includes executing a triangulation algorithm to determine three-dimensional coordinates of the particular feature point of interest corresponding to the two consecutive image frames. Finally, the method includes executing a non-linear optimization algorithm to determine the calibrated camera extrinsic parameters based on the three-dimensional coordinates of the particular feature point of interest corresponding to the two consecutive image frames, the particular feature point of interest corresponding to the two consecutive image frames, and the camera pose corresponding to the two consecutive image frames.

[0021] In one aspect, the method includes executing a feature detection algorithm to detect the particular feature point in the two consecutive image frames.

[0022] In another aspect, the method includes tracking the feature point detected in the immediately preceding image frame with the current image frame and selecting the particular feature point of interest between the two consecutive image frames.

[0023] In another aspect, the method includes determining a motion change between the particular feature point of interest between the two consecutive image frames.

[0024] In one aspect, the method includes comparing the motion change between the particular feature point of interest between the two consecutive image frames to a threshold amount of motion change, wherein the threshold amount of motion change is a predefined tuning parameter that selects two image frames as part of a sliding window, and determining that the motion change between the particular feature point of interest between the two consecutive image frames is equal to or greater than the threshold amount of motion change.

[0025] In another aspect, executing the non-linear optimization algorithm includes minimizing a re-projection error with respect to the calibrated camera extrinsic parameters and the three-dimensional coordinates of the particular feature point of interest while solving a cost function.

[0026] In yet another embodiment, the method includes saving a result of the non-linear optimization algorithm to a memory, clearing the sliding window containing the consecutive image frames, re-determining the calibrated camera extrinsic parameters, and performing one or more filtering and averaging operations after a threshold number of calibrated camera extrinsic parameters are obtained.

[0027] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0028] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way.

[0029] Figure 1 is a schematic diagram of an exemplary autonomous transportation vehicle including a disclosed system according to exemplary embodiments, wherein the system includes one or more autonomous driving controllers in electronic communication with a plurality of cameras;

[0030] Figure 2 is a schematic diagram of an exemplary autonomous transportation vehicle including a disclosed system according to exemplary embodiments, wherein the system includes one or more autonomous driving controllers in electronic communication with a plurality of cameras; Figure 1 is a block diagram of an autonomous driving controller for calibrating camera extrinsic parameters shown;

[0031] Figure 3 is a process flow diagram illustrating an exemplary method for calibrating camera extrinsic parameters according to exemplary embodiments. DETAILED DESCRIPTION

[0032] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0033] Referring to Figure 1 , an exemplary autonomous transportation vehicle 10 including a system 12 for calibrating camera extrinsic parameters is shown. The autonomous transportation vehicle 10 can be any type of transportation vehicle such as, but not limited to, a car, a truck, a sport utility vehicle, a van, or a recreational vehicle. In one non-limiting embodiment, the autonomous transportation vehicle 10 is a fully autonomous transportation vehicle including an autonomous driving system (ADS) for performing all driving tasks. Alternatively, in another embodiment, the autonomous transportation vehicle 10 is a semi-autonomous transportation vehicle including an advanced driver assistance system (ADAS) for assisting a driver with steering, braking, and / or acceleration. The system 12 includes an autonomous driving system 14 having one or more autonomous driving controllers 20. The autonomous driving controllers 20 are in electronic communication with a plurality of autonomous sensors 22 and a plurality of transportation vehicle systems 24. In the example shown, the autonomous driving controllers 20 are in electronic communication with a plurality of cameras 26, a plurality of lidar sensors 28, a plurality of radar sensors 30, and a plurality of ultrasonic sensors 32. The autonomous driving controllers 20 are also in electronic communication with a plurality of transportation vehicle systems 24 including a plurality of steering systems 34, a plurality of braking systems 36, and a plurality of acceleration systems 38. Figure 1In the illustrated example, the plurality of autonomous sensors 22 includes a plurality of cameras 30, a radar 32, an inertial measurement unit (IMU) 34, a global positioning system (GPS) 36, and a lidar 38, however, it should be appreciated that other sensors can also be used. The plurality of cameras 30 are mounted on the side 25, front 26, and rear 28 of the autonomous vehicle 10. The plurality of cameras 30 each capture image data 48 representative of the surrounding environment of the autonomous vehicle 10. The plurality of vehicle systems 24 include, but are not limited to, a braking system 40, a steering system 42, a power system 44, and a suspension system 46.

[0034] Figure 2 is a block diagram of the one or more autonomous driving controllers 20. Referring to Figure 1 and Figure 2 , the one or more autonomous driving controllers 20 determine a calibrated extrinsic camera parameter 50 associated with a particular camera 30 of the autonomous vehicle 10. The calibrated extrinsic camera parameter 50 converts readings represented in a camera coordinate system of the particular camera 30 to a world coordinate system. Specifically, the calibrated extrinsic camera parameter 50 includes a calibrated translation vector tB C and a calibrated rotation matrix RB C for converting readings in the camera coordinate system to the world coordinate system. As described below, the calibrated extrinsic camera parameter 50 is determined based on a localization algorithm performed by a localization module 60 that is part of the autonomous driving controller 20. The calibrated extrinsic camera parameter 50 is also based on initial extrinsic camera parameters 52 saved in memory of the one or more autonomous driving controllers 20 and determined by tracking feature points between two consecutive image frames 80 collected by the particular camera 30. It should be appreciated that the two consecutive image frames 80 can or can not be consecutive with respect to each other. Rather, the image frames 80 are consecutive so long as the image frames 80 are within the same sliding window 82. The sliding window 82 is described below and shown in Figure 2 , which contains five consecutive image frames 80.

[0035] Referring to Figure 1 and Figure 2 , the one or more autonomous driving controllers 20 include a localization module 60, a camera pose estimation module 62, a feature detection module 64, a feature tracking module 66, a feature selection module 68, a motion detection module 70, a triangulation module 72, an optimization module 74, and a filtering module 76. The localization module 60 receives image data 48 from the plurality of autonomous sensors 22 Figure 1) receives sensor data 58, where the sensor data 58 is indicative of a position and movement of the autonomous vehicle 10 for determining the vehicle pose estimate 54. The sensor data 58 can include, for example, angular velocities and accelerations collected by the IMU 34, a position of the autonomous vehicle 10 collected by the GPS 36, and wheel speeds collected by wheel speed sensors (not shown). The localization module 60 executes a localization algorithm to determine the vehicle pose estimate 54 based on the position and movement of the autonomous vehicle 10 indicated by the sensor data 58, where the vehicle pose estimate 54 includes a vehicle translation vector and a vehicle rotation matrix where the variable k represents a time stamp.

[0036] With continued reference to Figure 2 , the initial camera extrinsics 52 are stored in a memory of the one or more autonomous driving controllers 20. The initial camera extrinsics 52 are expressed in the vehicle body coordinate system and include an initial translation vector and an initial rotation matrix It should be appreciated that the initial camera extrinsics represent an initial approximation or initial guess of the actual camera extrinsics and that the values are fixed, thus requiring further calibration of the initial camera extrinsics before they can be used to transform readings expressed in the camera coordinate system of a particular camera 30 to the world coordinate system. The vehicle pose estimate 54 and the initial extrinsics 52 are sent to the camera pose estimation module 62.

[0037] The camera pose estimation module 62 determines a camera pose 84 of a particular camera 30 based on the vehicle pose estimate 54 determined by the localization module 60 and the initial camera extrinsics 52. Specifically, the camera pose estimation module 62 transforms the initial camera extrinsics 52 expressed in the vehicle body coordinate system to the world coordinate system. The camera pose 84 is expressed in the world coordinate system by a camera translation vector and a camera rotation matrix Specifically, the camera translation vector is determined based on the vehicle translation vector the initial translation vector and the vehicle rotation matrix and the camera translation vector is expressed in Equation 1 below: The camera rotation matrix is determined based on the vehicle rotation matrix the initial rotation matrix and the camera rotation matrix is expressed in Equation 2 below:

[0038]

[0039]

[0040] Continue to refer to Figure 2 The feature detection module 64 receives image data 48 from a specific camera 30, wherein the image data 48 includes a plurality of image frames 80 ordered by timestamps. Each image frame 80 is captured at a unique time, and therefore each image frame includes a unique timestamp. In one embodiment, one or more autonomous driving controllers 20 receive a plurality of image frames 80 based on a sliding window buffering technique. The sliding window buffering technique involves containing a current number of sequentially ordered image frames 80 in a storage buffer at a single time, which... Figure 2 The sliding window 82 is shown as a dashed line. Figure 2 In the example shown, the sliding window 82 contains five image frames 80, where the sliding window 82 represents a storage buffer. However, it should be understood that... Figure 2 This is merely an example; other numbers may be used. In one embodiment, the number of image frames that can be stored in the storage buffer at any given time must not exceed 10.

[0041] Feature detection module 64 executes a feature detection algorithm to detect feature points in two consecutive image frames 80, where the detected feature points are used as measurements to determine calibrated camera extrinsic parameters 50. For example, feature points may be keypoint features located at specific locations (e.g., building corners), or feature points may be edges representing object boundaries. In an embodiment, feature detection module 64 detects a predetermined number of feature points (e.g., 200 feature points) based on quality. Feature detection module 80 may also output a quality value along with each detected feature point. Feature tracking module 66 uses the current image frame 80 to track feature points detected in the immediately preceding image frame 80. Feature selection module 68 selects specific feature points of interest between two consecutive image frames 80, removing any outliers or feature points representing noise. In an embodiment, feature selection module 68 removes feature points representing the main vehicle, such as feature points representing the hood of the main vehicle, and then executes an outlier detection algorithm, such as the Random Sample Consensus (RANSAC) algorithm. Therefore, modules 64, 66, and 68 execute algorithms to identify specific feature points of interest between two consecutive image frames 80.

[0042] In one embodiment, the motion detection module 70 first determines a total number of the particular feature points of interest that exist between the two consecutive image frames 80 and compares the total number of the particular feature points of interest to a threshold. The threshold of the particular feature points of interest indicates that there is a sufficient number of sample points to accurately determine movement between the two consecutive image frames 80. For example, in one embodiment, the threshold number of the particular feature points of interest is 50.

[0043] The motion detection module 70 calculates a change in motion between the particular feature points of interest between the two consecutive image frames 80 and compares the change in motion between the particular feature points of interest between the two consecutive image frames 80 to a threshold amount of change in motion. The threshold amount of change in motion is a predefined tuning parameter that selects two image frames in the sliding window 82. It should be appreciated that a greater amount of motion enables triangulation of the two-dimensional feature points to obtain three-dimensional points. If the motion detection module 70 determines that the change in motion between the particular feature points of interest between the two consecutive image frames 80 is equal to or greater than the threshold change in motion, the motion detection module 70 determines that the change in motion is sufficient and the motion detection module 70 sends the particular feature points of interest to the triangulation module 72. However, if the motion detection module 70 determines that the change in motion is not sufficient, the feature detection module 64 evaluates the next consecutive image frame 80.

[0044] The triangulation module 72 receives the particular feature points of interest corresponding to the two consecutive image frames 80 from the motion detection module 70 and the camera poses 84 corresponding to the two consecutive image frames 80 from the camera pose estimation module 62 and performs a triangulation algorithm to determine three-dimensional coordinates of the particular feature points of interest corresponding to the two consecutive image frames 80. It should be appreciated that the particular feature points of interest corresponding to the two consecutive image frames 80 are represented in a two-dimensional image plane coordinate system, while the three-dimensional positions of the particular feature points of interest are represented in a world coordinate system.

[0045] The optimization module 74 executes a non-linear optimization algorithm to determine the calibrated camera extrinsic parameters 50 associated with the particular camera 30 based on the three-dimensional coordinates of the particular feature points of interest corresponding to two consecutive image frames, the particular feature points of interest corresponding to the two consecutive image frames 80, and the camera poses 84 corresponding to the two consecutive image frames 80. Specifically, the optimization module 74 executes the non-linear optimization algorithm to minimize the re-projection error with respect to the calibrated camera extrinsic parameters 50 and the three-dimensional coordinates of the particular feature points of interest while solving a cost function. The result of the optimization is saved to memory, the sliding window 82 containing the consecutive image frames 80 is cleared, and the one or more autonomous driving controllers 20 re-determine the calibrated camera extrinsic parameters 50 (i.e., the feature detection module 64 re-detects the feature points in two new consecutive image frames 80). After a threshold number of calibrated camera extrinsic parameters 50 are determined, the filtering module 76 can perform one or more filtering and averaging operations to remove outliers and noise. In one embodiment, the threshold number of calibrated camera extrinsic parameters 50 is 300.

[0046] Figure 3 is a process flow diagram illustrating a method 200 for calibrating the initial camera extrinsic parameters 52. Referring generally to Figures 1 to 3 The method 200 can begin at block 202. In block 202, the localization module 60 executes a localization algorithm to determine the vehicle pose estimate 54 based on the position and movement of the autonomous vehicle 10 indicated by the sensor data 58. The method 200 can then proceed to block 204.

[0047] In block 204, the camera pose estimation module 62 determines the camera pose 84 of the particular camera 30 based on the vehicle pose estimate 54 and the initial camera extrinsic parameters 52. The method 200 can then proceed to block 206.

[0048] In block 206, the feature detection module 64 executes a feature detection algorithm to detect feature points in two consecutive image frames 80. The method 200 can then proceed to block 208.

[0049] In block 208, the feature tracking module 66 tracks the feature points detected in the immediately preceding image frame 80 with the current image frame 80. The method 200 can then proceed to block 210.

[0050] In block 210, the feature selection module 68 selects the particular feature points of interest between the two consecutive image frames 80. The method 200 can then proceed to block 212.

[0051] In block 212, the motion detection module 70 determines the change in motion between the particular feature points of interest between the two consecutive image frames 80 and compares the change in motion between the particular feature points of interest between the two consecutive image frames 80 to a threshold amount of change in motion. Then, the method 200 can proceed to decision block 214.

[0052] In block 214, if the motion detection module 70 determines that the amount of change in motion between the particular feature points of interest between the two consecutive image frames 80 is equal to or greater than the threshold amount of change in motion, the method 200 can proceed to block 216. Otherwise, the method 200 can return to block 206.

[0053] In block 216, the triangulation module 72 executes a triangulation algorithm to determine the three-dimensional coordinates corresponding to the particular feature points of interest of the two consecutive image frames 80. Then, the method 200 can proceed to block 218.

[0054] In block 218, the optimization module 74 executes a non-linear optimization algorithm to determine the calibrated extrinsic camera parameters 50 associated with the particular camera 30 based on the three-dimensional coordinates corresponding to the particular feature points of interest of the two consecutive image frames, the particular feature points of interest corresponding to the two consecutive image frames, and the camera poses 84 corresponding to the two consecutive image frames 80. Then, the method 200 can end or return to block 202.

[0055] In general, with reference to the drawings, the disclosed system provides various technical effects and benefits by calibrating extrinsic camera parameters. Specifically, the disclosed system only needs to track feature points from two image frames and reuses the vehicle pose information from the localization module instead of re-estimating the vehicle pose, thus not requiring repetitive work to determine the vehicle pose. Therefore, the disclosed system reduces the complexity of computation compared to current state-of-the-art methods, and in embodiments, the execution speed can be orders of magnitude faster compared to current methods.

[0056] A controller can refer to an electronic circuit, a combinational logic circuit, a field programmable gate array (FPGA), a processor executing code (shared, dedicated, or group), or a combination of some or all of the above (for example, in a system on a chip) or a portion thereof. Additionally, the controller can be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor can operate under the control of an operating system that resides in the memory. The operating system can manage computer resources so that computer program code embodied as one or more computer software applications, such as applications that reside in the memory, can have instructions executed by the processor. In alternative embodiments, the processor can execute the applications directly, in which case the operating system can be omitted.

[0057] The description of the present disclosure is merely exemplary in nature and variations that do not depart from the gist of the present disclosure are intended to be within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

Claims

1. A system for determining calibrated camera extrinsic parameters of an autonomous transportation vehicle, the system comprising: a camera mounted to the autonomous transportation vehicle, the camera collecting image data comprising a plurality of image frames; one or more autonomous driving controllers in electronic communication with the camera, the one or more autonomous driving controllers executing instructions to: determine a transportation vehicle pose estimate based on a location and movement of the autonomous transportation vehicle through a localization algorithm; determine a camera pose of the camera based on the transportation vehicle pose estimate and initial camera extrinsic parameters; receive a plurality of image frames based on a sliding window buffering technique; identify particular feature points of interest between two consecutive image frames that are part of the plurality of image frames; determine a motion change between the particular feature points of interest between the two consecutive image frames; compare the motion change between the particular feature points of interest between the two consecutive image frames to a threshold amount of motion change, wherein the threshold amount of motion change is a predefined tuning parameter that selects two image frames in a sliding window; determine that the motion change between the particular feature points of interest between the two consecutive image frames is equal to or greater than the threshold amount of motion change; execute a triangulation algorithm to determine three-dimensional coordinates of the particular feature points of interest corresponding to the two consecutive image frames; and execute a non-linear optimization algorithm to determine the calibrated camera extrinsic parameters based on the three-dimensional coordinates of the particular feature points of interest corresponding to the two consecutive image frames, the particular feature points of interest corresponding to the two consecutive image frames, and the camera pose corresponding to the two consecutive image frames; wherein executing the non-linear optimization algorithm comprises: minimizing re-projection errors with respect to the calibrated camera extrinsic parameters and the three-dimensional coordinates of the particular feature points of interest while solving a cost function.

2. The system of claim 1, wherein, Identifying the particular feature points of interest between the two consecutive image frames further comprises: executing a feature detection algorithm to detect the particular feature points in the two consecutive image frames; tracking feature points detected in an immediately preceding image frame with a current image frame; and selecting the particular feature points of interest between the two consecutive image frames.

3. The system of claim 1, wherein, The one or more autonomous driving controllers execute instructions to: save results of solving the non-linear optimization algorithm to a memory; purge a sliding window containing consecutive image frames; re-determine the calibrated camera extrinsic parameters; and after obtaining a threshold number of calibrated camera extrinsic parameters, perform one or more filtering and averaging operations on the calibrated camera extrinsic parameters. The camera pose is determined by transforming the initial camera extrinsic parameters from a transportation vehicle coordinate system to a world coordinate system.

4. The system of claim 1, wherein, The initial camera extrinsic parameters are saved in a memory of the one or more autonomous driving controllers.

5. The system of claim 1, wherein, The initial camera extrinsic parameters represent an initial approximation of actual extrinsic parameters of the camera.

6. The system of claim 1, wherein, The one or more autonomous driving controllers are located on-board the autonomous transportation vehicle.

7. The system of claim 1, wherein, The initial camera extrinsic parameters are represented based on a traffic carrier body coordinate system, and the initial camera extrinsic parameters include an initial translation vector and an initial rotation matrix.

Images