Method of self-calibration of a stereo system without prior knowledge of the scene

Abstract

Systems and methods for self-calibration of a stereo system without prior knowledge of the scene are disclosed herein. For example, a method for calibrating a stereo imaging system may comprise extracting a set of feature points from a first image captured from a first view of a stereo image and a second image captured from a second view and matching the set of feature points between the two images to form a list of matched points. The method may also comprise iteratively adjusting a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y-dimension parallax value in the list of matched points, determining a set of x-dimension parallax values for the set of feature points, and adjusting a convergence calibration value (Ry) of the stereo imaging system. The method may be performed while the system is in operation without the need for a special calibration setup.

Description

Method of Self-Calibration of a Stereo System without Prior Knowledge of the SceneCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application Number 63 / 640,874, as filed on April 30, 2024.BACKGROUND

[0002] Stereo cameras work by mimicking human binocular vision— they capture two images of the same scene from slightly different viewpoints using two lenses spaced a fixed distance apart. Because of this separation, objects in the scene appear at slightly different positions in the left and right images, an effect known as parallax. The difference in position of a specific point between the two images is called disparity. The disparity of a specific position in two images is a quantification of the effect of parallax and can also be referred to as a "parallax value." By measuring this disparity across the image, the stereo system can triangulate the distance to each point, effectively building a depth map of the scene. Closer objects show greater disparity, while farther objects show less, allowing the system to perceive depth and understand the 3D structure of its environment.

[0003] Stereo cameras need to be calibrated to ensure accurate depth perception and 3D reconstruction. Calibration determines the precise geometric relationship between the two camera lenses, including their relative positions and orientations. Without calibration, the system cannot correctly match points between the left and right images, leading to errors in calculating disparity— and therefore depth. Proper calibration ensures that corresponding features in each image are aligned correctly, enabling the stereo system to compute reliable and accurate depth maps.

[0004] Different methods exist for calibrating stereo camera systems. Certain methods rely on the use of test patterns or markers which are placed in the field of view of the stereo camera in a controlled environment. These methods require specific calibration procedures with an operator and a dedicated setup. Alternative calibration methods use feature point extraction from each of the left and right images. The feature points take the place of the test patterns ormarkers and are used in a similar way to calibrate the stereo camera. While feature pointbased methods do not require the use of calibration targets or operators, they are generally considered more limited in the type of errors they can compensate for.SUMMARY

[0005] This disclosure relates to the field of stereo imaging systems for producing three- dimensional images, and the calibration of such systems. Existing calibration methods fail to properly re-calibrate the external rotation parameters of a stereo camera system after a factory calibration. The calibration of a stereo system (intrinsic parameters and extrinsic parameters) is an important part of the calibration for the purpose of calibrating a stereo system that can accurately determine depth from the stereo images. The calibration ensures that the rectified images from the system come from, or appear to come from via rectification, a perfectly matched and aligned system (e.g., both cameras are perfectly parallel). A stereo camera system may become misaligned or de-calibrated over time due to vibrations, shocks (e.g., drops), or the shifting of components (e.g., the effect of time). Therefore, a method to adjust and compensate for these misaligning effects during use of the stereo camera system is critical.

[0006] Computer stereo vision takes two or more images with known relative camera positions that show an object from different viewpoints. The system may compare feature points, or groups of pixels, between images. A feature point may correspond to an object or surface within the scene captured by the cameras and may accordingly have a specific dimension and location (e.g., depth). For each feature point, the system may determine the corresponding point's depth in the scene (e.g., distance from the camera) by first finding a matching feature point (e.g., pixels showing the same scene point) in the other image and then applying triangulation to the found matches to determine their depth. Finding matches in stereo vision may use epipolar geometry; the system may search for the corresponding pixel in the second image at the same y-height (e.g., epipolar line) as the original feature point in the first image.

[0007] Calibration of the stereo camera system may include orienting the epipolar lines to be horizontal (e.g., at a consistent height). If two images are coplanar (e.g., taken such that the right camera is only offset horizontally compared to the left camera and not being moved towards the object or rotated), then each pixel's epipolar line is horizontal and at the samevertical position as that pixel. Horizontal epipolar lines may simplify the stereo matching process. However, when the right camera is not positioned or oriented correctly relative to the left camera, the epipolar lines may be slanted. State of the art methods for stereo camera system calibration perform for 2 of the 3 rotations that define the external parameters of a calibration (e.g., Rx, Ry, and Rz). Most of the time, the convergence calibration value (Ry) is the most difficult to obtain accurately. The description herein discloses a full and global approach to re-adjust parameters without any prior knowledge regarding the scene the stereo camera system will be calibrated with.

[0008] The calibration of a stereo camera system can include adjusting a tilt calibration value (Rx), a convergence calibration value (Ry), and a roll calibration value (Rz). The tilt calibration value (Rx) and the roll calibration value (Rz) may be based on a y-dimension parallax value in a list of matched points between two images and may be determined by optimizing the list of pairs of points. For Rx and Rz, the major impact of a misalignment will be on the delta-y of each pair of points. A tilt (Rz) creates a direct y-misalignment between both images. A Roll (Rx) creates an inverse misalignment on both sides of the images. In specific embodiments, an optimization is first conducted on the two parameters of the rotation Rx and Rz by ingesting the list of pairs of points into an optimizer. These processes calculate a new Rx and Rz value that can be used to rectify both left and right images.

[0009] In specific embodiments, once Rx and Rz are optimized, they can be used to determine Ry (Convergence). For Ry rotation there is very little impact on the delta-y of each point which makes optimization difficult and unstable. Instead, Ry rotation may be calibrated by determining a set of x-dimension parallax values for the set of feature points. "Parallax" may be the effect of the differences in the viewpoints of the left and right images and "disparity" or "parallax value" may each refer to the quantification of the effect of parallax. The x-disparity (dx or delta-x) of a point P may be the position of the corresponding point pl in the left image minus the position of the corresponding point p2 in the right image. In this configuration, if correctly calibrated, point p2 will appear further to the left in the right image compared to the location of point pl in the left image. Therefore, there is always a negative disparity for a calibrated system. Accordingly, any positive disparity may be flagged as an error. That is, ifpoint p2 in the right image is located to the right of point pl in the left image, then the system may not be calibrated, and corresponding adjustments may be made. The convergence calibration value (Ry) of the stereo imaging system may be adjusted based on the x-dimension parallax values (e.g., disparity values). Calibrating Rx, Rz, and Ry may be iterative since each adjustment in each direction can impact the others. For example, the Rx and Rz rotations can be obtained, can be used to obtain an Ry rotation, and then the Ry rotation so obtained can be used to obtain refined Rx and Rz rotations.

[0010] Calibration methods disclosed herein have many benefits compared to a factory calibration of a stereo imaging system. For example, the calibration methods can be performed live, while the system is in operation, without the need for a test target or special calibration setup. As another example, the calibration methods can be performed regularly, to compensate for any de-calibration that may happen while the stereo system is in operation. In specific embodiments, the disclosed methods can run in the background on a stereo imaging system and can result in a continuous adjustment of the system.

[0011] In specific embodiments of the invention, a method for calibrating a stereo imaging system is provided. The method comprises: extracting a set of feature points from a first image captured from a first view of a stereo image and a second image captured from a second view of the stereo image, matching the set of feature points between the first image and the second image to form a list of matched points, and iteratively, using the list of matched points, conducting a set of steps. The set of steps comprises: adjusting a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y-dimension parallax value in the list of matched points, determining a set of x-dimension parallax values for the set of feature points, and adjusting a convergence calibration value (Ry) of the stereo imaging system.

[0012] In specific embodiments of the invention, one or more non-transitory computer- readable medium is provided. The one or more non-transitory computer-readable medium store instructions that, when executed by one or more processors, cause the one or more processors to conduct a method. The method comprises: extracting a set of feature points from a first image captured from first view of a stereo image and a second image captured from a second view of the stereo image, matching the set of feature points between the first imageand the second image to form a list of matched points, and iteratively, using the list of matched points, conducting a set of steps. The set of steps comprises: adjusting a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y-dimension parallax value in the list of matched points, determining a set of x-dimension parallax values for the set of feature points, and adjusting a convergence calibration value (Ry).

[0013] In specific embodiments of the invention, a system is provided. The system comprises: a stereo camera having a first sensor and a second sensor, one or more processors, and one or more non-transitory computer-readable media. The one or more non-transitory computer- readable media stores instructions that, when executed by the one or more processors, cause the system to conduct a method. The method comprises: extracting a set of feature points from a first image captured, by the first sensor, from a first view of a stereo image and a second image captured, by the second sensor, from a second view of the stereo image, matching the set of feature points between the first image and the second image to form a list of matched points, and iteratively, using the list of matched points, conducting a set of steps. The set of steps comprises: adjusting a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y-dimension parallax value in the list of matched points, determining a set of x-dimension parallax values for the set of feature points, and adjusting a convergence calibration value (Ry) of the system.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings illustrate various embodiments of systems, methods, and various other aspects of the disclosure. A person with ordinary skills in the art will appreciate that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one example of the boundaries. It may be that in some examples one element may be designed as multiple elements or that multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component in another, and vice versa. Furthermore, elements may not be drawn to scale. Non-limiting and non-exhaustive descriptions are described with reference to the following drawings. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating principles.

[0015] Fig. 1 provides an example of camera rotations including Rx (tilt), Ry (yaw or pan), and Rz (roll) in accordance with specific embodiments of the inventions disclosed herein.

[0016] Fig. 2 provides an example of system calibration parameters in a stereo camera system in accordance with specific embodiments of the inventions disclosed herein.

[0017] Fig. 3 provides an example of a correctly calibrated stereo camera system in accordance with specific embodiments of the inventions disclosed herein.

[0018] Fig. 4 provides an example of a misaligned stereo camera system in accordance with specific embodiments of the inventions disclosed herein.

[0019] Fig. 5 provides an example of disparities between feature points of a left image and a right image in accordance with specific embodiments of the inventions disclosed herein.

[0020] Fig. 6 provides an example of a calibration method involving detecting and describing specific points on the images taken by cameras in a stereo imaging system in accordance with specific embodiments of the inventions disclosed herein.

[0021] Fig. 7 provides an example of a method for calibrating a stereo imaging system in accordance with specific embodiments of the inventions disclosed herein.DETAILED DESCRIPTION

[0022] Reference will now be made in detail to implementations and embodiments of various aspects and variations of systems and methods described herein. Although several exemplary variations of the systems and methods are described herein, other variations of the systems and methods may include aspects of the systems and methods described herein combined in any suitable manner having combinations of all or some of the aspects described.

[0023] Different systems and methods for calibrating stereo camera systems will be described in detail in this disclosure. The methods and systems disclosed in this section are nonlimiting embodiments of the invention, are provided for explanatory purposes only, and should not be used to constrict the full scope of the invention. It is to be understood that the disclosed embodiments may or may not overlap with each other. Thus, part of one embodiment, or specific embodiments thereof, may or may not fall within the ambit of another, or specific embodiments thereof, and vice versa. Different embodiments from different aspects may be combined or practiced separately. Many different combinations and sub-combinations of therepresentative embodiments shown within the broad framework of this invention, that may be apparent to those skilled in the art but not explicitly shown or described, should not be construed as precluded.

[0024] Existing calibration methods fail to properly re-calibrate the external rotation parameters of a stereo camera system after a factory calibration. The calibration of a stereo system (intrinsic parameters and extrinsic parameters) is an important part of the calibration for the purpose of calibrating a stereo system that can accurately determine depth from the stereo images. The calibration ensures that the rectified images from the system come from, or appear to come from via rectification, a perfectly matched and aligned system (e.g., both cameras are perfectly parallel). A stereo camera system may become misaligned or decalibrated over time due to vibrations, shocks (e.g., drops), or the shifting of components (e.g., the effect of time). Therefore, a method to adjust and compensate for these misaligning effects during use of the stereo camera system is critical.

[0025] Computer stereo vision takes two or more images with known relative camera positions that show an object from different viewpoints. The system may compare feature points, or groups of pixels, between images. A feature point may correspond to an object or surface within the scene captured by the cameras and may accordingly have a specific dimension and location (e.g., depth). For each feature point, the system may determine the corresponding scene point's depth (e.g., distance from the camera) by first finding the matching feature point (e.g., pixels showing the same scene point) in the other image and then applying triangulation to the found match to determine their depth. Finding matches in stereo vision may use epipolar geometry; the system may search for the corresponding pixel in the second image at the same y-height (e.g., epipolar line) as the original feature point in the first image.

[0026] Calibration of the stereo camera system may include orienting the epipolar lines to be horizontal (e.g., at a consistent height). In specific embodiments, a camera may physically rotate (e.g., roll, yaw, or tilt) as part of calibration such that the two images of the two cameras are taken with only a horizontal x-displacement between the cameras (e.g., the cameras are parallel with no relative rotation, y-offset, or z-offset between them). In specific embodiments, the camera system may rectify images from the camera, warping one or both images to appearas if they had been taken with only a horizontal displacement between the cameras. If two images are coplanar (e.g., taken such that the right camera is only offset horizontally compared to the left camera and not being moved towards the object or rotated), then each pixel's epipolar line is horizontal and at the same vertical position as that pixel. However, when the right camera is moved forward, moved backward, or rotated relative to the left camera, the epipolar lines may be slanted. Image rectification may warp one or both images such that they appear as if they had been taken with only a horizontal displacement, which may make all epipolar lines are horizontal. Horizontal epipolar lines may simplify the stereo matching process.

[0027] Figure 1 shows camera rotations including Rx (tilt), Ry (yaw or pan), and Rz (roll) in accordance with how these are oriented in the discussions herein. Rx rotates about the x-axis (which runs horizontally), Ry rotates about the y-axis (which runs vertically), and Rx rotates about the z-axis (which runs forwards-backwards). View 100 and view 150 are two instances of the same camera from different angles. Rotation calibration may be especially important for depth estimation processes. The disparity (e.g., "pixel" depth) to depth (e.g., metric depth) may only be valid when both cameras are strictly parallel to each other. Therefore, after calibration (or rectification), Rx - Ry - Rz - 0. State of the art methods for stereo camera system calibration perform for 2 of the 3 rotations that define the external parameters of a calibration (e.g., Rx, Ry, and Rz). Most of the time, the convergence calibration value (Ry) is the most difficult to obtain accurately. The description herein discloses a full and global approach to re-adjust parameters without any prior knowledge regarding the scene the stereo camera system will be calibrated with.

[0028] A method for calibrating a stereo imaging system is disclosed which comprises extracting a set of feature points from a first image captured from a first view of a stereo image and a second image captured from a second view of the stereo image. The method also comprises matching the set of feature points between the first image and the second image. The method also comprises determining a set of horizontal parallax values for the set of feature points. The method also comprises iteratively adjusting the convergence calibration value (Ry)of the stereo imaging system and determining the set of horizontal parallax values until all the values in the set of horizontal parallax values are negative.

[0029] In specific embodiments, the method also comprises calibrating a tilt calibration value (Rx) of the stereo imaging system and roll calibration value (Rz) of the stereo imaging system prior to iteratively adjusting the Ry of the stereo imaging system.

[0030] In specific embodiments, the method for calibrating a stereo imaging system also comprises applying rectification matrices to images obtained from the first view and the second view based on the Rx of the stereo imaging system, the Ry of the stereo imaging system, and the Rz of the stereo imaging system. The process of calibrating the system to obtain the Rx, Ry, and Rz values can be conducted periodically after the stereo imaging system has been deployed and does not require any priori knowledge of the scene upon which the calibration process will be conducted.

[0031] In specific embodiments, the methods disclosed herein can be used to re-calibrate the 3 rotations (Rx, Ry, and Rz) needed to rectify stereo images. The disclosed methods can use a common or state-of-the art approach for calculating the Rx and Rz rotations, and a complementary approach for calculating the Ry rotation. Both approaches can be iterative since they can impact each other. For example, the Rx and Rz rotations can be obtained, can be used to obtain an Ry rotation, and then the Ry rotation so obtained can be used to obtain refined Rx and Rz rotations.

[0032] Calibration methods disclosed herein have many benefits compared to a factory calibration of a stereo imaging system. For example, the calibration methods can be performed live, while the system is in operation, without the need for a test target or special calibration setup. As another example, the calibration methods can be performed regularly, to compensate for any de-calibration that may happen while the stereo system is in operation. In specific embodiments, the disclosed methods can run in the background on a stereo imaging system and can result in a continuous adjustment of the system.

[0033] Figure 2 provides an example of system calibration parameters in stereo camera system 200 in accordance with specific embodiments of the inventions disclosed herein. Left camera 201, right camera 202, and point P are physical features in the scene. Point P is at location(X,Y,Z). Left camera 201 is represented by the origin point 01 of the set of axis (xl, yl, zl). The image taken by left camera 201 is represented by image 203 located on the image plane of camera 201. Image 203 includes the axes (ul, vl). Point pl on image 203 is the representation of point P within (e.g., projected onto) image 203. Right camera 202 is represented by the origin point 02 of the set of axis (x2, y2, z2). The image taken by right camera 202 is represented by image 204 located on the image plane of camera 202. Image 204 includes the axes (u 2, v2). Point p2 on image 204 is the representation of point P within (e.g., projected onto) image 204. Left camera 201 and right camera 202 are examples of sensors that may be used. Stereo camera system 200 may include one or more processors. Stereo camera system 200 may include one or more non-transitory computer-readable media storing instructions.

[0034] Calibrating stereo camera system 200 may include extracting a set of feature points (such as points pl and p2) from image 203 captured from a first view of a stereo image and image 204 captured from a second view of the stereo image. For example, stereo camera system may detect and describe point pl on image 203 and point p2 on image 204. Point detection can be conducted using various techniques such as ORB, SIFT, SURF, HARRIS, and others.

[0035] Calibrating stereo camera system 200 may include matching the set of feature points between image 203 and image 204 to form a list of matched points. Points pl and p2 may be paired together (as they both correspond to point P) in an entry in a list of points that are in both images. The list can be referred to as a list of pairs because, while P is a single point, the same point P is at two locations (ul, vl) and (u2, v2) in the two images 203 and 204 respectively. Techniques for point matching can include RANSAC, FAST, SIFT, and others.

[0036] The calibration of stereo camera system 200 can include adjusting a tilt calibration value (Rx) and a roll calibration value (Rz) of camera 201, camera 202, or both based on a y- dimension parallax value in the list of matched points. The tilt calibration value (Rx) and roll calibration value (Rz) may be determined by optimizing the list of pairs of points. Two sets of optimizations can be conducted on the list of pairs. For Rx and Rz, the major impact of a misalignment will be on the delta-y (dy) of each pair of points. A tilt (Rz) creates a direct y- misalignment between both images. A Roll (Rx) creates an inverse misalignment on both sidesof the images. In specific embodiments, an optimization is first conducted on the two parameters of the rotation Rx and Rz by ingesting the list of pairs of points into an optimizer (e.g., least-square optimization, Heb optimization, or Levenberg optimization). These processes calculate a new Rx and Rz value that can be used to rectify both left and right images.

[0037] In specific embodiments, once Rx and Rz are optimized, they can be used to determine Ry (Convergence). For Ry rotation there is very little impact on the delta-y (dy) of each point which makes the optimization difficult and unstable. Instead, Ry rotation may be calibrated by determining a set of x-dimension parallax values for the set of feature points. The x-disparity (dx) of point P may be the position of point pl in left image 203 minus the position of point p2 in right image 204. In this configuration, if correctly calibrated, point p2 will appear further to the left in image 204 than point pl is in image 203. Therefore, there is always a negative disparity for a calibrated system. Accordingly, any positive disparity may be flagged as an error. That is, if point p2 in image 204 is located to the right of the point pl in image 203, then the system may not be calibrated. The convergence calibration value (Ry) of stereo camera system 200 may be adjusted based on the x-dimension parallax values (e.g., disparity values).

[0038] Figure 3 illustrates an example of correctly calibrated stereo camera system 300 in accordance with specific embodiments of the inventions disclosed herein. Left image 303 is taken from the image plane of left camera 301. Right image 304 is taken from the image plane of right camera 302. Objects 310, 311, and 312 of stereo camera system 300 are captured by left camera 301 in image 303 and by right camera 302 in image 304. Left camera 301 and right camera 302 are examples of sensors that may be used. Stereo camera system 300 may include one or more processors. Stereo camera system 300 may include one or more non-transitory computer-readable media storing instructions.

[0039] As shown in stereo camera system 300, the horizontal positions of objects 310, 311, and 312 within images 303 and 304 are different. The horizontal differences may be referred to as disparity 320, disparity 321, and disparity 322 respectively. The disparities 320, 321, and 322 allow stereo camera system 300 to calculate the z-value distances of objects 310, 311, and 312. The sign of disparities 320, 321, and 322 (positive or negative) depends on the convention used and the arrangement of the cameras. In this example, left camera 301 is considered thereference and right camera 302 captures the scene from a slightly rightward (positive-x) position relative to left camera 301. Thus, in this example, disparity 320 of object 310 is the position in left image 303 minus the position in right image 304. In this configuration, object 310 in right image 304 will appear to the left compared to their position in left image 303 and disparity 320 is negative. As shown, disparity 321 and disparity 322 are also negative. In fact, for a calibrated system, there is always a negative disparity according to the following equation: u2 - ul - horizontal parallax (e.g., x-disparity). From this, any positive disparity may be flagged as an error. That is, if a feature point (e.g., object or portion of an object) in the right image is located to the right of the feature point location in the left image, then the right camera is miscalibrated.

[0040] Figure 4 illustrates an example of misaligned stereo camera system 400 in accordance with specific embodiments of the inventions disclosed herein. Stereo camera system 400 is similar to stereo camera system 300 except camera 302 is rotated in the Ry (yaw) direction so that it is no longer parallel with camera 301, as it was in stereo camera system 300. This rotation misaligns stereo camera system. Left image 303 is taken from the image plane of left camera 301. Right image 404 is taken from the image plane of right camera 302 and is different than image 304 due to the rotation of camera 302 in system 400 compared to system 300.

[0041] Horizontal disparities 420, 421, and 422 of objects 310, 311, and 312 (respectively) are different than horizontal disparities 320, 321, and 322 due to the rotation of camera 302. Using the same convention used in system 300, left camera 301 is the reference and right camera 302 captures the scene from a slightly rightward (positive-x) position relative to left camera 301. Thus, disparity 420 of object 310 is calculated as the position in left image 303 minus the position in right image 404. In this configuration, object 310 in right image 404 appears to the right compared to its position in left image 303, making disparity 420 positive. Object 311 in right image 404 appears to the left compared to its position in left image 303, making disparity 421 negative. Object 312 in right image 404 appears to the right compared to its position in left image 303, making disparity 422 positive. As discussed above, positive disparity is an error resulting from misalignment in the Ry rotation.

[0042] Stereo camera system 400 may determine that disparity 420 and disparity 422 are positive and may correct the position of camera 302. For example, stereo camera system 400 may rotate camera 302 in the positive Ry direction. In specific embodiments, camera 302 may be rotated only a small amount such that it may still not be correctly aligned. Stereo camera system may capture additional images using left camera 301 and right camera 302, may determine disparities between objects 310, 311, and 312 in those images, and may rotate camera 302 an additional amount if any disparities are (still) positive. Stereo camera system 400 may repeat the process of determining disparities and rotating camera 302 until no detected disparities are positive. If there are no pairs of points with a positive disparity, then camera 302 may refrain from rotating in the Ry directions.

[0043] Right camera 302 may rotate (e.g., adjust) a fixed amount or a variable amount in response to the largest positive disparity determined. In the example of Fig. 4, the largest positive disparity is disparity 422. In specific embodiments, right camera 302 may rotate a fixed or preset amount (e.g., number of degrees, fractions of a degree) based on whether a positive disparity is detected or not. For example, system 400 may rely on iterations of the calibration method to provide corrections that are larger than the single fixed amount that camera 302 rotates in each iteration. In specific embodiments, right camera 302 may rotate a variable amount (e.g., number of degrees, fractions of a degree) based on the magnitude (e.g., absolute value) of the largest positive (nonrational) disparity detected. For example, right camera 302 may rotate a larger amount the larger disparity 422 is, and may rotate a smaller amount the closer disparity 422 is to zero. In specific embodiments, system 400 may estimate a rotation amount for camera 302 to reduce disparity 422 to zero. Camera 302 may rotate this estimated amount or may rotate an amount smaller than the estimated amount to reduce risk of overcorrection. Further Ry corrections may be performed iteratively with additional image captures. In specific embodiments, additional Ry corrections may be performed interleaved with additional calculations or corrections for Rx and Rz.

[0044] In specific embodiments, camera 302 may physically rotate to calibrate system 400. In specific embodiments, image 404 may be rectified to account for a physical misalignment of Ry rotation in camera 302. Calibration may ensure that the rectified images from system 400come from, or appear to come from via rectification, a perfectly matched and aligned system (e.g., both cameras 301 and 302 are perfectly parallel). Rectification matrices may be applied to the left image and the right image based on the Rx of the stereo imaging system, the Ry of the stereo imaging system, and the Rz of the stereo imaging system.

[0045] In specific embodiments, stereo camera calibration may be performed while the stereo system is in operation (e.g., without a calibration pattern). The calibration may be performed periodically, continuously, manually, or based on detecting a possible misalignment. The calibration program may continue to run (e.g., iteratively, looping) until stopped by a manual input or until the system determines itself to be sufficiently calibrated (e.g., no disparities are nonrational). A possible misalignment may be detected by detecting rotational motion, translational motion, impacts, or a period of inactivity (e.g., system turned off, asleep, or otherwise unused) of camera 301 or camera 302. In specific embodiments, a possible misalignment may be detected by detecting a change in the environment (e.g., scene). For example, calibration may be automatically initiated based on detecting a shift in the view of camera 301, camera 302, or both. A shift in the views may be determined by comparing feature points between multiple images. For example, a corner where two walls meet the ceiling in a warehouse may be consistently included in images captured by stereo camera system 400. If several past images constantly detect the corner at point (xl, yl, zl,), but a new image detects a corner at point (x2, y2, z2), and does not detect a corner at point (xl, yl, zl), then stereo camera system 400 may perform a calibration.

[0046] Figure 5 illustrates an example of disparities between feature points of left image 501 (captured by a left camera) and right image 502 (captured by a right camera) in accordance with specific embodiments of the inventions disclosed herein. Feature points in Fig. 5 include groups of 9 pixels, although a feature point may be a group of any quantity of pixels. Some of the pixels in the features points relate to objects in the scene. Using left image 501 as the reference, feature point 510 has a disparity of positive one pixel, feature point 511 has a disparity of negative one pixel, and feature point 512 has a disparity of negative two pixels.

[0047] The sign of the x-disparity (positive or negative) depends on the convention used and the arrangement of the cameras. The setup in stereo vision of Fig. 5 is such that the left camerais considered the reference, and the right camera captures the scene from a slightly rightward position relative to the left camera. The x-disparity (dx of each pixel) of a pixel of an object is the position in the left image minus the position in the right image (left - right). In this configuration, if correctly calibrated, objects in the right image will appear to the left compared to their position in the left image. Therefore, there is always a negative disparity for pixels in a correctly calibrated system according to the following equation: u2 - ul - horizontal parallax (e.g., x-disparity), where ul is the x-value of the pixel in the left image and u2 is the x-value of the pixel in the right image. Because feature point 510 has a positive disparity, the camera system that captured left image 501 and right image 502 is not correctly calibrated in the Ry dimension. Accordingly, the right camera may be adjusted in Ry.

[0048] Determining the disparities of feature points and correcting Ry may be performed iteratively. If there are no feature points (e.g., pairs of points across the images 501 and 502) with a positive disparity, then the stereo camera system may refrain from making any changes to Ry. If there is at least one feature point with a positive disparity, then the stereo camera system may make a small change in Ry to converge the values in the correct direction. In specific embodiments, calibration may include iterations of calibrating Rx and Rz. For example, calibration may continue, after adjusting Ry, with solving for new values of Rx and Rz, and then evaluating the dx again (e.g., in another iteration) to determine if another change to Ry is required (e.g., nonrational x-disparities are present). The calibration process can be conducted continuously as the stereo camera system is used.

[0049] Figure 6 illustrates an example of calibration method 600 involving detecting and describing specific points on the images of the cameras in a stereo imaging system in accordance with specific embodiments of the inventions disclosed herein. Stereo imaging system may include a right camera, a left camera, and a processing system capable of performing steps of method 600. The left and right cameras may be horizontally displaced from each other (in the x-direction), but may be aligned in the Y and Z directions. Some steps, such as steps 601 and 602, may be performed simultaneously. In specific embodiments, some steps such as step 606, may be skipped. In specific embodiments, method 600 may be iterative such that after performing step 608, the system may loop back and repeat the method startingat steps 601 and 602. The process of method 600 (e.g., calibrating the system to obtain the Rx, Ry, and Rz values) may be conducted periodically after the stereo imaging system has been deployed and may not require any priori knowledge of the scene upon which the calibration process will be conducted.

[0050] At step 601, an image may be taken by a left camera (or sensor) in the stereo imaging system. At step 602, an image may be taken by a right camera (or sensor) in the stereo imaging system. Step 601 and step 602 may be performed at the same time or within a short time.That is, the right image and the left image may be captured for the same scene.

[0051] Method 600 may continue with step 603. At step 603, feature points may be extracted from the left image. Feature points may include groups of pixels that may relate all or part of an object, edge, surface, or feature of a surface (e.g., a text pattern on a box). The left image may use vectors (e.g., (ul, vl)) to describe feature points and may organize these vectors into a list. Feature point extraction may include detecting and describing specific points on the left image. Point detection can be conducted using various techniques such as ORB, SIFT, SURF, HARRIS, and others.

[0052] Method 600 may perform step 604 at the same time as step 603, may perform step 604 after step 603, or may perform step 604 before step 603. At step 604, feature points may be extracted from the left image. The left image may use vectors (e.g., (u2, v2)) to describe feature points and may organize these vectors into a list. Feature point extraction may include detecting and describing specific points on the image of the right camera in a stereo imaging system. Point detection can be conducted using various techniques such as ORB, SIFT, SURF, HARRIS, and others.

[0053] In specific embodiments, method 600 may continue with step 605. Step 605 may include matching between the left and right list of vectors (e.g., points) found at steps 603 and 604 respectively. Points in the two images can be paired together in this manner to form a list of points that are in both images. The list can be referred to as a list of pairs because, while it is the same point, the same point is at two locations (e.g., (ul, vl) and (u2, v2)) in the two images. Techniques for point matching can include RANSAC, FAST, SIFT, and others. In specific embodiments, at least 50 well-distributed feature matches may be determined. In specificembodiments, the eight-point algorithm may be used to estimate the fundamental matrix related to the stereo camera system.

[0054] In specific embodiments, method 600 may continue with step 606. At step 606, the list of pairs of points may be optimized. Two sets of optimizations can be conducted on the list of pairs. For Rx and Rz, the major impact of a misalignment will be on the delta-y (dy) of each pair of points. A tilt (Rz) creates a direct y-misalignment between both images. A Roll (Rx) creates an inverse misalignment on both sides of the images. In specific embodiments, an optimization is conducted on the two parameters of the rotation Rx and Rz by ingesting the list of pairs of points into an optimizer (e.g., least-square optimization, Heb optimization, or Levenberg optimization). These processes calculate a new Rx and Rz value that can be used to rectify both left and right images.

[0055] In specific embodiments, method 600 may continue with step 607. At step 607, Ry (convergence) may be determined using the optimized Rx and Rz. For Ry rotation there is very little impact on the delta-y (dy) of each point which makes the optimization difficult and unstable. An incorrect Ry rotation may shift objects along the x-axis; however, a calibrated stereo system also includes variable differences between horizontal positions of objects between the two images. The horizontal differences (disparity) of a calibrated machine allow for the system to calculate 3D distances. The sign of the x-disparity (positive or negative) depends on the convention used and the arrangement of the cameras. In this example, left camera is considered the reference and the right camera captures the scene from a slightly rightward position relative to the left camera. The x-disparity (dx of each pixel) of a pixel of an object is the position in the left image minus the position in the right image (left - right). In this configuration, objects in the right image will appear to the left compared to their position in the left image. Therefore, there is always a negative disparity (or zero) according to the following equation: u2 - ul - horizontal parallax (e.g., x-disparity). The disparities (e.g., x-dimension parallax values) are rational (e.g., make sense in the physical world) when they are negative; and are not rational when they are positive. From this, any positive disparity may be flagged as an error. That is, if a feature point in the right image is located to the right of the feature point location in the left image, then the right camera is mis-calibrated.

[0056] The right camera may rotate (e.g., adjust) a fixed amount or a variable amount in response to the largest positive disparity determined. In specific embodiments, the right camera may rotate a fixed or preset number of degrees based on whether a disparity is positive or not. For example, the system may rely on the iterative nature of method 600 to provide corrections that are larger than the single fixed number of degrees. In specific embodiments, the right camera may rotate a variable number of degrees based on the magnitude (e.g., absolute value) of the positive disparity. For example, the right camera may rotate more degrees if the positive disparity is larger and may rotate less degrees if the positive disparity is closer to zero.

[0057] Steps of method 600 may be conducted multiple times. In specific embodiments, steps 606 and 607 may be iteratively completed. For example, steps 606 and 607 may be conducted in response to detecting that a disparity (e.g., an x-dimension parallax value) is not rational (e.g., positive) or may be conducted until the horizontal disparities are all rational (e.g., negative or zero). In specific embodiments, step 607 may be completed after, and based on aspects of, step 606.

[0058] In specific embodiments, method 600 may continue with step 608. At step 608 rectification matrices may be applied to the left image and the right image based on the Rx of the stereo imaging system, the Ry of the stereo imaging system, and the Rz of the stereo imaging system. Rectification matrices may be applied to images obtained from the left camera view and the right camera view based on the Rx of the stereo imaging system, the Ry of the stereo imaging system, and the Rz of the stereo imaging system

[0059] Method 600 may be performed multiple times. For example, method 600 may be conducted periodically after the stereo imaging system has been deployed. In specific embodiments, method 600 may be conducted continuously as the stereo camera system is used. Method 600 may also be conducted multiple times within a short time frame to improve the calibration. For example, if there is a pair of points with a positive disparity then the system may make a change in Ry to converge the values in the correct direction. In specific embodiments, the stereo system may make a fixed small change in Ry. In specific embodiments, the stereo system may may a variable change in Ry based on an absolute valueof the largest positive disparity (e.g., the largest nonrational x-dimension parallax value). The system may solve for new values of Rx and Rz, and then evaluate the dx in another iteration to determine if another change to Ry is required. If there are no pairs of points with a positive disparity, then the system may refrain from making any changes to Ry.

[0060] In specific embodiments, method 600 may be performed while the stereo system is in operation. For example, at least steps 603, 604, 605, 606, and 607 may be performed while the stereo system is in operation and without a calibration pattern. Method 600 (e.g., at least steps 603, 604, 605, 606, and 607) may be performed periodically. Each periodic use of method 600 may include one or more iterations of method 600. For example, method 600 may be performed three times in a short time frame, then wait one hour before performing another burst of three iterations. The number of iterations in each periodic use of method 600 may be set or may be based on reaching a threshold state (e.g., no non-rational disparities), which may include a variable number of iterations. Method 600 (e.g., at least steps 603, 604, 605, 606, and 607) may be performed continuously while the stereo system is in operation. Method 600 may be performed due to a manual input. Method 600 may continue to iteratively run until stopped by a manual input. Method 600 may be automatically initiated based on detecting a possible cause for misalignment such as rotational motion, translational motion, impacts, a period of inactivity (e.g., system turned off, asleep, or otherwise unused), or a change in the environment (e.g., scene). For example, method 600 may be automatically initiated based on detecting a shift in the view of the first camera, the second camera, or both. A shift in the views may be determined by comparing feature points between multiple images. For example, a pillar support in a warehouse may be consistently included in images captured by the stereo camera system. If several past images constantly detect the pillar base at point (xl, yl, zl,), but a new image detects a pillar base at point (x2, y2, z2), and does not detect a pillar base at point (xl, yl, zl), then the stereo camera system may perform calibration method 600.

[0061] In specific embodiments, the stereo system may determine, using the list of matched points, a depth map. In specific embodiments, the stereo system may determine a depth map on using information gathered during the first iteration of the method (e.g., if disparities are determined to be rational). In specific embodiments, the stereo system may determine a depthmap on a subsequent (e.g., not the first) iteration of the method, as performing at least one iteration may improve the calibration of the system and thus the accuracy of the depth map. For example, the stereo system may extract a second set of feature points from a third image captured from a first calibrated view of a second stereo image and a fourth image captured from a second calibrated view of the second stereo image. The first calibrated view and the second calibrated view may correspond to a second image taken at the first camera and a second image taken at the second camera (e.g., as part of a second iteration of method 600). The stereo system may match the second set of feature points between the third image and the fourth image to form a second list of matched points. The stereo system may determine, using the second list of matched points, a depth map. To create the depth map, the stereo system may measure the disparity of the list (or the second list) of matched points. The stereo system can triangulate the distance to each point. Closer objects show greater disparity, while farther objects show less, allowing the system to perceive depth and understand the 3D structure of its environment.

[0062] Figure 7 provides an example of method 700 for calibrating a stereo imaging system in accordance with specific embodiments of the inventions disclosed herein. Method 700 may be implemented by a system including a stereo camera having a first sensor (e.g., camera) and a second sensor (e.g., camera), one or more processors, and one or more non-transitory computer-readable media storing instructions. The one or more non-transitory computer- readable medium may store instructions that, when executed by the one or more processors, cause the one or more processors to conduct method 700. Method 700 may be implemented by a system including means for performing the steps of method 700. Steps, or portions of steps, of method 700 may be duplicated, omitted, rearranged, or otherwise deviate from the form shown. Additional steps may be added to method 700. Steps, or portions of steps, of method 700 may be performed in series or parallel.

[0063] At step 702, a set of features points may be extracted from a first image captured from a first view of a stereo image and a second image captured from a second view of the stereo image. In specific embodiments, extracting the set of feature points may be based on (e.g., triggered by) detecting a shift in the first view, the second view, or both. In specificembodiments, extracting the set of feature points may be based on (e.g., triggered by) detecting a manual input.

[0064] At step 704, the set of feature points may be matched between the first image and the second image to form a list of matched points.

[0065] At step 706, a set of steps (e.g., steps 708, 710, and 712) may be iteratively conducted using the list of matched points. In specific embodiments, additional iterations of the set of steps may be conducted in response to detecting that an x-dimension parallax value in the set of x-dimension parallax values is not rational. In specific embodiments, additional iterations of the set of steps may be conducted until the set of x-dimension parallax values are all rational. However, in specific embodiments not all the set of x-dimension parallax values need to be rational for the set of steps to cease being conducted. For example, the process could account for the fact that mismatched pairs of points will continue to provide irrational x-dimension parallax values even when the process has been executed to completion. Accordingly, in specific embodiments, additional iterations of the set of steps are conducted until a subset of the set of x-dimension parallax values are all rational, and a cardinality of the subset and a cardinality of the set differ by a tolerance value. The tolerance value can be adjusted based on a degree in confidence as to the matching of points (e.g., the tolerance value can increase as a confidence in the matching of points decreases). In specific embodiments, the x-dimension parallax values are rational when they are negative, and the x-dimension parallax values are not rational when they are positive.

[0066] As part of the set of steps that may be iteratively conducted, at step 708, a tilt calibration value (Rx) and a roll calibration value (Rz) may be adjusted based on a y-dimension parallax value in the list of matched points.

[0067] As part of the set of steps that may be iteratively conducted, at step 710, a set of x- dimension parallax values may be determined for the set of feature points. In specific embodiments, step 710 may be performed after step 708. That is, determining the set of x- dimension parallax values for the set of feature points may be performed after adjusting the tilt calibration value (Rx) and the roll calibration value (Rz).

[0068] As part of the set of steps that may be iteratively conducted, at step 712, a convergence calibration value (Ry) of the stereo imaging system may be adjusted. In specific embodiments, step 712 may be performed after, and based on, step 710. That is, adjusting the convergence calibration value (Ry) of the stereo imaging system may be performed after, and based on, determining the set of x-dimension parallax values for the set of feature points.

[0069] In specific embodiments and as part of adjusting the convergence calibration value (Ry), at step 714, the convergence calibration value (Ry) may be adjusted by a fixed amount.

[0070] In specific embodiments and as part of adjusting the convergence calibration value (Ry), at step 716, the convergence calibration value (Ry) may be adjusted by an amount based on an absolute value of a largest nonrational x-dimension parallax value in the set of x-dimension parallax values.

[0071] In specific embodiments, at step 718, rectification matrices may be applied to images obtained from the first view and the second view based on the Rx of the stereo imaging system, the Ry of the stereo imaging system, and the Rz of the stereo imaging system.

[0072] In specific embodiments, at step 720, a depth map may be determined using the list of matched points (e.g., formed at step 704).

[0073] In specific embodiments, at least steps 702, 704, and 706 of method 700 may be performed while the stereo imaging system is in operation (e.g., without a calibration pattern). For example, these steps may be performed periodically or continuously. As another example, these steps may be triggered by a manual input, a change in the environment (e.g., scene) of the system, camera movement, etc. Each performance (e.g., use) of method 700 may include one or more iterations of method 700. For example, method 700 may be performed ten times in a short time frame, then wait four hours before performing another burst of ten iterations. The number of iterations in each use of method 700 may be set (e.g., by the system or manually) or may be based on reaching a threshold state (e.g., no non-rational disparities), which may include a variable number of iterations. In specific embodiments, method 700 may continue to iteratively run until stopped by a manual input. Method 700 may be automatically initiated based on detecting a possible cause for misalignment such as rotational motion, translational motion, impacts, a period of inactivity (e.g., system turned off, asleep, orotherwise unused), or a change in the environment (e.g., scene). For example, method 700 may be automatically initiated based on detecting a shift in the view of the first camera, the second camera, or both. A shift in the views may be determined by comparing feature points between multiple images to find environmental or stable feature points. If these feature points appear to move, then method 700 may be performed to recalibrate the stereo camera system.

[0074] Method may have many benefits compared to a factory calibration of a stereo imaging system. For example, method 700 can be performed live, while the system is in operation, without the need for a test target or special calibration setup. As another example, method 700 can be performed regularly, to compensate for any de-calibration that may happen while the stereo system is in operation. In specific embodiments, the method 700 can run in the background on a stereo imaging system and can result in a continuous adjustment of the system.

[0075] While the specification has been described in detail with respect to specific embodiments of the invention, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily conceive of alterations to, variations of, and equivalents to these embodiments. Any of the method steps discussed above can be conducted by a processor operating with a computer-readable non-transitory medium storing instructions for those method steps. The computer-readable medium may be memory within a personal user device or a network accessible memory. Although examples in the disclosure were generally directed to using the left camera as the reference, the same approaches could be utilized with the right camera as the reference. Additionally, the stereo camera system may be set up with the cameras arranged vertically with a y-displacement rather than an x- displacement. In this case, y-disparities may be analyzed for calibration. These and other modifications and variations to the present invention may be practiced by those skilled in the art, without departing from the scope of the present invention, which is more particularly set forth in the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A method (700) for calibrating a stereo imaging system (200, 300), comprising: extracting (702) a set of feature points from a first image captured from a first view of a stereo image and a second image captured from a second view of the stereo image; matching (704) the set of feature points between the first image and the second image to form a list of matched points; and iteratively, using the list of matched points, conducting a set of steps comprising: (i) adjusting (708) a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y- dimension parallax value in the list of matched points; (ii) determining (710) a set of x- dimension parallax values for the set of feature points; and (iii) adjusting (712) a convergence calibration value (Ry) of the stereo imaging system (200, 300).

2. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1, further comprising: applying (718) rectification matrices to images obtained from the first view and the second view based on the Rx of the stereo imaging system (200, 300), the Ry of the stereo imaging system (200, 300), and the Rz of the stereo imaging system (200, 300).

3. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1, wherein: additional iterations of the set of steps are conducted (706) in response to detecting that an x-dimension parallax value in the set of x-dimension parallax values is not rational.

4. The method (700) for calibrating a stereo imaging system (200, 300) from claim 3, wherein: the x-dimension parallax values are rational when they are negative; and the x-dimension parallax values are not rational when they are positive.

5. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1, wherein: additional iterations of the set of steps are conducted (706) until a subset of the set of x- dimension parallax values are all rational; and a cardinality of the subset and a cardinality of the set differ by a tolerance value.

6. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1, wherein: additional iterations of the set of steps are conducted (706) until the set of x-dimension parallax values are all rational.

7. The method (600, 700) for calibrating a stereo imaging system (200, 300) from claim 6, wherein: the x-dimension parallax values are rational when they are negative; and the x-dimension parallax values are not rational when they are positive.

8. The method (600, 700) for calibrating a stereo imaging system (200, 300) from claim 1, further comprising:Determining (720), using the list of matched points, a depth map.

9. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1 wherein: determining (710) the set of x-dimension parallax values for the set of feature points is performed after adjusting (708) the tilt calibration value (Rx) and the roll calibration value (Rz); and adjusting (712) the convergence calibration value (Ry) of the stereo imaging system (200, 300) is performed after, and based on, determining (710) the set of x-dimension parallax values for the set of feature points.

10. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1 wherein: extracting (702) the set of feature points, matching (704) the set of feature points, and conducting (706) the set of steps are performed while the stereo imaging system (200, 300) is in operation and without a calibration pattern.

11. The method (700) for calibrating a stereo imaging system (200, 300) from claim 10 wherein: extracting (702) the set of feature points, matching (704) the set of feature points, and conducting (706) the set of steps are performed periodically while the stereo imaging system (200, 300) is in operation.

12. The method (700) for calibrating a stereo imaging system (200, 300) from claim 10 wherein: extracting (702) the set of feature points, matching (704) the set of feature points, and conducting (706) the set of steps are performed continuously while the stereo imaging system (200, 300) is in operation.

13. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1 wherein adjusting (712) the convergence calibration value (Ry) of the stereo imaging system (200, 300) comprises: adjusting (714) the convergence calibration value (Ry) by a fixed amount.

14. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1 wherein (712) adjusting the convergence calibration value (Ry) of the stereo imaging system (200, 300) comprises: adjusting (716) the convergence calibration value (Ry) by an amount based on an absolute value of a largest nonrational x-dimension parallax value in the set of x-dimension parallax values.

15. The method (700) for calibrating a stereo imaging system (200, 300) from claim 1, wherein: extracting (702) the set of feature points is based on detecting a shift in the first view, the second view, or both.

16. The method (600, 700) for calibrating a stereo imaging system (200, 300) from claim 1, wherein: extracting (702) the set of feature points is based on detecting a manual input.

17. One or more non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to conduct a method (700) comprising: extracting (702) a set of feature points from a first image captured from first view of a stereo image and a second image captured from a second view of the stereo image; matching (704) the set of feature points between the first image and the second image to form a list of matched points; and iteratively, using the list of matched points, conducting (706) a set of steps comprising: (i) adjusting (708) a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y- dimension parallax value in the list of matched points; (ii) determining (710) a set of x- dimension parallax values for the set of feature points; and (iii) adjusting (712) a convergence calibration value (Ry).

18. The one or more non-transitory computer-readable medium from claim 17, the method (700) further comprising: applying (718) rectification matrices to images obtained from the first view and the second view based on the Rx, the Ry, and the Rz.

19. The one or more non-transitory computer-readable medium from claim 17, wherein:additional iterations of the set of steps are conducted (706) in response to detecting that an x-dimension parallax value in the set of x-dimension parallax values is not rational.

20. The one or more non-transitory computer-readable medium from claim 17, wherein: additional iterations of the set of steps are conducted (706) until the set of x-dimension parallax values are all rational.

21. A system (200, 300) comprising: a stereo camera (201, 202, 301, 302) having a first sensor and a second sensor; one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the system (200, 300) to conduct a method (700) comprising: extracting (702) a set of feature points from a first image captured, by the first sensor, from a first view of a stereo image and a second image captured, by the second sensor, from a second view of the stereo image; matching (704) the set of feature points between the first image and the second image to form a list of matched points; and iteratively, using the list of matched points, conducting (706) a set of steps comprising: (i) adjusting (708) a tilt calibration value (Rx) and a roll calibration value (Rz) based on a y-dimension parallax value in the list of matched points; (ii) determining (710) a set of x-dimension parallax values for the set of feature points; and (iii) adjusting (712) a convergence calibration value (Ry) of the system (300, 400).

Images