Real-time camera calibration and overlay for vehicle camera monitor system

Abstract

A method for calibrating a camera position on a vehicle includes receiving an image from a camera at a controller, the image including a portion of a vehicle to which the camera is mounted. The controller identifies a plurality of features on the portion of the vehicle in the image. The controller matches the identified plurality of features to a saved plurality of features. The controller determines a true extrinsic based upon step the matched plurality of features, wherein the true extrinsic relates a position and orientation of the camera in six degrees of freedom to the vehicle.

Description

PA991580WO - WO-67950-310 PCT REAL-TIME CAMERA CALIBRATION AND OVERLAY FOR VEHICLE CAMERA MONITOR SYSTEM TECHNICAL FIELD

[0001] This disclosure relates to real-time camera calibration and overlay for vehicle camera monitor system.BACKGROUND

[0002] Mirror replacement systems, and camera systems for supplementing mirror views, are utilized in commercial vehicles to enhance the ability of a vehicle operator to see a surrounding environment. Camera monitoring systems (CMS) utilize one or more cameras to provide an enhanced field of view to a vehicle operator. In some examples, the mirror replacement systems cover a larger field of view than a conventional mirror or include views that are not fully obtainable via a conventional mirror.

[0003] During operation of the vehicle, certain CMS features rely on and utilize multiple aspects of the images generated by the CMS cameras for object detection, operation of direct CMS functions (e.g. mirror replacement displays), provision of data to other vehicle systems, and the like. Some of these systems and functions utilize features within the image, such as a trailer corner or a static marker located at a known height and position relative to the cameras in order to determine a reference position within the image.

[0004] Existing systems either assume that the camera is at a predefined “stock” position relative to the ground plane, with the stock position being determined at assembly or assume that the position is at a calibration position determined via a calibration performed while the truck is stationary. Reliance on pre-existing calibrated reference positions can result in less accuracy when the camera position has changed since the default height was selected. The camera position and orientation may change due to multiple reasons such as foreign object impact or mechanical wear and tear. As a result, the camera would need to be re-calibrated. Otherwise, image processing features may suffer degraded performance due to incorrect camera extrinsic.PA991580WO - WO-67950-310 PCT

[0005] One existing system corrects for some changes to the camera position by calculating changes in a yaw angle and a pitch angle of the camera compared to an ideal position and orientation of the camera relative to the vehicle. Another existing system calculates height changes of the camera relative to the ideal position of the camera relative to the vehicle.SUMMARY

[0006] In some aspects, the techniques described herein relate to a method for calibrating a camera position on a vehicle including receiving an image from a camera at a controller, the image including a portion of a vehicle to which the camera is mounted. The controller identifies a plurality of features on the portion of the vehicle in the image. The controller matches the plurality of identified features to a saved plurality of features. The controller determines a true extrinsic based upon the matched features, wherein the true extrinsic relates a position and orientation of the camera in six degrees of freedom to the vehicle.

[0007] In some aspects, the techniques described herein may relate to a method, wherein the camera may be a rear facing arm mounted camera.

[0008] In some aspects, the techniques described herein may relate to a method further including comparing positions of the plurality of identified features to a plane. A subset of the plurality of identified features may be discarded based upon the comparison.

[0009] In some aspects, the techniques described herein may relate to a method further wherein the controller determines the true extrinsic based upon a saved extrinsic. The saved extrinsic may relate a saved position and orientation of the camera in six degrees of freedom to the vehicle.

[0010] In some aspects, the techniques described herein may relate to a method wherein the saved plurality of features may be based upon the saved position and orientation of the camera.PA991580WO - WO-67950-310 PCT

[0011] In some aspects, the techniques described herein may relate to a method wherein the camera is a rear facing arm mounted camera.

[0012] In some aspects, the techniques described herein may relate to a method wherein the vehicle is a tractor.

[0013] In some aspects, the techniques described herein may relate to a method wherein the controller determines a projective matrix based upon steps b) and c) and optionally based upon the saved extrinsic.

[0014] In some aspects, the techniques described herein may relate to a method wherein the controller generates an overlay based upon the projective matrix and displays the overlay and the image from the camera on a display.

[0015] In some aspects, the techniques described herein relate to a camera monitoring system (CMS) for a vehicle, the CMS including: a camera; a controller including at least one processor and at least one memory, the at least one memory stores instructions which when executed by the at least one processor cause the CMS to determine a position and an orientation of the camera relative to a vehicle on which the camera is mounted by: a) receiving an image from the camera, the image including a portion of the vehicle on which the camera is mounted; b) detecting a plurality of features on the portion of the vehicle in the image; c) matching the plurality of features identified in step b) to a saved plurality of features; and d) determining an extrinsic based upon step c), wherein the extrinsic relates a position and orientation of the camera in six degrees of freedom relative to the vehicle.

[0016] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the camera is mounted to the vehicle via an arm and wherein the camera faces rearwardly of the vehicle.

[0017] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the instructions would further cause the CMS to: e) compare positions of the plurality of features identified in step b) to a plane; and f) discard a subset of the plurality of features identified in step b) based upon step e).PA991580WO - WO-67950-310 PCT

[0018] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the controller performs step d) based upon a saved extrinsic. The saved extrinsic may relate a saved position and orientation of the camera in six degrees of freedom to the vehicle.

[0019] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the saved plurality of features may be based upon the saved position and orientation of the camera.

[0020] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the controller performs step d) based upon a saved extrinsic.

[0021] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the controller determines a projective matrix based upon steps b) and c) and optionally based upon the saved extrinsic.

[0022] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the camera is a rear facing arm mounted camera.

[0023] In some aspects, the techniques described herein may relate to a camera monitoring system wherein the vehicle is a tractor.BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The disclosure can be further understood by reference to the following detailed description when considered in connection with the accompanying drawings wherein:

[0025] Figure 1 A is a schematic front view of a commercial truck with a camera monitoring system (CMS) used to provide at least Class II and Class IV views.

[0026] Figure 1 B is a schematic top elevational view of a commercial truck with a camera monitoring system providing Class II, Class IV, Class V and Class VI views.

[0027] Figure 2 is a schematic top perspective view of a vehicle cabin including displays and interior cameras.PA991580WO - WO-67950-310 PCT

[0028] Figures 3 shows one of the cameras of Figures 1A-1B relative to a transformation matrix.

[0029] Figure 4 illustrates an exemplary raw image of a vehicle received from the camera relative to a designed location of the vehicle.

[0030] Figure 5 shows a schematic of a method for operating a CMS.

[0031] Figure 6 shows a display in which an example overlay is projected onto a camera view.

[0032] The embodiments, examples and alternatives of the preceding paragraphs, the claims, or the following description and drawings, including any of their various aspects or respective individual features, may be taken independently or in any combination. Features described in connection with one embodiment are applicable to all embodiments, unless such features are incompatible.DETAILED DESCRIPTION

[0033] A schematic view of a commercial truck and trailer 10 is illustrated in Figures 1A and 1B. The commercial truck and trailer 10 includes a tractor 12 pulling a trailer 14. It should be understood that the tractor 12 and / or trailer 14 may be any configuration and each of the tractor 12 and trailer 14 may be considered a “vehicle” as used herein. Although a commercial truck and trailer is contemplated in this disclosure, the invention may also be applied to other types of vehicles. The commercial truck and trailer 10 incorporates a camera monitoring system (CMS) 15 (Fig. 2) that has driver and passenger side camera arms 16a, 16b mounted to the outside of the tractor 12. If desired, the camera arms 16a, 16b may include conventional mirrors integrated with them as well, although the CMS 15 can be used to entirely replace mirrors. In additional examples, each side can include multiple camera arms, each arm housing one or more cameras and / or mirrors.

[0034] Each of the camera arms 16a, 16b includes a base that is secured to, for example, the tractor 12. A pivoting arm is supported by the base and may articulatePA991580WO - WO-67950-310 PCT relative thereto. At least one rearward facing camera 20a, 20b is arranged respectively within camera arms. The exterior cameras 20a, 20b respectively provide an exterior field of view FOVEX1, FOVEX2that each include at least one of the Class II and Class IV views (Fig. 1 B), which are legal prescribed views in the commercial trucking industry. The Class II view on a given side of the commercial truck and trailer 10 is a subset of the class IV view of the same side of the commercial truck and trailer 10. Multiple cameras also may be used in each camera arm 16a, 16b to provide these views, if desired. Class II and Class IV views are defined in European R46 legislation, for example, and the United States and other countries have similar drive visibility requirements for commercial trucks. Any reference to a “Class” view is not intended to be limiting, but is intended as exemplary for the type of view provided to a display by a particular camera. Each arm 16a, 16b may also provide a housing that encloses electronics, e.g., a controller 30, that are configured to provide various features of the CMS 15.

[0035] First and second video displays 18a, 18b are arranged on each of the driver and passenger sides within the tractor 12 on or near the A-pillars 19a, 19b to display Class II and Class IV views on its respective side of the commercial truck and trailer 10, which provide rear facing side views along the commercial truck and trailer 10 that are captured by the exterior cameras 20a, 20b. In some examples, the first and second video displays 18a, 18b operate as mirror replacement displays, while in other examples they can be operated as supplemental displays to physical mirrors.

[0036] If video of Class V and Class VI views are also desired, a camera housing 16c and camera 20c may be arranged at or near the front of the commercial truck and trailer 10 to provide those views (Fig. 1B). A third display 18c arranged within the tractor 12 near the top center of the windshield can be used to display the Class V and Class VI views, which are toward the front of the commercial truck and trailer 10, to the driver.

[0037] If additional video views are desired, camera housings can be positioned at the sides and rear of the commercial truck and trailer 10 to provide fields of view including some or all of these zones of the commercial truck and trailer 10. In suchPA991580WO - WO-67950-310 PCT examples, the third display 18c can include one or more frames displaying these views. Alternatively, additional displays can be added near the first, second and third displays 18a, 18b, 18c and provide a display dedicated to providing one or more of these views. The displays 18a, 18b, 18c face a driver region 24 within the cabin 22 where an operator is seated on a driver seat 26. The location, size and field(s) of view streamed to any particular display may vary from the configurations described in this disclosure and still incorporate the disclosed invention.

[0038] The controller 30 is in communication with the cameras 20 and the displays 18. The controller 30 is configured to implement the various functionality disclosed in this application. The controller 30 may include one or more discrete units.

[0039] In terms of hardware architecture, the controller 30 includes at least one processor, at least one memory, and one or more input and / or output (I / O) device interface(s) that are communicatively coupled via a local interface. The local interface can include, for example but not limited to, one or more buses and / or other wired or wireless connections. The local interface may have additional elements, which are omitted for simplicity, such as controllers, buffers (caches), drivers, repeaters, and receivers to enable communications. Further, the local interface may include address, control, and / or data connections to enable appropriate communications among the aforementioned components.

[0040] The controller 30 includes at least one processor for executing software, particularly software stored in the at least one memory, which when executed by the at least one processor performs the functions described herein. The controller 30 can be a custom made or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set) or generally any device for executing software instructions.

[0041] The memory can include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, VRAM, etc.)) and / or nonvolatile memory elements (e.g., ROM, hard drive, tape, CD-ROM, etc.).PA991580WO - WO-67950-310 PCT Moreover, the memory may incorporate electronic, magnetic, optical, and / or other types of storage media. The memory can also have a distributed architecture, where various components are situated remotely from one another, but can be accessed by the processor.

[0042] The software in the memory may include one or more separate programs, each of which includes an ordered listing of executable instructions for implementing logical functions. A system component embodied as software may also be construed as a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed. When constructed as a source program, the program is translated via a compiler, assembler, interpreter, or the like, which may or may not be included within the memory.

[0043] The disclosed input and output devices that may be coupled to system I / O interface(s) may include input devices, for example but not limited to, a keyboard, mouse, scanner, microphone, camera, mobile device, proximity device, etc. Further, the output devices, for example but not limited to, a printer, display, etc. Finally, the input and output devices may further include devices that communicate both as inputs and outputs, for instance but not limited to, a modulator / demodulator (modem; for accessing another device, system, or network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, etc.

[0044] When the controller 30 is in operation, the processor can be configured to execute software stored within the memory, to communicate data to and from the memory, and to generally control operations of the computing device pursuant to the software. Software in memory, in whole or in part, is read by the processor, perhaps buffered within the processor, and then executed. Execution of the instructions in the programs in the memory causes the controller to perform the functions described herein.

[0045] Referring to Fig. 2, for each camera, the controller 30 stores a saved extrinsic 32, a true extrinsic 34, and projective matrix 36.

[0046] The saved extrinsic 32 is an ideal or designed extrinsic of the camera relative to the tractor 12 specifying all six degrees of freedom. The saved extrinsic 32PA991580WO - WO-67950-310 PCT specifies the position and orientation of the camera 20 (i.e. camera 20a or camera 20b) relative to the tractor 12, e.g. three-dimensional coordinates (x, y, z) and Euler angles. Other coordinate systems could be used.

[0047] The true extrinsic 34 is the actual extrinsic of the camera relative to the tractor 12 specifying all six degrees of freedom. The true extrinsic 34 specifies the actual position and orientation of the camera 20 relative to the tractor 12, e.g. three-dimensional coordinates (x, y, z) and Euler angles. Other coordinate systems could be used. As explained herein, the true extrinsic 34 will often be different from the saved extrinsic 32 because the camera 20 will not be installed in exactly the designed position and orientation, or because the camera 20 will be subsequently moved or rotated, such as by mechanical impact, or because of wear and tear, or because of some combination of these factors.

[0048] The projective matrix 36 maps objects viewed by the camera 20 between a three-dimensional space and a two-dimensional plane.

[0049] Figure 3 shows the exterior camera 20a on camera arm 16a; however, it should be recognized that the calibration techniques disclosed herein could be used for any of the cameras 20. For cameras mounted to the tractor 12, the position and orientation would be calibrated relative to the tractor 12. For cameras mounted to the trailer 14, the position and orientation would be calibrated relative to the trailer 14.

[0050] As shown in Figure 3, the exterior camera 20a will have a position and orientation relative to the tractor 12 that can be described in six degrees of freedom. As is well-known, the six degrees of freedom may include a position in three orthogonal axes (e.g. x, y, and z) and a rotation about each of the three orthogonal axes (e.g. roll, pitch, and yaw). Other coordinate systems that fully describe the relative position and orientation could also be used.

[0051] As mentioned above, there is an ideal camera position and orientation relative to the tractor 12. The saved extrinsic 32 may set forth a hypothetical or a designed or an ideal camera position and orientation relative to the tractor 12 and relative to points and contours on the tractor 12. Alternatively, the saved extrinsic may be a previouslyPA991580WO - WO-67950-310 PCT saved and previously updated extrinsic. The techniques described herein create the true extrinsic 34, which describes in six degrees of freedom the relative position and orientation of the camera 20 relative to the tractor 12. The techniques described herein also determine the projective matrix 36.

[0052] Figure 4 shows an example image from the camera 20a. The image shows a portion of the exterior of the tractor 12. In broken lines, Figure 4 shows the relative position of the portion of the exterior of the tractor 12’ in the ideal image, as if taken from a camera at the ideal position. More than one hundred, and more preferably more than five hundred points on the exterior of the tractor 12 are located in the images (a significantly reduced number of representative points is shown for clarity). These points on the exterior of the tractor 12, e.g. edges, corners, joints, openings, etc on the body of the tractor 12 are fixed relative to one another and relative to the rest of the tractor 12 and do not change over time. The positions of these points on the exterior of the tractor 12 are located in the actual image and linked to the corresponding points in the ideal image. These points in the actual image and in the ideal image are used to determine deltas between the saved extrinsic 32 and the true extrinsic 34. in six degrees of freedom.

[0053] Figure 5 schematically illustrates a method for determining the true extrinsic 34 and the projective matrix 36. The calibration could be performed periodically (e.g. at every startup of the CMS) or upon request by the user (via a user interface).

[0054] A raw image 40 is received from the camera 20 (such as camera 20a or camera 20b). A feature detection module 42 in the controller 30 analyzes the raw image 40 from the camera 20 to find the points, edges, contours and / or other features of Figure 4. The feature detection module 42 may use ORB (Oriented Fast and Rotated Brief) or SURF method (Speeded Up Robust Features) or other known image analysis techniques.

[0055] Saved features 44 are a template or 3D model of the tractor 12 shape, including the points, edges, contours and / or other features of Figure 4. A feature matching module 46 compares and matches the points, edges, contours and / or other features detected by the feature detection module 42 based upon the actual, raw image 40 from camera 20 to the saved features 44.PA991580WO - WO-67950-310 PCT

[0056] In an un-distortion and co-plane clustering module 48, undistortion is performed on the matched features, such as the correction of lens distortion. Additionally, in un-distortion and co-plane clustering module 48, the features (points, edges, contours and / or other features) that are within a certain threshold of a plane are clustered. Such features that are outside the threshold of the plane are discarded. The side of the tractor 12 is substantially planar. A large majority of detected features should be within a threshold of a plane. Features that are detected significantly outside that plane are not part of the tractor 12 but may be features from the ground or other objects in the background of the raw image 40. Those features outside the plane are discarded in the un-distortion and co-plane clustering module 48.

[0057] An extrinsic calculation module 52 receives the matched features and estimates the delta / changes in real time between the saved features 44 and the detected features in the raw image 40. The extrinsic calculation module 52 estimates the delta / changes in the extrinsic in real time and using the saved extrinsic.

[0058] Based upon the deltas or changes between the saved features 44 and the detected features in the raw image 40 and based upon the saved extrinsic 32, the extrinsic calculation module 52 calculates the true extrinsic 34 and the projective matrix 36.

[0059] The projective matrix 36 can then be used in any of several ways. The actual position of an object in the two-dimensional image can be determined in three dimensions correctly relative to the tractor 12. Overlays onto the object in the image as displayed on one of the displays 18 will be more accurately aligned with the actual positions of the objects in the image, including the tractor 12 and the trailer 14. Locations of objects relative to the tractor 12 will also be more accurately determined based upon the projective matrix 36.

[0060] Alternatively, or additionally, the extrinsic calculation module 52 can compare the current camera position and orientation to the saved extrinsic 32 at startup or periodically and warn the driver if the deltas exceed a predetermined threshold. The CMS can then give the driver the option of performing a full recalibration of the cameraPA991580WO - WO-67950-310 PCT position and orientation. The extrinsic calculation module 52 can wait to calculate the true extrinsic 34 and the projective matrix 36 until receiving confirmation from the driver.

[0061] A more specific example technique will be described below. The matched feature points (Fig. 4) from the two images taken from camera location A and B are denoted as as XAand XB. The feature points were matched in feature matching module 46 and as explained above, some feature points were discarded in the undistortion and co-plane clustering module 48. Camera location A is the ideal camera location and camera location B is the actual camera location

[0062] Then the fundamental matrix F relates the matched points in the two images through the following:

[0063] X FXA= 0

[0064] F can be estimated using methods like the eight-point algorithm, which is well-known in computer vision research area.

[0065] Suppose we know the camera intrinsic as / fx0 cx\

[0066] K = 0 fycyo 0 17

[0067] The essential matrix can be estimated as

[0068] Ess= KTFK

[0069] Then we can perform singular value decomposition with Ess

[0070] Ess= UXVT

[0071] To get U and V, where U is an m x m complex unitary matrix with nonnegative numbers on the diagonal and V is an n x n complex unitary matrix.

[0072] The translation vector is the last column of the U matrix

[0073] t = U(:,3)

[0074] The rotation matrix R has two candidates

[0075] Ri = UWVT

[0076] R2= UWTVT

[0077] WherePA991580WO - WO-67950-310 PCT 0 -1 O'

[0078] W = 0 0 0.0 0 1.

[0079] And the correct rotation matrix is selected through triangulation depth method.

[0080] In the end, the extrinsic from A to B is determined as

[0082] And the extrinsic at position B with respect to vehicle ISO is

[0083] EB — E E -B

[0084] Where EBis the true extrinsic 34 and EAis the saved extrinsic 32.

[0085] The projective matrix 36 (P) is the product of the camera intrinsic K and the true extrinsic 34.

[0086] P = [ ]3X4[ ]4X4

[0087] As is known, the projective matrix 36 projects points from the real world 3-dimensional space onto a 2-dimensional plane, such as one of the displays 18a, 18b, 18c. One example way of determining the projection is shown below, where x, y, z are the three-dimensional coordinates of the point in the real world and « v*) are pixels without normalization.

[0089] In the final two-dimensional image, the pixel location (u, v) would be:

[0090] u =

[0091] v =

[0092] There are many known potential uses for the projective matrix. For example, the projective matrix 36 is used to project (i.e. overlay) a warning distance line onto the current view from one of the cameras one of the displays 18a, 18b, 18c.PA991580WO - WO-67950-310 PCT

[0093] The projective matrix 36 can also be used to predict vehicle trajectory onto one of the displays 18a, 18b, 18c, such as by overlaying one or more lines showing the predicted vehicle trajectory onto the current view from the camera, so the driver can have a better visualization.

[0094] As shown in Fig. 6, a warning line 62 can be overlayed onto the camera view relative to an object 60 on the display.

[0095] In accordance with the provisions of the patent statutes and jurisprudence, exemplary configurations described above are considered to represent a preferred embodiment of the invention. However, it should be noted that the invention can be practiced otherwise than as specifically illustrated and described without departing from its spirit or scope. Alphanumeric identifiers for operations or method steps are for ease of reference in dependent claims and do not indicate a required sequence unless explicitly stated in the claims.

Claims

PA991580WO - WO-67950-310 PCT CLAIMSWhat is claimed is:

1. A method for calibrating a camera position on a vehicle including:a) receiving an image from a camera at a controller, the image including a portion of a vehicle to which the camera is mounted;b) the controller identifying a plurality of features on the portion of the vehicle in the image;c) the controller matching the plurality of features identified in step b) to a saved plurality of features; andd) the controller determining a true extrinsic based upon step c), wherein the true extrinsic relates a position and orientation of the camera in six degrees of freedom to the vehicle.

2. The method of claim 1, wherein the camera is a rear facing arm mounted camera.

3. The method of claim 1 or 2, further including:e) comparing positions of the plurality of features identified in step b) to a plane; andf) discarding a subset of the plurality of features identified in step b) based upon step e).

4. The method of any preceding claim, further wherein the controller performs step d) based upon a saved extrinsic, wherein the saved extrinsic relates a saved position and orientation of the camera in six degrees of freedom to the vehicle.

5. The method of claim 4 wherein the saved plurality of features are based upon the saved position and orientation of the camera.PA991580WO - WO-67950-310 PCT6. The method of any preceding claim, wherein the vehicle is a tractor.

7. The method of any one of claims 4 to 6, wherein the controller determines a projective matrix based upon steps b) and c) and based upon the saved extrinsic.

8. The method of claim 7, wherein the controller generates an overlay based upon the projective matrix and displays the overlay and the image from the camera on a display.

9. A camera monitoring system (CMS) for a vehicle, the CMS comprising:a camera;a controller including at least one processor and at least one memory, the at least one memory stores instructions which when executed by the at least one processor cause the CMS to determine a position and an orientation of the camera relative to a vehicle on which the camera is mounted by:a) receiving an image from the camera, the image including a portion of the vehicle on which the camera is mounted;b) detecting a plurality of features on the portion of the vehicle in the image; c) matching the plurality of features identified in step b) to a saved plurality of features; andd) determining an extrinsic based upon step c), wherein the extrinsic relates a position and orientation of the camera in six degrees of freedom relative to the vehicle.

10. The camera monitoring system of claim 9, wherein the camera is mounted to the vehicle via an arm and wherein the camera faces rearwardly of the vehicle.PA991580WO - WO-67950-310 PCT 11. The camera monitoring system of claim 9 or 10, wherein the instructions would further cause the CMS to:e) compare positions of the plurality of features identified in step b) to a plane; and f) discard a subset of the plurality of features identified in step b) based upon step e).

12. The camera monitoring system of any one of claims 9 to 11, wherein the controller performs step d) based upon a saved extrinsic.

13. The camera monitoring system of claim 12, wherein the saved extrinsic relates a saved position and orientation of the camera in six degrees of freedom to the vehicle.

14. The camera monitoring system of claim 12 or 13, wherein the saved plurality of features are based upon the saved position and orientation of the camera.

15. The camera monitoring system of any one of claims 12 to 14, wherein the controller determines a projective matrix based upon steps b) and c) and based upon the saved extrinsic.

16. The camera monitoring system of any one of claims 9 to 15, wherein the camera is a rear facing arm mounted camera.

17. The camera monitoring system of any one of claims 9 to 16, wherein the vehicle is a tractor.

Images