Image-based reference location identification and use for camera surveillance systems

Abstract

A method for determining an image-referenced location includes receiving at least one image from at least one camera at a controller, the image including a road lane defined by two distinguishable lane lines, the camera being a component of a CMS (camera monitoring system) for a vehicle, using image-based analysis to identify an inner edge of each lane line in the two identified lane lines, and using the controller to identify vanishing points of the two identified lane lines by extending the identified inner edge of each lane line to a point where the extended inner edges intersect, and providing the vanishing points to the at least one CMS system as image-referenced locations.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Application No. 63 / 358,926, filed July 7, 2022.

[0002] The present disclosure relates to image-based detection of reference points in image feeds from cameras in camera surveillance systems. [Background technology]

[0003] Mirror replacement systems, and camera systems that supplement the mirror view, are utilized in commercial vehicles to enhance the vehicle operator's ability to view the surrounding environment. Camera surveillance systems (CMS) utilize one or more cameras to provide the vehicle operator with an enhanced field of view. In some instances, mirror replacement systems cover a wider field of view than traditional mirrors or include views not fully available through traditional mirrors.

[0004] During vehicle operation, certain CMS functions rely on and utilize multiple aspects of the images generated by the CMS cameras for object detection, operation of direct CMS functions (e.g., mirror replacement display), providing data to other vehicle systems, etc. Some of these systems and functions utilize features within the images, such as trailer corners or static markers placed at known heights and positions relative to the camera, to determine reference locations within the images.

[0005] Existing systems assume that the camera is in a predefined "stock" position relative to the ground surface that is determined at assembly time, or that its position is in a calibrated position that is determined by a calibration performed while the truck is stationary. Based on these assumptions, the position of the image relative to the ground surface is assumed. Relying on an existing calibrated reference position can result in reduced accuracy if the camera position is changed after the default height is selected. Summary of the Invention

[0006] In one exemplary embodiment, a method for determining an image-referenced location includes receiving at least one image from a camera at a controller, the image including a road lane defined by two identified lane lines, the camera being a component of a camera surveillance system (CMS) for a vehicle, using image-based analysis to identify an inner edge of each of the two identified lane lines, and using the controller to identify vanishing points of the two identified lane lines by extending the identified inner edge of each identified lane line to a point where the extended identified inner edges intersect, and providing the vanishing points to the at least one CMS system as image-referenced locations.

[0007] In a further embodiment of any of the above, the at least one camera is at least one rear-facing wing-mounted camera.

[0008] In a further embodiment of any of the above, the identified inside edge of each of the identified lane lines is the inward-facing edge of a lane bounded by the two identified lane lines.

[0009] In a further embodiment of any of the above, the CMS system includes a real-time camera height determination system that determines the camera height relative to the ground surface by determining an area of a triangle defined by the vanishing point and an edge of each identified lane line, the area of the triangle being converted to a corresponding camera height using the controller.

[0010] In a further embodiment of any of the above, converting the area of the triangle to a corresponding camera height using the controller includes identifying an entry in a lookup table that corresponds to the area of the triangle.

[0011] In a further embodiment of any of the above, the lookup table includes a series of triangle area ranges, each triangle area range within the series of triangle area ranges being correlated with a corresponding camera height.

[0012] In a further embodiment of any of the above, converting the area of the triangle to a corresponding camera height using the controller comprises inputting the determined area of the triangle into an equation, and determining the camera height as an output of the equation.

[0013] In a further embodiment of any of the above, the method further includes the steps of comparing the identified vanishing point with a reference vanishing point, and determining that the identified vanishing point is accurate in response to the identified vanishing point being within a threshold distance of the reference vanishing point.

[0014] In a further embodiment of any of the above, the method further comprises verifying that preconditions are met before using the controller to identify vanishing points of the two distinguishable lane lines.

[0015] In a further embodiment of any of the above, the precondition is that the vehicle speed exceeds a first threshold and the vehicle yaw is below a second threshold.

[0016] In a further embodiment of any of the above, the first threshold is at least 40 kilometers per hour and the second threshold is at least 1 degree per second.

[0017] In a further embodiment of any of the above, receiving the at least one image from at least one camera in a controller includes receiving a first image from a first camera and a second image from a second camera, wherein a first reference point is identified for the first image and a second reference point is identified for the second image, and the CMS includes a display alignment system configured to align a first representation of the first image and a second representation of the second image by positioning a reference point of the first image and a reference point of the second image at the same vertical height on corresponding display screens.

[0018] In a further embodiment of any of the above, positioning the reference points of the first image and the reference points of the second image at the same vertical height of the corresponding display screen includes aligning a raw image with one of a top edge of the image and a bottom edge of the image, determining a difference in vertical height between the corresponding reference points, and adjusting at least one of the first image and the second image so that the difference in vertical height between the corresponding reference points is zero.

[0019] In a further embodiment of any of the above, adjusting at least one of the first and second images includes cropping at least one of the first and second images and resizing at least one of the first and second images.

[0020] In an alternative exemplary embodiment, a camera surveillance system (CMS) for a vehicle includes first and second rear-facing cameras and a controller including at least a processor and a memory storing instructions configured to: determine, in real time, a camera height of at least one of the first and second rear-facing cameras by using image-based analysis to identify an inside edge of each of two identified lane lines; and, using the controller, identify a vanishing point of the two identified lane lines by extending the identified inside edge of each lane line to a point where the extended inside edges intersect. The vanishing point is provided to the at least one CMS system as an image reference location.

[0021] In a further embodiment of any of the above, the at least one CMS system includes a real-time height determination system that determines the area of a triangle defined by the vanishing point and an edge of each distinguishable lane line, and converts the area of the triangle to a corresponding camera height using the controller.

[0022] In a further embodiment of any of the above, converting the area of the triangle to a corresponding camera height using the controller includes identifying an entry in a lookup table that corresponds to the area of the triangle.

[0023] In a further embodiment of any of the above, the lookup table includes a series of triangle area ranges, each triangle area range within the series of triangle area ranges being correlated with a corresponding camera height.

[0024] In a further embodiment of any of the above, converting the area of the triangle to a corresponding camera height using the controller comprises inputting the determined area of the triangle into an equation, and determining the camera height as an output of the equation.

[0025] In a further embodiment of any of the above, the CMS further includes the steps of comparing the identified vanishing point with a reference vanishing point, and determining that the identified vanishing point is accurate in response to the identified vanishing point being within a threshold distance of the reference vanishing point.

[0026] In a further embodiment of any of the above, the at least one CMS system includes a display alignment system configured to align a first representation of a first image and a second representation of a second image by positioning a reference point of the first image and a reference point of the second image at the same vertical height on corresponding display screens. [Brief explanation of the drawings]

[0027] The present disclosure can be further understood by reference to the following detailed description taken in conjunction with the accompanying drawings.

[0028] [Figure 1A] FIG. 1 is a schematic front view of a commercial truck equipped with a camera surveillance system (CMS) used to provide at least Class II and Class IV views.

[0029] [Figure 1B] FIG. 1 is a schematic top view of a commercial truck equipped with a camera surveillance system providing Class II, Class IV, Class V, and Class VI views.

[0030] [Figure 2] FIG. 1 is a schematic top perspective view of a vehicle cab including a display and an interior camera.

[0031] [Figure 3] 1 is a flowchart illustrating a method for identifying reference points in an image based on an image received from a camera.

[0032] [Figure 4] 1 shows an exemplary raw image received from a camera for the present method.

[0033] [Figure 5] 5 shows lane line edge based reference point detection performed on the raw image of FIG.

[0034] [Figure 6] 10 illustrates alternative lane line edge based reference point detection performed on an alternative raw image.

[0035] [Figure 7] 4 illustrates an exemplary method for using the reference points of FIG. 3 to assist in generating a real-time camera height determination.

[0036] [Figure 8] The detected vanishing points and the geometric analysis of the detected lane line edges are shown in Figure 5.

[0037] [Figure 9] 4 shows an exemplary lookup table for use in the method of FIG. 3.

[0038] [Figure 10] 4 illustrates an exemplary method for aligning a CMS display using reference points identified by the method of FIG. 3.

[0039] The embodiments, examples and alternatives of the preceding paragraphs, the claims, or the following description and drawings, including any of their various aspects or their respective individual features, may be taken independently or in any combination. Features described in connection with one embodiment are applicable to all embodiments, except where such features are incompatible. DETAILED DESCRIPTION OF THE INVENTION

[0040] Schematic diagrams of a commercial vehicle 10 are shown in FIGS. 1A and 1B. The vehicle 10 includes a vehicle cab or tractor 12 for towing a trailer 14. It should be understood that the vehicle cab 12 and / or trailer 14 may be of any configuration. While commercial trucks are contemplated in this disclosure, the present invention is applicable to other types of vehicles. The vehicle 10 incorporates a camera surveillance system (CMS) 15 (FIG. 2) that includes driver and passenger side camera arms 16a, 16b mounted on the exterior of the vehicle cab 12. If desired, the camera arms 16a, 16b may also include conventional mirrors integrated therewith, although the CMS 15 may be used to replace the mirrors entirely. In additional examples, multiple camera arms may be included on each side, with each arm housing one or more cameras and / or mirrors.

[0041] Each camera arm 16a, 16b includes a base fixed to, for example, the cab 12. A pivoting arm is supported by the base and may be articulated relative thereto. At least one rear-facing camera 20a, 20b is disposed within each camera arm. Each exterior camera 20a, 20b has an exterior field of view (FOV) that includes at least one of a Class II view and a Class IV view (FIG. 1b), which are legally defined views in the commercial trucking industry. EX1 , FOV EX2 A Class II view of a given side of the vehicle 10 is a subset of a Class IV view of the same side of the vehicle 10. If desired, multiple cameras may be used in each camera arm 16 a, 16 b to provide these views. For example, Class II and Class IV views are defined in the European R46 legislation, and the United States and other countries also establish similar driving visibility requirements for commercial trucks. References to "Class" views are not intended to be limiting, but rather as an illustration of the type of view provided on the display by a particular camera. Each arm 16 a, 16 b may also provide a housing enclosing electronics, such as the controller 30, configured to provide various features of the CMS 15.

[0042] First and second video displays 18a, 18b are positioned on the driver's and passenger's sides, respectively, within the vehicle cab 12 on or near the A-pillars 19a, 19b and display Class II and Class IV views on each side of the vehicle 10, which provide rear-facing views along the vehicle 10 captured by exterior cameras 20a, 20b. In some examples, the first and second video displays 18a, 18b operate as mirror replacement displays, while in other examples, they may operate as auxiliary displays to physical mirrors.

[0043] If Class V and Class VI view footage is also desired, a camera housing 16c and camera 20c may be located at or near the front of the vehicle 10 to provide these views (FIG. 1b). A third display 18c located within the cab 12 near the top center of the windshield may be used to display Class V and Class VI views forward of the vehicle 10 to the driver.

[0044] If Class VIII view footage is required, camera housings can be positioned on the sides and rear of the vehicle 10 to provide a field of view that includes some or all of the Class VIII zone of the vehicle 10. In such an example, the third display 18c can include one or more frames displaying the Class VIII view. Alternatively, additional displays can be added near the first, second, and third displays 18a, 18b, 18c to provide dedicated displays providing the Class VIII view. The displays 18a, 18b, 18c face the driver's area 24 within the cab 22, where the driver is seated in the driver's seat 26. The location, size, and field of view(s) streamed to a particular display can vary from the configurations described herein and still encompass the invention of this disclosure.

[0045] The controller 30 is in communication with the camera 20 and the display 18. The controller 30 is configured to implement the various functions disclosed in this application and may include one or more individual units.

[0046] With respect to hardware architecture, such a computing device may include a processor, memory, and one or more input / output (I / O) device interfaces communicatively coupled via a local interface. The local interface may include, for example, but is not limited to, one or more buses and / or other wired or wireless connections. The local interface may also include additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers that enable communication, which are omitted for simplicity. Additionally, the local interface may include address, control, and / or data connections to enable appropriate communication between the aforementioned components.

[0047] The controller 30 may be a hardware device for executing software, particularly software stored in a memory, and may be a custom or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chipset), or any device for general-purposely executing software instructions.

[0048] The memory may include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, VRAM, etc.)) and / or non-volatile memory elements (e.g., ROM, hard drive, tape, CD-ROM, etc.). Furthermore, the memory may incorporate electronic, magnetic, optical, and / or other types of storage media. The memory may have a distributed architecture where various components are located remotely from each other but are accessible by the processor.

[0049] Software in memory may include one or more separate programs, each containing an ordered list of executable instructions for implementing a logical function. A system component embodied as software may be constructed as a source program, an executable program (object code), a script, or any other entity containing a set of instructions to be executed. If constructed as a source program, the program is translated via a compiler, assembler, interpreter, etc., which may or may not be contained in memory.

[0050] Input / output devices of the present disclosure that may be coupled to the system I / O interface(s) may include, but are not limited to, input devices such as a keyboard, mouse, scanner, microphone, camera, mobile device, proximity device, etc. They may also include, but are not limited to, output devices such as a printer, display, etc. Finally, input / output devices may further include devices that communicate as both input and output, such as, but are not limited to, a modulator / demodulator (i.e., for accessing another device, system, or network), a radio frequency (RF) or other transceiver, a telephone interface, a bridge, a router, etc.

[0051] When controller 30 is in operation, the processor may be configured to execute software stored in the memory, communicate data to and from the memory, and generally control the operation of the computing device in accordance with the software. The software in the memory is read, in whole or in part, by the processor and often buffered within the processor before being executed.

[0052] Certain functions of the CMS 15, such as CMS display alignment and real-time camera height determination, rely on reference points in the image to provide accurate estimation, analysis, alignment, and HMI placement. In existing systems, reference points are provided by placing markers or stickers at known locations on the trailer or by placing fiducial markers at known locations relative to the camera while the vehicle is in a stationary position. The reference points provide accurate information about the image location relative to the ground surface the vehicle is traveling on. This information is then used by one or more CMS systems.

[0053] To provide a currently accurate reference point within the image, the CMS includes a method 300 that uses image features and image analysis to identify a vanishing point reference location 530 located outside the image, either within the image or at a location extrapolated from the image. Method 300 is shown in FIG. 5 and is stored within the CMS or in memory in communication with controller 30. Method 300 is executed by controller 30. In other examples, method 300 may be stored and executed by other controller system(s) in communication with controller 30. Although shown and described in FIG. 5 with respect to a single image, it will be understood that multiple images can be subjected to the same process simultaneously, with reference locations determined from each image.

[0054] Initially, a controller operating the CMS 15 receives raw images 304 from the camera. Simultaneously, the controller receives a set of information 302 including mileage information. The mileage information includes at least the rate of change of the vehicle's position and yaw rate. If the set of information meets predefined conditions, the controller determines that the vehicle is traveling straight and at a speed above a required threshold so that the method 300 can proceed.

[0055] For example, if the yaw rate is less than approximately 1 degree per second and the speed is greater than 40 kilometers per hour, the preconditions indicate that the vehicle is traveling in a straight line, and method 300 can provide an accurate real-time reference position determination. In alternative examples, the speed threshold may be set at 50 kilometers per hour, or within a range of 40 to 50 kilometers per hour. In other examples, other data received directly or indirectly from a vehicle bus, such as a CAN bus, may be used to identify that the preconditions corresponding to forward motion are met. Furthermore, the described speed and yaw preconditions are exemplary and not limiting, and other sets of preconditions may be utilized to similar effect.

[0056] Once the prerequisites are met, the method 300 analyzes the raw image (FIG. 4) using edge detection to detect line segments 510 at the inside edges of lane lines 520 in a "Detect Inside Edge" step 310. The inside edge of a lane line 520 is the edge of the lane line 520 that is closest to the inside of the lane defined by the lane line. In alternative examples, another form of object detection may be used to identify the location of the lane line 520, and the inside edge 510 may be detected accordingly.

[0057] Once a line segment corresponding to the inside edge 510 of the lane line 520 of an adjacent lane is detected, the inside edge line 510 is extrapolated by the controller to the point in the image plane where they intersect in a "detect vanishing point" step 320. The vanishing point 530 is a single point in the image plane where two lane lines 520 that are parallel (or nearly parallel) in the real world intersect. In the example image of FIG. 5, the vanishing point 530 is located within the image frame itself. In another illustration, shown in FIG. 6, the extrapolation of the inside edge 510 of the lane line 520 extends outside the image frame to the vanishing point 530. The controller 30 can track and calculate whether the vanishing point location 530 is within the image frame (FIG. 5) or outside the image frame (FIG. 6), and the method 300 described herein works similarly in either example.

[0058] Once the vanishing point 320 is detected, the position of the reference vanishing point 306 is received from storage memory within CMS 15 and compared to the positions in the image plane of the detected vanishing point 530 and the reference vanishing point 306 in a "Is the detected vanishing point close to the reference?" step 330. The reference vanishing point 306 is, in one example, an expected vanishing point based on the most recently calibrated camera height. In an alternative example, the reference vanishing point is the aggregate (e.g., average) vanishing point position of the last several determinations.

[0059] Alternatively, the reference vanishing point 306 may be stored in a location remote from the controller 30 and retrieved as needed. The check performed in step 330 determines whether the vanishing point is within a predetermined distance from the reference vanishing point and acts as a "sanity check." If the detected vanishing point 530 is a predetermined number of pixels away from the reference vanishing point 306, and that number of pixels is greater than a threshold, the controller 30 determines that the detected vanishing point is inaccurate, and the calibration of the current camera height is stopped in a "skip this scan" step 332.

[0060] If the determined vanishing point 530 is sufficiently close to the reference vanishing point, the CMS system identifies the determined vanishing point 530 as "reasonable" and provides the vanishing point to one or more CMS systems as a reference position (alternatively referred to as a vanishing point reference position) in a "Provide VP to System" step 340.

[0061] One system configured to receive and utilize the determined reference position is a real-time camera height estimator 700. The process 700 for estimating camera height first defines a triangle 540 using the inner edge line 510 and vanishing point 530 (the determined reference position) in a "Calculate Area of Edge Line Triangle" step 340. The triangle 540 is defined within the image plane of the raw image 304 being analyzed and is determined with pixels as the unit of measurement. In alternative examples, other units of measurement can be used to the same effect. It will be understood that the area of the triangle 540 correlates to the camera height relative to the ground. The area of the triangle is determined using any conventional image analysis or via geometric calculation according to known processes. FIG. 7 shows the example image of FIG. 5 with the determined triangle 540 defined within the image added.

[0062] Once the area of triangle 540 has been determined by the controller, method 300 proceeds to use the area of triangle 540 to determine the actual height of the camera generating raw image 304 in a "Determine Camera Height" step 350. In one example, the camera height is determined by comparing the area of triangle 540 to a lookup table 800 (shown in FIG. 9). The lookup table includes a series of ranges 802 of triangle area 802, with each range 802 corresponding to a single camera height 804. In an alternative example, an equation may be utilized to determine the estimated height, which relates the triangle area to the camera height based on an internal test.

[0063] The correlation between the area of triangle 540 and actual height 804 is determined through testing for a particular vehicle configuration in a laboratory, real-world, or a combination of both, and a significant number of test results are used to validate the correlation. Using a range of triangle areas for each camera height allows the system to accommodate minor variations that may occur due to imprecise lane spacing, minor inaccuracies in edge detection, and similar variations that may occur due to the natural conditions of real-world road systems.

[0064] After determining the camera height, the CMS stores the new camera height and provides the new camera height to any active systems that utilize the camera height or to systems where a change in camera height may affect their operation.

[0065] The real-time camera height determination may be repeated each time the preconditions are met, once per engine cycle, each time the preconditions are met again, or at any other frequency that ensures that continuously up-to-date camera height information is provided to the controller 30.

[0066] 3 is provided to an image registration process 900 (shown in FIG. 9), where the reference positions are utilized to register images displayed to the vehicle operator on displays 18a, 18b. Initially, raw images 902, 904 from each side of the vehicle and the corresponding reference positions (determined via method 300) are provided to the registration system.

[0067] The alignment system generates an initial alignment of the images by aligning the top edges of each image 902, 904 in an "Align Raw Images" step 910. If the cameras are not at the same height above the ground, the images will appear misaligned and the image's reference position will be offset by a vertical amount 532. To determine if this is occurring, process 900 compares the vertical positions of the two reference positions in a Compare Reference Positions step 920.

[0068] If the reference location 530 is offset by a vertical distance 532, the displayed image may appear misaligned and may provide a less representative view of the actual mirror. To compensate and improve the view, in a "crop and resize" step 930, the image is cropped and resized so that the reference location 530 is at the same vertical height 534.

[0069] In a further alternative, the registration method 900 and the height estimation method 700 are performed simultaneously on the same set of images using the same set of reference locations. In other examples, either or both of the methods 700, 900 are performed in conjunction with one or more additional systems that utilize reference locations.

[0070] Also, while particular component arrangements are disclosed in the illustrated embodiments, it should be understood that other arrangements would benefit from the present disclosure. Although a particular sequence of steps is shown, described, and claimed, it should be understood that, unless otherwise indicated, the steps may be performed in any order, separated, or combined, and still benefit from the present invention.

[0071] Although the different examples have specific components shown, embodiments of the present invention are not limited to those specific combinations, and some components or features of one example may be used in combination with features or components of another example.

[0072] While exemplary embodiments have been disclosed, those of ordinary skill in this art would recognize that certain modifications would come within the scope of the following claims, and for that reason the following claims should be studied to determine their true scope and content.

Claims

1. 1. A method for determining an image reference position, comprising: receiving at least one image from a camera at a controller, said image including a road lane defined by two identified lane lines, said camera being a component of a CMS (camera surveillance system) for a vehicle; using image-based analysis to identify an inside edge of each of the two identified lane lines; using the controller to identify a vanishing point of the two identified lane lines by extending the identified inner edge of each identified lane line to a point where the extended identified inner edges intersect; providing said vanishing points as said image reference locations to at least one CMS system; A method comprising:

2. The method of claim 1 , wherein the at least one camera is at least one rear-facing wing-mounted camera.

3. The method of claim 1 , wherein the identified inside edge of each identified lane line is the inward-facing edge of a lane bounded by the two identified lane lines.

4. 2. The method of claim 1, wherein the CMS system includes a real-time camera height determination system that determines the height of the camera relative to the ground by determining an area of a triangle defined by the vanishing point and an edge of each identified lane line, and converting the area of the triangle to a corresponding camera height using the controller.

5. The method of claim 4 , wherein converting the area of the triangle to a corresponding camera height using the controller comprises identifying an entry in a lookup table that corresponds to the area of the triangle.

6. the lookup table includes a range of areas for a series of triangles; The method of claim 5 , wherein each range of triangle areas in the set of triangle area ranges is correlated with a corresponding camera height.

7. 5. The method of claim 4, wherein converting the triangle area to a corresponding camera height using the controller comprises inputting the determined triangle area into an equation and determining the camera height as an output of the equation.

8. 2. The method of claim 1, further comprising: comparing the identified vanishing point with a reference vanishing point; and determining that the identified vanishing point is accurate in response to the identified vanishing point being within a threshold distance of the reference vanishing point.

9. The method of claim 1 , further comprising verifying that preconditions are met before using the controller to identify the vanishing points of the two distinguishable lane lines.

10. 10. The method of claim 9, wherein the precondition is that the speed of the vehicle exceeds a first threshold and the yaw of the vehicle is below a second threshold.

11. 11. The method of claim 10, wherein the first threshold is at least 40 kilometers per hour and the second threshold is at least 1 degree per second.

12. receiving at the controller the at least one image from the at least one camera includes receiving a first image from a first camera and a second image from a second camera, wherein a first reference point is identified for the first image and a second reference point is identified for the second image; 2. The method of claim 1, wherein the CMS includes a display alignment system configured to align a first representation of a first image and a second representation of a second image by positioning the reference point of the first image and the reference point of the second image at the same vertical height on corresponding display screens.

13. Positioning the reference point of the first image and the reference point of the second image at the same vertical height of the corresponding display screens includes: aligning a raw image with one of a top edge of the image and a bottom edge of the image; determining the vertical height difference between the corresponding reference points; and adjusting at least one of the first image and the second image so that the vertical height difference between the corresponding reference points is zero; 13. The method of claim 12, comprising:

14. 14. The method of claim 13, wherein adjusting at least one of the first image and the second image includes cropping at least one of the first image and the second image and resizing at least one of the first image and the second image.

15. A CMS (camera monitoring system) for a vehicle, first and second rear-facing cameras; a controller including at least a processor and a memory; wherein the memory comprises: identifying an inner edge of each of the two identified lane lines using image-based analysis; using the controller to identify a vanishing point of the two identified lane lines by extending the identified inner edge of each lane line to a point where the extended inner edges intersect; and providing said vanishing points as image reference locations to at least one CMS system; determining a camera height of at least one of the first and second rear-facing cameras in real time by:

16. the at least one CMS system includes a real-time height determination system; 16. The CMS of claim 15, wherein the real-time height determination system determines the area of a triangle defined by the vanishing point and the edge of each distinct lane line and converts the area of the triangle to a corresponding camera height using the controller.

17. 17. The CMS of claim 16, wherein converting the area of the triangle to a corresponding camera height using the controller comprises identifying an entry in a lookup table that corresponds to the area of the triangle.

18. the lookup table includes a range of areas for a series of triangles; The CMS of claim 17 , wherein each range of triangle areas in the set of triangle area ranges is correlated with a corresponding camera height.

19. 17. The CMS of claim 16, wherein converting the triangle area to a corresponding camera height using the controller comprises inputting the determined triangle area into an equation and determining the camera height as an output of the equation.

20. 16. The CMS of claim 15, further comprising: comparing the identified vanishing point to a reference vanishing point; and determining that the identified vanishing point is accurate in response to the identified vanishing point being within a threshold distance of the reference vanishing point.

21. 16. The CMS of claim 15, wherein the at least one CMS system includes a display alignment system configured to align a first representation of the first image and a second representation of the second image by positioning the reference point of a first image and the reference point of a second image at the same vertical height on corresponding display screens.

Images