Camera monitoring system with automatic HMI adjustment for commercial vehicle displays

Abstract

A method for adjusting a position of a human machine interface (HMI) element in a commercial vehicle display includes displaying, on an electronic display of the commercial vehicle, an image from a camera mounted on the commercial vehicle depicting a trailer of the commercial vehicle and its surrounding environment, determining a reference position of a reference point of the trailer in the image, overlaying an HMI element in the image on the electronic display at the determined HMI position relative to the reference position, detecting a change in the reference position in the image and adjusting the HMI position accordingly. Methods for a vehicle camera monitoring system (CMS) and a camera mirror system are also disclosed.

Description

[Technical field]

[0001] The present disclosure relates to camera monitor systems (CMS) for use in commercial vehicles, including trailers, and more particularly to methods for adjusting the position of human machine interface (HMI) elements within a commercial vehicle display.

[0002] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 330,460, filed April 13, 2022, the disclosure of which is incorporated by reference herein in its entirety. [Background technology]

[0003] Mirror replacement systems, and camera systems that supplement the mirror view, are utilized in commercial vehicles to enhance the vehicle operator's ability to view the surrounding environment. Camera surveillance systems (CMS) utilize one or more cameras to provide an enhanced field of view to the vehicle operator. In some instances, mirror replacement systems cover a wider field of view than a traditional mirror or include views that are not fully available via a traditional mirror.

[0004] The CMS uses multiple displays positioned in the vehicle cab to display mirror replacement and / or supplemental views to the vehicle operator. In some cases, the displays include human machine interface (HMI) elements overlaid on top of the image. The HMI elements may provide relative position information, vehicle feature identification, and / or other information to the vehicle operator. The information supplements the view provided by the camera and, in some cases, allows the CMS display to provide greater functionality than traditional mirror systems. The CMS conveys at least some of the supplemental information by positioning the HMI elements at defined positions and / or orientations relative to objects in the image. However, if the camera image is angled or tilted relative to the expected position, the HMI elements may be overlaid in a position different from the desired position and / or in an orientation other than the desired orientation. Summary of the Invention

[0005] A method for adjusting a position of a human-machine interface (HMI) element in a commercial vehicle display according to one embodiment of the present disclosure includes displaying, on an electronic display of the commercial vehicle, an image from a camera mounted on the commercial vehicle, depicting a trailer of the commercial vehicle and its surrounding environment, determining a reference position of a reference point of the trailer in the image, overlaying an HMI element in the image on the electronic display at an HMI position determined relative to the reference position, detecting a change in the reference position in the image and adjusting the HMI position accordingly.

[0006] In a further embodiment of the above embodiment, the HMI element is a distance marker superimposed to extend laterally away from a side of the trailer in the image.

[0007] In a further embodiment of any of the above embodiments, adjusting the HMI position includes adjusting the HMI position to minimize or avoid overlap of the HMI element and the trailer in the image.

[0008] In a further embodiment of any of the above embodiments, the reference point of the trailer is a rear corner of the trailer.

[0009] In a further embodiment of any of the above embodiments, determining a reference position of the reference point of the trailer is based on a first trailer angle, the first trailer angle being an angle between a longitudinal axis of the trailer and a longitudinal axis of a tractor connected to the trailer, and a first arc representing a predetermined range of possible positions of the reference point within the image.

[0010] In a further embodiment of any of the above embodiments, the reference point of the trailer is a first reference point, and determining the reference position of the first reference point includes determining a trailer image angle, the trailer image angle being an angle in one of the images defined relative to a trailer reference line in the image based on the trailer angle, the trailer reference line extending from a second reference point of the commercial vehicle to a third reference point of the trailer, the second reference point being closer to a front of the trailer than the third reference point, and the third reference point being closer to the first reference point than the second reference point. Determining the reference position of the first reference point includes determining the reference position of the first reference point based on the trailer image angle, an intersection of the trailer reference line with a second arc representing a predetermined range of possible positions of the third reference point in the image, and a distance between the first arc and the second arc in the image.

[0011] In a further embodiment of any of the above embodiments, the third reference point is located at a rear wheel of a trailer, the first reference point is located proximate a front lower corner of the trailer, and the distance between the first arc and the second arc in the image is the vertical distance between the first arc and the second arc.

[0012] In a further embodiment of any of the above embodiments, the camera is attached to a camera arm configured to rotate from a retracted position to an extended position at a predetermined rotational speed during normal operation, and detecting includes measuring a rotational speed of the camera arm when extending from a first position intended to be the retracted position to a second position intended to be the extended position, and determining that the reference position has changed within the image based on the measured rotational speed differing from the predetermined rotational speed.

[0013] In a further embodiment of any of the above embodiments, the camera is attached to a camera arm configured to rotate from a stored position to an extended position at a predetermined angular velocity during normal operation, and wherein the camera arm is configured to take a predetermined duration to extend from the stored position to the extended position during normal operation, and the detecting includes measuring a duration that the camera arm takes to extend from a first position, intended to be the stored position, to a second position, intended to be the extended position, and determining that the reference position has changed within the image based on the measured duration differing from the predetermined duration.

[0014] A camera surveillance system (CMS) for a vehicle according to an exemplary embodiment of the present disclosure includes a camera configured to record an image of a trailer of a commercial vehicle and its surrounding environment, an electronic display, and a CMS controller configured to display the image on the electronic display, determine a reference position of a reference point of the trailer in the image, overlay a human machine interface (HMI) element in the image on the electronic display at an HMI position determined relative to the reference position, detect when the reference position changes in the image and adjust the HMI position of the HMI element accordingly.

[0015] In a further embodiment of the above embodiment, the HMI element is a distance marker superimposed to extend laterally away from a side of the trailer in the image.

[0016] In a further embodiment of any of the above embodiments, to adjust the HMI position of the HMI element on the electronic display, the CMS controller is configured to adjust the HMI position to minimize or avoid overlap of the HMI element with the trailer in the image.

[0017] In a further embodiment of any of the above embodiments, the reference point of the trailer is a rear corner of the trailer.

[0018] In a further embodiment of any of the above embodiments, the CMS controller is configured to determine the reference position of the reference point based on a first trailer angle, the first trailer angle being an angle between a longitudinal axis of the trailer and a longitudinal axis of a tractor connected to the trailer, and a first arc representing a predetermined range of possible positions of the reference point in the image.

[0019] In a further embodiment of any of the above embodiments, the reference point of the trailer is a first reference point, and to determine the reference position of the first reference point, the CMS controller is configured to determine a trailer image angle based on the trailer angle, the trailer reference line extending from a second reference point of the commercial vehicle to a third reference point of the trailer, the second reference point being closer to the front of the trailer than the third reference point, and the third reference point being closer to the first reference point than the second reference point. The CMS controller is also configured to determine the reference position of the first reference point based on the trailer image angle, an intersection of the trailer reference line with a second arc representing a predetermined range of possible positions of the third reference point in the image, and a distance between the first arc and the second arc in the image.

[0020] In a further embodiment of any of the above embodiments, the third reference point is located at a rear wheel of a trailer, the first reference point is located near a front lower corner of the trailer, and the distance between the first arc and the second arc in the image is the vertical distance between the first arc and the second arc.

[0021] In a further embodiment of any of the above embodiments, the camera is attached to a camera arm configured to rotate from a retracted position to an extended position at a predetermined rotational speed during normal operation, and to detect that the reference position has changed in the image, the CMS controller is configured to measure a rotational speed of the camera arm when extended from a first position, intended to be the retracted position, to a second position, intended to be the extended position, and determine that the reference position has changed in the image based on the measured rotational speed differing from the predetermined rotational speed.

[0022] In a further embodiment of any of the above embodiments, the camera is attached to a camera arm configured to rotate during normal operation at a predetermined angular velocity from a retracted position to an extended position, and configured such that during normal operation the camera arm takes a predetermined duration to extend from the retracted position to the extended position. To detect that the reference position has changed in the image, the CMS controller is configured to measure the duration it takes the camera arm to extend from a first position, intended to be the retracted position, to a second position, intended to be the extended position, and determine that the reference position has changed in the image based on the measured duration differing from the predetermined duration.

[0023] A method for a camera mirror system according to an exemplary embodiment of the present disclosure includes initiating an extension process on a camera arm during normal operation configured to rotate the camera arm from a stowed position to an extended position at a predetermined rotational speed such that the camera arm takes a predetermined duration to extend from a stowed position to an extended position during normal operation, measuring the rotational speed of the camera arm during the extension process or the duration it takes the camera arm to complete the extension process, and based on the rotational speed or duration differing from an expected value by more than a predetermined threshold, determining that the camera arm has not reached the extended position and adjusting a position of a human machine interface (HMI) element in a vehicle display.

[0024] In a further embodiment of the above embodiment, measuring includes measuring both the rotational speed of the camera arm during a camera arm extension process and the duration it takes the camera arm to complete the extension process, and determining that the camera arm has not reached the extended position is based on either the measured rotational speed differing from the predetermined expected rotational speed threshold by more than the predetermined rotational speed threshold or the measured duration differing from the predetermined expected time threshold by more than the predetermined time threshold.

[0025] The embodiments, examples and alternatives of the preceding paragraphs, the claims, or the following description and drawings, including any of their various aspects or their respective individual features, may be taken independently or in any combination. Features described in relation to one embodiment are applicable to all embodiments, except where such features are incompatible. [Brief description of the drawings]

[0026] The present disclosure can be further understood by reference to the following detailed description taken in conjunction with the accompanying drawings.

[0027] [Figure 1A]FIG. 1 is a schematic front view of a commercial truck equipped with a camera mirror system (CMS) used to provide at least Class II and Class IV views.

[0028] [Figure 1B] FIG. 2 is a schematic top view of the commercial truck of FIG. 1 with a trailer angle of 0.

[0029] [Figure 1C] FIG. 2 is a schematic top view of the commercial truck of FIG. 1 with a trailer angle other than 0.

[0030] [Figure 1D] FIG. 2 is a schematic top view of the commercial truck of FIG. 1 equipped with a CMS providing Class II, Class IV, Class V, and Class VI views.

[0031] [Diagram 2] FIG. 1 is a schematic top perspective view of a vehicle cab including a display and an in-cabin camera.

[0032] [Figure 3A] FIG. 1 is a schematic diagram of a simplified camera arm in a stowed position.

[0033] [Figure 3B] FIG. 3B is a schematic diagram of the simplified camera arm of FIG. 3A in an extended position.

[0034] [Figure 4] FIG. 1 is a schematic diagram of an example CMS display including multiple human machine interface (HMI) elements.

[0035] [Diagram 5] FIG. 2 is a schematic diagram of an exemplary method for adjusting an HMI element.

[0036] [Figure 6] 6 is a schematic diagram of an example implementation of steps from the method of FIG. 5.

[0037] [Figure 7] 1 is a schematic diagram of an exemplary method for determining HMI adjustments.

[0038] [Figure 8] 1 is a schematic diagram of an exemplary method for determining that an HMI adjustment is required. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] Schematic diagrams of a commercial vehicle 10 are shown in Figures 1A, 1B, 1C, and 1D. Vehicle 10 includes a vehicle cab or "tractor" 12 for towing a trailer 14. Although vehicle 10 is shown in this disclosure as a commercial truck, it should be understood that other types of vehicles may be used and other configurations for vehicle cab 12 and / or trailer 14 may be used.

[0040] As shown in Figures 1B-1C, hitch 11 attaches trailer 14 to tractor 12 and allows trailer 14 to pivot relative to tractor 12 during a turn. Tractor 12 has a central longitudinal axis L1 and trailer 14 has a central longitudinal axis L2. As shown in Figure 1B, when tractor 12 is not turning, axes L1, L2 are parallel or coaxial and there is no angle between axes L1, L2. As shown in Figure 1C, when tractor 12 is turning, there is an angle θ between axes L1, L2. T The angle between axes L1, L2, which is approximately 20° in FIG. 1C, is referred to herein as the "trailer angle."

[0041] 2 with continued reference to FIGS. 1A and 1D, a camera mirror system (CMS) 15 includes driver and passenger side camera arms 16a, 16b mounted on the exterior of the vehicle cab 12 (FIG. 1A). If desired, the camera arms 16a, 16b may also include conventional mirrors integrated therewith, although the CMS 15 may be used to replace the mirrors entirely. In additional examples, multiple camera arms may be included on each side, with each arm housing one or more cameras and / or mirrors. As shown in FIG. 2, the CMS 15 includes a CMS controller 13 that includes processing circuitry to support the operation of the CMS 15 and is operably connected to memory. The processing circuitry may include one or more microprocessors, microcontrollers, application specific integrated circuits (ASICs), and the like.

[0042] 2, first and second electronic displays 18a, 18b, which may be video displays, are positioned within the vehicle cab 12 on or near the A-pillars 19a, 19b on the driver's and passenger's sides, respectively, and display Class II and Class IV views on each side of the vehicle 10. These provide rearward facing side views along the vehicle 10 that are captured by external cameras 20a, 20b.

[0043] If video of Class V and Class VI views is also required, a camera housing 16c and camera 20c may be located at or near the front of the vehicle 10 to provide these views (FIG. 1B). A third display 18c located within the cab 12 near the top center of the windshield can be used to display the Class V and Class VI views forward of the vehicle 10 to the driver.

[0044] If Class VIII view video is required, camera housings can be positioned on the sides and rear of the vehicle 10 to provide a field of view that includes some or all of the Class VIII zone of the vehicle 10. In such an example, the third display 18c can include one or more frames that display the Class VIII view. Alternatively, additional displays can be added near the first, second and third displays 18a, 18b, 18c to provide a dedicated display that provides the Class VIII view. The displays 18a, 18b, 18c face a driver area 34 within the cab 32 where the driver is seated in the driver's seat 36.

[0045] With continued reference to Figures 1A, 1D and 2, Figure 3A illustrates an example of a camera arm 16a in a stored position, and Figure 3B illustrates an example of a camera arm 16A in an extended position. Each camera arm 16A, 16B includes at least one rear-facing camera 20 and is mounted to a respective base 24A, 24B that is attached to the tractor 12. Each camera 20 is connected to a respective base 24 via a respective linkage 28A-28B (e.g., a ball joint), and the camera arms 16A-16B are configured to rotate about their respective linkages 28A-28B between a respective stored position (shown for camera arm 16A in Figure 3A) and an extended position (shown for camera arm 16A in Figure 3B). In particular, each camera arm 16 is configured to extend from a stored position to an extended position via an extension process, and to retract from an extended position to a stored position via a retraction process.

[0046] During normal operation, each camera arm 16 is configured to rotate at a predetermined rotational speed during its extension process, such that it takes a predetermined duration for the camera arm 16 to complete the extension process and extend from the stowed position to the extended position. Similarly, during normal operation, the camera arms 16 are configured to rotate at the same or different predetermined rotational speeds during their retraction process, such that it takes the same or different predetermined duration for the camera arm to complete the retraction process and retract from the extended position to the stowed position.

[0047] Each camera arm includes a respective angular motion sensor 26A-26B configured to measure the angular rate of rotation of the camera arm 16 between the retracted and extended positions. In one example, the angular motion sensor 22 is an inertial measurement unit (IMU). In another example, the angular motion sensor 22 is an accelerometer. In yet another example, the angular motion sensor 222 may be any sensor or combination of sensors capable of detecting relative motion, including angular motion, of the sensing units. The CMS controller 13 is configured to utilize the angular motion sensors to measure the rate of rotation of the camera arm 16 during the extension and / or retraction process. For purposes of illustration, the camera arms 16A-16B and linkages 26A-26B have been simplified, it being understood that more complex support structures, including articulating arms, powered articulating joints, and similar devices, may be utilized to achieve the positioning of the cameras 20a, 20b described herein.

[0048] Continuing with reference to the CMS 15 described in Figures 1-3, Figure 4 illustrates an exemplary Class II image 204 from camera 20a displayed to a vehicle operator (e.g., on electronic display 18a). Image 204 includes a trailer 210, trailer wheels 220, and a view of road 230 and the surrounding environment. Overlaid on top of image 204 are several Human Machine Interface (HMI) elements including distance lines 142 (projection of trailer end onto ground), 144 (30m from distance line 142), and 146 (100m from distance line 142). Also shown are wheel centers 150A-150C, wheel center 150, rear lower trailer corner 160A, rear upper trailer corner 160B, and rear edge 162 of wheel 220. In alternative embodiments, alternative HMI elements such as trailer end top indicators, reverse track, etc. may be included in addition to or in place of some or all of the HMI elements shown in FIG. 4, and the illustrated HMI elements and configurations are purely exemplary in nature.

[0049] Each HMI element is displayed at a particular location within image 204 that is determined relative to a reference location within image 204, such as rear lower trailer corner 160A or rear upper trailer corner 106B.

[0050] The reference position within image 204 may change for a variety of reasons, such as the vehicle turning (forward or reverse), vehicle loading (which causes the trailer to compress the tires more and therefore closer to the ground than they would be in an unloaded state), changes in road gradient (which causes camera 20 to be angled either up or down relative to the horizontal ground), collision of an external object (e.g., a tree branch) with camera arms 16A-16B, and similar events.

[0051] For example, if the camera 20 is angled downward with respect to gravity and has no orientation shift, objects in the image will appear to be shifted upward and higher in the field of view, and vice versa if the camera 20 is angled upward. Similarly, if the camera 20 is rotated towards the vehicle 10, skew may occur such that objects near the sides of the vehicle 10 appear larger than objects farther away from the sides of the vehicle 10.

[0052] If the reference position (e.g., of corners 160A and / or 160B) is shifted (e.g., due to one of the events described above), this can cause a change in the image 204 presented to the operator on the corresponding display (e.g., displays 18A and / or 18B), resulting in inaccurate positioning of the HMI elements 142, 144, 146. For example, if a static position is maintained for the HMI elements 142, 144, 146, the elements may be superimposed on the trailer 14 during a turn. If cameras 20a, 20b are used to generate Class II / Class IV images, the above modifications may be applied to both Class II and Class IV fields of view.

[0053] 4 convey information to the operator about the vehicle 10 and / or the surrounding environment based, at least in part, on the relative position of the HMI element and the portion of the vehicle 10 that is in view. As an example, distance lines 142, 144, 146 represent distances from rear edge 162 of trailer 110 and the positions of lines 142, 144, 146 relative to the end of trailer 14. Additionally, the direction of the lines extending away from the end of trailer 14 provides an indication as to which side in the image is at the distance of distance lines 142, 144, 146.

[0054] Movement of the reference position in the image relative to camera 20 (e.g., trailer corner 160A), whether due to movement of trailer 14 or due to unintended movement of camera 20 (e.g., due to camera arm 16 not reaching the expected extended position due to contact with a tree branch and instead reaching another extended position), will cause the HMI element to be placed in an incorrect position in the CMS image without adjustment of the HMI element position. As described in more detail below, CMS controller 13, utilizing software in CMS 15, is configured to detect when the reference position in the image has changed and adjust the HMI position accordingly, so that the HMI element in the image is moved to account for the moved position and allow for continued accuracy of the information being conveyed.

[0055] 5 is a flow chart of an exemplary method 400 for adjusting an HMI element position within a commercial vehicle electronic display 18. An image depicting the trailer 14 of the commercial vehicle 10 and its surrounding environment is acquired from the camera 20 (step 402). The image is displayed on the electronic display 18 of the commercial vehicle 10 (step 404). A reference position of a first reference point of the trailer 14 within the image is determined (e.g., by the CMS controller 13) (step 406). The HMI elements 142, 144, 146 are superimposed within the image on the electronic display 18 at the determined HMI positions relative to the reference position (step 408).

[0056] A determination is made (e.g., by CMS controller 13) whether the reference position has changed in the image (step 410). If the reference position has changed (step 410, "yes"), the HMI position is adjusted accordingly (step 412). Otherwise, if the reference position has not changed in the image (step 410, "no"), the HMI position is maintained (step 414). In one example, the change must exceed a predefined threshold (e.g., a minimum pixel change) to trigger the adjustment in step 412. As described in more detail below, the determination of whether the reference position has changed in step 410 may be based on image analysis and / or based on movement of camera arm 16. Method 400 may also be performed for multiple HMI elements, each having a respective HMI position determined relative to a reference position.

[0057] As mentioned above, the HMI element may be distance markers 142, 144 and / or 146 superimposed to extend laterally away from a side of the trailer 14 in the image (represented by rear edge 162), and the reference point of the trailer 14 may be, for example, rear corner 160A or 160B. In one example, adjusting the HMI position in step 412 includes adjusting the HMI position horizontally and / or vertically to minimize or avoid overlap of the HMI element(s) and trailer 14 in the image, and instead overlap the HMI element(s) with a road or similar portion of the image.

[0058] FIG. 6 illustrates an example implementation 450 of step 406 from FIG. 4 for determining a reference position of a reference point of a trailer 14 in a CMS image.

[0059] FIG. 7 shows two annotated CMS images 206A-206B used to explain an example implementation 450 of step 406. Images 206A-206B are presented adjacent to one another in FIG. 7 for illustrative purposes (even though in practice they would likely be provided to separate CMS displays 18A, 18B). Image 206A represents a CMS image from camera 20A and image 206B represents a CMS image from camera 20B. Point 612 represents a calibration point on the commercial vehicle (e.g., the lower front corner of trailer 14) and has coordinates (x t , y t Point 616, having coordinates (x i , y i ) represents a third reference point of the commercial vehicle 10 (e.g., wheel center 150A, 150B, or 150C in FIG. 4).

[0060] Arc 650 represents the trajectory / range of possible values ​​of a third reference point in the image (e.g., the range of one possible value of wheel center 150), and arc 640 represents the trajectory range of possible values ​​of a first reference point in the CMS image (e.g., the range of possible values ​​of lower rear trailer corner 160A). The distance labeled ΔY represents the vertical distance between arcs 640, 650. Trailer reference line 614 extends through first reference point 616, second reference point 612, and third reference point 618 and indicates the orientation of trailer 14 relative to tractor 12. Trailer image angle θ T1 represents the angle defined relative to the trailer line 614 (in the example of FIG. 7, defined relative to the image edge 620).

[0061] 6, in step 452, the location of the trailer calibration point (e.g., second reference point 612) is determined. This step may be performed using, for example, conventional image analysis. The trailer calibration point is the trailer position relative to which the trailer reference line 614 is plotted. In one example, the calibration point 612 remains generally constant throughout operation of the vehicle and may be detected using trailer markings and / or marking independent image analysis.

[0062] Trailer angle θ T is determined between the tractor longitudinal axis L1 and the trailer longitudinal axis L2 (see FIG. 1C) (step 454). T may be a detected trailer angle from a trailer angle sensor (not shown), a derived trailer angle determined from image analysis, composite data from multiple vehicle sensors, or any combination thereof. In the context of implementation 450, the trailer angle θ T represents the three-dimensional angle, and the trailer image angle θ T1 represents a two-dimensional angle.

[0063] Trailer image angle θ T is the trailer angle θ T (step 456). A third reference point 618 corresponds to the intersection of the trailer reference line 614 and the arc 650 is determined (step 458). A first reference position of the first reference point 616 is determined based on the intersection of the trailer reference line 614 and the arc 650 and the distance ΔY (pixel deviation) between the arcs 640, 650 (step 460).

[0064] Therefore, step 406 is performed by determining at least the trailer angle θ T , and an arc 640 representing a predetermined range of possible locations of the first reference point within the image.

[0065] Step 406 is executed based on the trailer angle θ T Based on the trailer image angle θ T1 and determining a reference position of the first reference point based on the intersection of the trailer reference line 614 and the arc 650 and the distance between the arcs 640, 650.

[0066] The process described and illustrated in Figures 6 and 7 may be repeated to dynamically maintain the proper position of the trailer end line. Additionally, the process of Figures 6 and 7 may be applied to other HMI elements and is not limited to trailer end lines.

[0067] 8 illustrates in schematic form an exemplary method 800 for determining that an HMI position adjustment is required. As described above, the extension process of the camera arm 16 extends the camera arm from a retracted position (see, e.g., FIG. 3A) to an extended position (see, e.g., FIG. 3B). During normal operation, the extension is performed at a predetermined rotational speed and takes a predetermined duration to complete. If the extension process is performed at an irregular speed or lasts for an irregular time, it may be interpreted as an indication that the camera arm 16 is obstructed by an external object (e.g., a tree branch) or that the camera arm 16 is damaged (e.g., from contact with an external object) such that the camera 20 of the camera arm 16 is not properly positioned (e.g., not properly positioned and oriented).

[0068] 8, the camera arm extension process is initiated (step 702). The rotational speed of the camera arm 16 during the extension process is measured using the angular motion sensor 26 of the camera arm 16, as well as the duration it takes the camera arm 16 to complete the extension process (step 704). The measured rotational speed is compared to a predetermined expected rotational speed, and it is determined whether the difference between the measured rotational speed and the expected rotational speed exceeds a speed threshold (step 706).

[0069] If the difference exceeds the threshold value (step 706: "YES"), it is determined that the camera arm 16 has not reached the extended position, and the position of the HMI element within the vehicle display 18 is adjusted (step 708). Step 708 also represents a determination that the initial reference point of step 406 of method 400 has changed. If the difference does not exceed the threshold value (step 706: "NO"), the method proceeds to step 710.

[0070] At step 710, the measured duration is compared to the predicted predetermined duration to determine if the difference between the measured duration and the predicted duration exceeds the duration threshold (step 710). If the difference exceeds the duration threshold (step 710, "yes"), the method proceeds to step 708. Otherwise, if the difference does not exceed the duration threshold (step 710, "no"), it is determined that the camera arm 16 has properly reached the extended position and that the camera 20 is properly positioned and oriented accordingly.

[0071] Although measuring and comparing both rotation rate and duration are described in steps 704, 706 and 710, it is understood that the measurement and comparison of rotation rate may be omitted such that the determination in step 708 is based on duration rather than rotation rate, or the measurement and comparison of duration may be omitted such that the determination in step 708 is based on rotation rate rather than duration.

[0072] The above features describe how HMI elements can be automatically positioned and relocated based on various factors that cause relative movement of the trailer reference point in the CMS image (e.g., vehicle turns, vehicle positioning, changes in road grade, and / or irregular camera arm extension).

[0073] Although exemplary embodiments have been disclosed, a person of ordinary skill in this art would recognize that certain modifications would come within the scope of the following claims, and for that reason the following claims should be studied to determine their true scope and content.

Claims

1. A method for adjusting the position of HMI (Human-Machine Interface) elements in a commercial vehicle display, In an electronic display of a commercial vehicle, the display shows images from a camera mounted on the commercial vehicle, depicting the trailer of the commercial vehicle and its surrounding environment. To determine the reference position of the reference point of the trailer in the aforementioned image, The HMI elements are superimposed within the image on the electronic display at the HMI position determined relative to the aforementioned reference position, The system detects when the reference position changes within the aforementioned image and adjusts the HMI position accordingly. Includes, Determining the reference position of the trailer's reference point is: The first trailer angle is the angle between the longitudinal axis of the trailer and the longitudinal axis of the tractor connected to the trailer, and A first arc representing a predetermined range of possible positions of the reference point in the aforementioned image. It is based on, The reference point of the trailer is a first reference point, and determining the reference position of the first reference point is Based on the trailer angle, the trailer image angle is determined to be the angle in one of the images defined with respect to the trailer reference line in the image, wherein the trailer reference line extends from a second reference point of the commercial vehicle toward a third reference point of the trailer, the second reference point is closer to the front of the trailer than the third reference point, and the third reference point is closer to the first reference point than the second reference point, and The reference position of the first reference point is determined based on the trailer image angle, the intersection of the trailer reference line and a second arc representing a predetermined range of possible positions of the third reference point in the image, and the distance between the first arc and the second arc in the image. Methods that include...

2. The method according to claim 1, wherein the HMI element is a distance marker superimposed so as to extend laterally away from the side of the trailer in the image.

3. The method according to claim 2, wherein adjusting the HMI position includes adjusting the HMI position to minimize or avoid overlap between the HMI element and the trailer in the image.

4. The method according to claim 1, wherein the first reference point of the trailer is the rear corner of the trailer.

5. The third reference point is located at the rear wheel of the trailer, The first reference point is located close to the front lower corner of the trailer, The method according to claim 1, wherein the distance between the first arc and the second arc in the image is the perpendicular distance between the first arc and the second arc.

6. A camera configured to record images of a commercial vehicle trailer and its surrounding environment, Electronic displays and CMS controller and The CMS controller is equipped with, Displaying the image on the aforementioned electronic display, To determine the reference position of the reference point of the trailer in the aforementioned image, At the HMI position determined relative to the aforementioned reference position, the HMI (Human-Machine Interface) elements are superimposed within the image on the electronic display. The system detects when the reference position changes within the image and adjusts the HMI position of the HMI element accordingly. It is configured to do the following: The CMS controller is The first trailer angle is the angle between the longitudinal axis of the trailer and the longitudinal axis of the tractor connected to the trailer, and A first arc representing a predetermined range of possible positions of the reference point in the aforementioned image. The system is configured to determine the reference position of the reference point based on the following: The reference point of the trailer is a first reference point, and in order to determine the reference position of the first reference point, the CMS controller, Based on the trailer angle, the trailer image angle is determined to be the angle in one of the images defined with respect to the trailer reference line in the image, wherein the trailer reference line extends from a second reference point of the commercial vehicle toward a third reference point of the trailer, the second reference point is closer to the front of the trailer than the third reference point, and the third reference point is closer to the first reference point than the second reference point, and The reference position of the first reference point is determined based on the trailer image angle, the intersection of the trailer reference line and a second arc representing a predetermined range of possible positions of the third reference point in the image, and the distance between the first arc and the second arc in the image. A camera surveillance system (CMS) for vehicles, configured to perform the following actions.

7. The camera surveillance system according to claim 6, wherein the HMI element is a distance marker superimposed so as to extend laterally away from the side of the trailer in the image.

8. The camera surveillance system according to claim 7, wherein the CMS controller is configured to adjust the HMI position of the HMI element in the electronic display so as to minimize or avoid overlap between the HMI element and the trailer in the image.

9. The camera monitoring system according to claim 6, wherein the first reference point of the trailer is the rear corner of the trailer.

10. The third reference point is located at the rear wheel of the trailer, The first reference point is located close to the front lower corner of the trailer, The camera monitoring system according to claim 6, wherein the distance between the first arc and the second arc in the image is the vertical distance between the first arc and the second arc.

Images