Camera monitoring system with trailer impact alert on the curb and trailer impact area.

Abstract

A camera monitor system (CMS) for a vehicle includes a camera that is configured to face rear and provide a captured image of a field of view that includes at least a portion of a trailer. A display is in communication with the camera and configured to depict a displayed image that includes at least a portion of the captured image that includes the portion of the trailer. A controller is in communication with the camera and the display. The controller includes a collision alert module that is configured to provide an overlay on the displayed image that corresponds to a region that encompasses a predicted trailer path relative to a current trailer position. The overlay includes a first boundary that is provided by curved line indicative of an inside trailer path and a striking area from the first boundary to the trailer.

Description

1 / 27 Camera monitoring system with trailer impact alert on the curb and impact area. TOWED PRIORITY CLAIM

[001] The present application claims priority over the Application of United States Patent Serial No. 18 / 116,627, filed March 2, 2023, entitled TRAILER STRIKING PREDICTION USING CAMERA MONITORING SYSTEM. FIELD OF TECHNIQUE

[002] This disclosure relates to a camera monitoring system (CMS) for use on a tractor pulling a trailer and, in particular, to a system for increasing driver awareness during a turning operation. BACKGROUND

[003] Mirror replacement systems and camera systems to supplement mirror vision are used in commercial vehicles to increase the operator's ability to view the surrounding environment. Camera monitoring systems (CMS) use one or more cameras arranged around the vehicle to provide an enhanced field of view to the operator on one or more displays located in the vehicle cab. In some examples, mirror replacement systems within the CMS may cover a wider field of view than a conventional mirror or may include views that are not fully obtained through a conventional mirror.

[004] Forward turning maneuvers in commercial tractor configurations require a wider turn than other vehicles in order to prevent the side of the trailer from inadvertently striking objects on the inside of the turning arc. Even when the inside of the turn is visible through mirrors and / or systems of Petition 870250077905, dated 01 / 09 / 2025, page 9 / 68 2 / 27 camera monitoring, it can be difficult for less experienced operators to assess trailer side movement using only conventional views.

[005] Typically, vehicle operators compensate for the difficulty by making unnecessarily wide turns to ensure that objects on the inside of the turn are not hit by the trailer. SUMMARY

[006] In an exemplary embodiment, a camera monitoring system (CMS) for a vehicle includes a camera configured to face backward and provide a captured image of a field of view that includes at least part of a trailer. A display is in communication with the camera and configured to display an image that includes at least part of the captured image that includes part of the trailer. A controller is in communication with the camera and the display. The controller includes a collision warning module configured to provide an overlay on the displayed image that corresponds to a region encompassing a predicted trajectory of the trailer relative to the current position of the trailer. The overlay includes a first boundary provided by a curved line indicating an inner trajectory of the trailer and an impact area from the first boundary to the trailer.

[007] In an additional embodiment, according to any of the above items, the controller includes a memory and a processor. The controller is connected to multiple cameras, including the camera. The multiple cameras are arranged around a vehicle and configured to receive a video signal from each of the cameras. The controller includes at least one side camera configured to define a rear side view and at least one rear camera configured to generate a rear-facing view. The memory that arms Petition 870250077905, dated 01 / 09 / 2025, page 10 / 68 3 / 27 Zena, a trailer end detection module, is configured to identify a trailer end in at least one image generated by the multiple cameras. The memory also stores a trailer impact area prediction module, configured to define an impact area geometry using a set of predicted future positions of prediction points in a prediction set. The prediction points are defined along an edge of the trailer.

[008] In an additional embodiment, according to any of the above items, the impact area prediction module defines the impact area geometry by a process that includes identifying a current location t0 of a set of trailer prediction points along an inner edge of the trailer and storing the current location t0 of the prediction point set in a prediction set, identifying a first predicted future position of each prediction point at a time t1 based on a set of parameters that include at least one vehicle towing angle, a vehicle steering angle, and the current location t0 of a corresponding prediction point and storing the first predicted future position t1 in the prediction set, identifying at least one additional predicted future position of each prediction point at a time tn based on a second set of parameters that include at least the vehicle towing angle,The vehicle's steering angle and a location at a previous time tn-1 of the corresponding prediction point are used, and each location in the prediction set is converted from a three-dimensional real-world position to a two-dimensional position within a rear-view display image. A geometry that includes each two-dimensional position is then generated, and the geometry is represented on the display over the captured image as an overlay. Petition 870250077905, dated 01 / 09 / 2025, page 11 / 68 4 / 27

[009] In an additional embodiment, according to any of the above items, the controller is configured to iterate the process that is defined in the trailer impact area prediction module during a turning operation.

[0010] In an additional embodiment, according to any of the above items, the memory further includes a collision alert module that is configured to cause the controller to identify at least one object that includes an object within an image, compare a location of the object within the image with the geometry, and issue a collision warning in response to the object overlapping with the geometry.

[0011] In an additional embodiment, according to any of the above items, the object includes a curb, and the controller is configured to provide another overlay on the curb.

[0012] In an additional embodiment, according to any of the above items, the parameter set includes at least one vehicle towing angle, one vehicle steering angle, the current location t0 of the corresponding prediction point, towing angle change rate, vehicle speed and yaw rate.

[0013] In an additional embodiment, according to any of the above items, the first predicted future position of each prediction point at time t1 and of each additional predicted future position is determined by applying the parameter set to a kinematic model.

[0014] In an additional embodiment, according to any of the above items, the set of prediction points is distributed unevenly along one side of the trailer.

[0015] In an additional embodiment, according to any of the above items, the set of prediction points is concentrated at or near the identified end of the trailer. Petition 870250077905, dated 01 / 09 / 2025, p. 12 / 68 5 / 27

[0016] In an additional embodiment, according to any of the above items, the impact area is at least one of the following: shaded and hatched, with the captured image visible through the area.

[0017] In another exemplary embodiment, a method for displaying a potential trailer impact area to a vehicle operator, the method includes predicting a trailer impact area by defining an impact area geometry using a set of predicted future positions of prediction points in a prediction set, the prediction points are defined along an edge of a trailer, converting the geometry into a two-dimensional overlay and applying the two-dimensional overlay to a display of an image captured from a camera during a turning operation.

[0018] In a further embodiment, according to any of the above, predicting a trailer impact area by defining an impact area geometry using a set of predicted future positions of prediction points in a prediction set includes identifying a current location t0 of a set of prediction points of the trailer along an inner edge of the trailer and storing the current location t0 of the prediction point set in a prediction set, identifying a first predicted future position of each prediction point at a time t1 based on a set of parameters that include at least one trailer angle of a vehicle,a vehicle steering angle and the current location t0 of a corresponding prediction point and store the first predicted future point t1 in the prediction set and identify at least one additional predicted future position of each prediction point at a time tn based on a second set of parameters that include at least the vehicle's towing angle, the vehicle's steering angle, and a location at a previous time tn-1 of the, Petition 870250077905, dated 01 / 09 / 2025, page 13 / 68 6 / 27 corresponding forecast point.

[0019] In an additional embodiment, according to any of the above items, the method also includes iteration of the method during the curve operation.

[0020] In an additional embodiment, according to any of the above items, the parameter set includes at least one vehicle towing angle, one vehicle steering angle, the current location t0 of the corresponding prediction point, towing angle change rate, vehicle speed and yaw rate.

[0021] In an additional embodiment, according to any of the above items, the first predicted future position of each prediction point at time t1 and of each additional predicted future position is determined by applying the parameter set to a kinematic model.

[0022] In an additional embodiment, according to any of the above items, the conversion of geometry into a two-dimensional overlay includes converting each location in the prediction set from a three-dimensional real-world position to a two-dimensional position within a back-view display image and generating a geometry that includes each two-dimensional position, with the geometry defining the overlay.

[0023] In an additional embodiment, according to any of the above items, the method further includes a controller that identifies at least one object within an image, comparing the location of an object within the image with the geometry and issuing a collision warning in response to the object overlapping with the geometry.

[0024] In an additional embodiment, according to any of the above items, the object includes a curb, and the controller is configured to provide another overlay on the curb. Petition 870250077905, dated 01 / 09 / 2025, page 14 / 68 7 / 27

[0025] In an additional embodiment, in accordance with any of the above items, the impact area is at least one of the following: shaded and hatched, with an image captured by the camera visible through the impact area. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The disclosure can be better understood by reference to the following detailed description when considered in connection with the accompanying drawings, in which:

[0027] Figure 1A is a schematic front view of a commercial truck with a camera monitoring system (CMS) used to provide at least Class II and Class IV views.

[0028] Figure 1B is a schematic top-elevation view of a commercial truck with a camera mirror system providing Class II, Class IV, Class V, Class VI and Class VIII views.

[0029] Figure 2 is a schematic illustration of the interior of a vehicle cabin and the CMS system.

[0030] Figure 3 depicts a kinematic model for each tractor and trailer.

[0031] Figure 4 shows an aerial view of the impact areas of tractors and trailers based on the kinematic model of Figure 3.

[0032] Figures 5A-5C schematically illustrate a commercial truck turning maneuver at the beginning (Figure 3A), in the middle (Figure 3B) and at the end (Figure 3C) of a forward-moving turning maneuver.

[0033] Figure 6 illustrates a method for providing trailer impact area collision warnings using CMS.

[0034] Figure 7 schematically illustrates a predicted impact area for a trailer. Petition 870250077905, dated 01 / 09 / 2025, page 15 / 68 8 / 27

[0035] Figure 8 illustrates a detailed method for generating the trailer impact area of ​​Figure 6 and using it to generate an overlay.

[0036] Figure 9 is a flowchart of the various functionalities performed in the CMS.

[0037] Figure 10 is an aerial view of the vehicle traveling along a curved highway in traffic.

[0038] Figures 11A-11B are, respectively, the driver's and passenger's fields of view provided on the CMS displays for the vehicle position shown in Figure 10.

[0039] Figures 12A-12B show, respectively, the driver's and passenger's fields of view provided on the CMS displays for the vehicle position shown in Figure 3B.

[0040] Figures 13A-13C are alternative overlays to the overlay shown in Figure 12B.

[0041] The embodiments, examples and alternatives of the preceding paragraphs, of the claims or of the description and drawings that follow, including any of their various aspects or respective individual features, may be considered independently or in any combination. The features described in connection with one embodiment are applicable to all embodiments unless such features are incompatible. DETAILED DESCRIPTION

[0042] A schematic view of a commercial vehicle 10 is illustrated in Figures 1A and 1B. Figure 2 is a schematic top perspective view of the cab of vehicle 10, including internal displays and cameras. Vehicle 10 includes a cab or tractor 12 for pulling a trailer 14. It should be understood that the cab 12 and / or the trailer 14 may have any configuration. Although a commercial truck is contemplated in this disclosure, the invention may also be applied Petition 870250077905, dated 01 / 09 / 2025, page 16 / 68 9 / 27 of other types of vehicles. Vehicle 10 incorporates a camera monitoring system (CMS) 15 (Figure 2) which has camera arms 16a and 16b on the driver's and passenger's side (usually camera arm 16 or wing) mounted on the outside of the cabin 12. If desired, camera arms 16a and 16b may also include integrated conventional mirrors, although CMS 15 may be used to completely replace the mirrors. In further examples, each side may include multiple camera arms, each arm housing one or more cameras and / or mirrors.

[0043] Each of the camera arms 16a and 16b includes a base fixed, for example, to the cab 12. An articulated arm is supported by the base and can articulate relative to it. Fixed wings can also be used. At least one rear-facing camera 20a and 20b is arranged, respectively, within the camera arms. The external cameras 20a and 20b each have an image capture unit that captures an external field of view (FOVEX1 and FOVEX2), each including at least one of the Class II and Class IV views (Figure 1B), which are views prescribed by law in the commercial road transport sector. It is desirable to capture at least a part of the trailer 14 in the field of view, for example, the side and / or the end of the trailer, throughout the operation of the vehicle. Multiple cameras can also be used in each camera arm 16a and 16b to provide these views, if desired.Class II and Class IV views are defined in European legislation R46, for example, and the United States and other countries have similar driving visibility requirements for commercial trucks. Any reference to a Class view is not intended to be limiting, but rather illustrative of the type of view provided to a display by a specific camera. Each arm 16a and 16b may also provide a compartment housing configurable electronic components. Petition 870250077905, dated 01 / 09 / 2025, page 17 / 68 10 / 27 rados to provide various CMS features 15.

[0044] The first and second video displays 18a, 18b are arranged on each of the driver and passenger sides inside the vehicle cab 12, on or near the A-pillars 19a, 19b, to display Class II (narrow-angle view) and Class IV (wide-angle view) views (e.g., Class II represented above Class IV in a portrait-style configuration) on their respective side of the vehicle 10, which provide rear-facing side views along the vehicle 10 (e.g., parts of the trailer) that are captured by the external cameras 20a, 20b.

[0045] If you would also like video views of Class V and / or Class VI, a camera housing 16c and a camera 20c can be arranged on or near the front of the vehicle 10 to provide these views (Figure 1B). A third display 18c, arranged inside the cab 12, near the upper center of the windshield, can be used to display the Class V and Class VI views, which are facing the front of the vehicle 10, to the driver. Displays 18a, 18b, and 18c are facing a driver region 24 inside the cab 22, where an operator is seated in a seat 26. The location, size, and field(s) of view transmitted to any specific display may vary from the configurations described in this disclosure and still incorporate the disclosed invention.

[0046] If Class VIII view videos are desired, camera boxes can be arranged on the sides and rear of vehicle 10 to provide fields of view that include some or all of the Class VIII zones of vehicle 10. As illustrated, the Class VIII view includes views immediately surrounding the trailer and in the rear vicinity of the vehicle, including the rear of the trailer. In one example, a view of the rear proximity of the vehicle is generated by a rear-facing camera arranged at the rear of the vehicle and may include both Petition 870250077905, dated 01 / 09 / 2025, page 18 / 68 11 / 27 the immediate rear proximity as a traditional rear view (e.g., a view extending back to the horizon, such as might be generated by a rearview mirror on vehicles without a trailer). In such examples, the third display 18c may include one or more frames displaying Class VIII views. Alternatively, additional displays may be added next to the first, second, and third displays 18a, 18b, 18c (generally display 18) and provide a dedicated display to provide a Class VIII view.

[0047] In some cases, the Class VIII view is generated using a trailer-mounted camera 30. The trailer-mounted camera 20d is a rear-facing camera that provides a field of view behind the trailer. This rear view can be provided to one of displays 18a, 18b and / or another display 18c inside the vehicle cab 22 as a replacement for the rearview mirror or as a supplement to the rearview mirror. This view is particularly beneficial as the trailer 14 may block some or all of the views provided by a conventional rearview mirror.

[0048] The CMS 15 is also configured to use images from cameras 20a, 20b, 20c, 20d (usually camera 20), as well as images from other cameras that may be located in the vehicle or in communication with the vehicle, to determine vehicle characteristics, identify objects, and facilitate driver assistance features such as display overlays and semi-automated driver assistance systems.

[0049] These CMS 15 features and functions are used to implement multiple CMS 15 systems that assist in vehicle operation. It should be noted that a controller 30 (Figure 2) for the CMS 15 can be used to implement the various functionalities disclosed in this application. The controller 30, which communicates with the displays 18 and the cameras 20, may include one or more units. Petition 870250077905, dated 01 / 09 / 2025, page 19 / 68 12 / 27 discrete. For example, a centralized architecture might have a common controller located in vehicle 10, while a decentralized architecture might use a controller provided in each of the displays 18, for example. Furthermore, part of the controller 30 might be provided in vehicle 10, while another part of the controller 30 might be located elsewhere, for example, in the arms of camera 16. In another example, a master-slave display configuration might be used, in which one display includes the controller 30 while the other receives commands from the controller 30.

[0050] In terms of hardware architecture, such a controller may include a processor, memory (e.g., memory 31, Figure 2), and one or more input / output (I / O) device interfaces communicatively coupled via a local interface. The local interface may include, for example, but not limited to, one or more buses and / or other wired or wireless connections. The local interface may have additional elements, which are omitted for simplicity, such as controllers, buffers (caches), drivers, repeaters, and receivers to enable communication. Furthermore, the local interface may include address, control, and / or data connections to allow proper communication between the aforementioned components.

[0051] The controller 30 can be a hardware device for executing software, particularly software stored in memory (e.g., memory 31, Figure 2). The controller 30 can be a custom or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set), or, more generally, any device for executing software instructions.

[0052] Memory 31 may include any one or a combination Petition 870250077905, dated 01 / 09 / 2025, page 20 / 68 13 / 27 tion of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, VRAM, etc.)) and / or non-volatile memory elements (e.g., ROM, hard disk, tape, CD-ROM, etc.). Furthermore, memory 31 may incorporate electronic, magnetic, optical, and / or other types of storage media. Memory 31 may also have a distributed architecture, in which multiple components are located remotely from each other but can be accessed by the processor.

[0053] The software in memory 31 may include one or more separate programs, each of which includes an ordered list of executable instructions to implement logical functions. A system component embedded as software may also be interpreted as a source program, an executable program (object code), a script, or any other entity containing a set of instructions to be executed. When constructed as a source program, the program is translated by means of a compiler, assembler, interpreter, or similar, which may or may not be included in memory 31.

[0054] The disclosed input and output devices that can be coupled to the system's I / O interfaces may include input devices, for example, but not limited to, a keyboard, mouse, scanner, microphone, camera, mobile device, proximity device, etc. In addition, output devices, for example, but not limited to, a printer, display, etc. Finally, input and output devices may also include devices that communicate as inputs and outputs, for example, but not limited to, a modulator / demodulator (modem; to access another device, system or network), a radio frequency (RF) or other transceiver, a telephone interface, a bridge, a router, etc.

[0055] When controller 30 is in operation, the processor Petition 870250077905, dated 01 / 09 / 2025, page 21 / 68 14 / 27 can be configured to execute software stored in memory 31, communicate data to and from memory 31, and generally control the operations of the computing device according to the software. The software in memory 31, in whole or in part, is read by the processor, possibly buffered in the processor, and then executed.

[0056] In several examples, the controller 30 includes one or more modules with algorithm(s), equation(s), and / or decision manager(s) that receive input(s) from sensors and / or stored values. Example modules include Lane Detection Module 100, Object Detection Module 101, Trailer End Detection Module 102, Kinematic Module 104, Trailer Impact Area Prediction Module 106, Tractor Impact Area Prediction Module 108, and Collision Alert Module 110. Example inputs include a steering angle sensor 32, a vehicle speed sensor 34, and other sensor data. The vehicle configuration information 36, which relates to vehicle characteristics (e.g., trailer length, axle position, trailer type / wheelbase, tractor configuration / wheelbase, hitch point location, etc.), is provided by the manufacturer, operator, and / or determined by one or more modules.During vehicle operation, controller 30 can communicate information to the driver, fleet operator, or others using an output 39 (e.g., displays 18, loudspeaker, etc.).

[0057] The object detection module 101 includes one or more image processing algorithms configured to identify objects in captured images. The algorithms can be used to identify VRUs (e.g., pedestrians or cyclists), tractor 12 and / or trailer 14 attributes, other vehicles, signs, curbs, trees, buildings, and / or other inanimate objects. Petition 870250077905, dated 01 / 09 / 2025, page 22 / 68 15 / 27

[0058] The lane detection module 100 also uses captured image processing to identify lane markings, such as lane markers that visually divide adjacent lanes. An example algorithm is described in U.S. Publication No. US2023 / 117.719, entitled CAMERA MIRROR SYSTEM DISPLAY FOR COMMERCIAL VEHICLES INCLUDING SYSTEM FOR IDENTIFYING ROAD MARKINGS'', which is incorporated by reference in its entirety. In this publication, a lane detection module is described in which an object detection algorithm identifies a lane marking on a roadway by filtering a lane marking color from a surrounding part of the captured image. Other techniques based on deep learning technology or other computer vision methods may be used, if desired.

[0059] The trailer end detection module 102 is another image processing module that extracts one or more trailer features from captured images to determine the location of the trailer end in three-dimensional space. These extracted attributes can be used to detect objects such as tractor wheels, trailer edges, and other features. Exemplary wheel detection algorithm techniques are disclosed in U.S. Publication No. US2023 / 202.394, entitled "CAMERA MONITOR SYSTEM FOR COMMERCIAL VEHICLES INCLUDING WHEEL POSITION ESTIMATION", which is incorporated herein by reference in its entirety. Examples of trailer edge detection algorithm techniques are disclosed in U.S. Publication No. US2023 / 125.045 entitled "TRAILER END". 'TRACKING IN CAMERA MONITORING SYSTEM', which is incorporated herein by reference in its entirety. Other techniques may be used if desired.

[0060] Many of the functions described use a kinematic model. Petition 870250077905, dated 01 / 09 / 2025, page 23 / 68 16 / 27 co (provided by kinematic module 104) to determine where one or more elements of the tractor 12 and / or trailer 14 are currently located or predicted to be located. The kinematic model, schematically illustrated in Figure 3, models a tractor turning radius R1 and a trailer turning radius R2, with distinct and distinct radii. The tractor 12 has front wheels 42 and rear wheels 43. The kinematic model models the front wheels 42 in a way that takes into account their Ackerman steering characteristics, which is a common steering geometry approach that allows the outer and inner wheels to travel in different radial trajectories to reduce tire friction. Trailer 14, which is connected to tractor 12 at a hitch point (i.e., fifth wheel), has 44 rear wheels. VT is the speed of the tractor (corresponding to the speed of vehicle N, at 34 in Figure 2) and Vn is the speed of the trailer in its longitudinal direction.Vy is the velocity of the trailer end in the same direction as the tractor's direction of travel, and Vx is the component of the trailer end velocity in a direction transverse to Vy. The trailer angle θ, which is the angle between trailer 14 and tractor 12, can be determined using several suitable approaches. The trailer angle is calculated with a kinematic model during forward driving and estimated with an image processing method during reverse driving. Another method can use the LiDAR point cloud to calculate the angle and dimensions of the trailer. Yet another method could be a relative / absolute angular position sensor mounted at the hitch location.

[0061] The kinematic model of vehicle 10 is simplified by using two bicycle models, or half-tracks. That is, not all wheels need to be represented in the model. In other words, the bicycle model, used as a kinematic model in Kinematic Module 104, is a simplified representation of a four-wheeled vehicle. Petition 870250077905, dated 01 / 09 / 2025, page 24 / 68 17 / 27 wheels. Only the inner wheels of a turning maneuver can be modeled, as this is the side of vehicle 10 that presents the greatest risk of collision. It is used to predict the position of vehicle 10 using the instantaneous position, angles, speeds, and accelerations acting on the system. Furthermore, this kinematic model assumes that all slip angles are zero. As a result, the speed and future displacement of a vehicle component (e.g., the location of the wheels and / or the end of the trailer) can be propagated by the mathematical algorithm very quickly. Additionally, only the trailer angle, vehicle speed, and steering angle are required as input data, providing a simple, accurate, and fast approach to trajectory prediction.

[0062] The kinematic model disclosed is provided by a first bicycle model with Ackerman steering, indicating a predicted trajectory of the tractor (112 in Figure 4). A second bicycle model is connected to the first bicycle model by the hitch point 48, where the second bicycle model indicates a predicted trajectory of the trailer (114 in Figure 4). The trajectory of the inner wheels of the tractor and trailer is shown respectively at 112' and 114', at the inner limit of their respective tractor and trailer trajectories 112 and 114.

[0063] The kinematic module 104 receives the current trailer angle, steering angle, and vehicle speed to calculate the predicted trajectories of the tractor and trailer 112 and 114. If desired, one or both of the predicted trajectories of the tractor and trailer 112 and 114 can be illustrated on one or more of the displays 18 as overlays (e.g., in at least one of the Class II and Class IV views) to assist the driver in maneuvering the vehicle 10 (e.g., Figures 12A to 13C and / or aerial view, as in Figure 4). In one example, the kinematic algorithm is executed (i.e., calculated) Petition 870250077905, dated 01 / 09 / 2025, page 25 / 68 18 / 27 continuously while driving. In one example, the impact area is displayed when the trailer angle exceeds a certain limit (e.g., 5 degrees or 10 degrees). This limit can be adjusted by the driver if desired.

[0064] In an example of operation, the CMS 15 uses the kinematic module 104 to predict a trailer impact zone 14 during a turning operation and generates a two-dimensional overlay to digitally overlay at least one of the displayed Class II / IV images, showing the vehicle operator an expected trailer impact zone 14 and allowing the operator to adjust vehicle operations accordingly. The CMS 15 uses captured images received from cameras 20a and 20b, as well as any other cameras and vehicle operation data received from a vehicle master controller via a data connection such as a CAN or LIN bus, to estimate a predicted tractor and / or trailer side position at each of multiple lateral positions and at multiple points in time. These positions are converted into a geometric area encompassing all positions.Therefore, the shape and size of the geometric area are not fixed, but reflect a realistic predicted impact area of ​​the trailer.

[0065] With continued reference to Figures 1A-2, Figures 5A, Figures 5B and 5C provide a scene 200 illustrating an example of forward turning operation of a tractor 12 towing a trailer 14, with Figure 5A illustrating the beginning of the turn, Figure 5B illustrating the middle of the turn, and Figure 5C illustrating the end of the turn. The schematic tractor 12 and trailer 14 are schematic representations of the tractor 12 and trailer 14 illustrated in Figures 1A and 1B. However, the system and process for estimating the trailer impact area described herein can be incorporated into any similar tractor-trailer configuration, including a CMS 15, and disclosure is not limited. Petition 870250077905, dated 01 / 09 / 2025, page 26 / 68 19 / 27 to the specific environment of the example. The disclosed system and method determine and display a tractor impact area as well as a trailer impact area, although only a tractor impact area is described below for simplicity and because the trailer 14 is normally the part of the vehicle 10 most at risk of collision with an object.

[0066] In the illustrated curve sequence, tractor 12 pulls the trailer on a right-hand curve on a road 202. The curve occurs along a curve trajectory 210, with the trajectory 210 being directly controlled by the steering angle of tractor 12. As vehicle 10 (including tractor 12 and trailer 14) travels along the curve trajectory 210, trailer 14, and particularly an inner side 14' of trailer 14, cuts towards the inside of the curve, with trailer 14 crossing parts of the road and adjacent terrain 220 that tractor 12 did not pass. The portions of the area inside the curve where trailer 14 passes are called the impact area, as objects positioned in the impact area will be struck by the side 14' of trailer 14 if they are tall enough and likely to be struck by the tires, or will pass under trailer 14 if they are not tall enough to be struck by the side 14'.

[0067] To avoid accidental impacts, the impact area prediction system uses vehicle data (e.g., steering angle, steering rate, trailer angle, vehicle speed, trailer wheelbase, tractor wheelbase, hitch point location, yaw rate, and the like) to generate a predicted impact zone over time using a 300 process illustrated in Figure 6. The predicted impact zone is a prediction of the path the trailer will take along the curve and is continuously recalculated as the curve progresses.

[0068] The process initially identifies that an operation of Petition 870250077905, dated 01 / 09 / 2025, page 27 / 68 20 / 27 curve is occurring in a 310 Identify Curve Operation step. The curve operation can be automatically identified by detecting a change in steering angle, a change in the direction of the vehicle's route detected geospatially, a combination of both, or a manual turn start input from the vehicle operator.

[0069] Once the curve is identified, the trailer impact area prediction system 40 determines an expected impact area in the Trailer Impact Area Prediction step 320. The predicted impact area is the zone extending away from the side of the trailer 14, through which the trailer 14 will pass as vehicle 10 completes the curve. In some examples, the prediction is based on a snapshot of the current steering angle and other vehicle parameters. In other examples, a change in steering angle over time is used instead of an instantaneous steering angle. Similarly, in other examples, one or more additional vehicle parameters may be values ​​over time, rather than snapshots. In another example, known, knowable, or detectable external factors (e.g., road conditions, road gradient, weather conditions, and the like) are incorporated into the prediction process.

[0070] In addition to predicting the impact area, process 300 identifies objects (e.g., sign 222, tree 224, and curb 226) within the received images or another system within the CMS 15 communicating with the trailer impact area prediction system 40 provides object identifications to the trailer impact area prediction system 40. An overlay can be represented on the display 18 over the curb to improve visibility for the driver (Figure 12B). After receiving the object identifications, process 300 compares the location of the identified objects 222, 224, and 226. Petition 870250077905, dated 01 / 09 / 2025, page 28 / 68 21 / 27 with the impact area estimated in step 330 of Compare Impact Area with Object Detection. When objects 222, 224, and 226 cross the estimated impact area, process 300 issues a warning to the vehicle operator in step 340 of Issue Warning When Detected Object Is in the Impact Area. The warning may be an audio output and / or a visual output that alerts the vehicle operator to a possible collision. Because the process is predictive in nature, the alert allows the vehicle operator to adjust the turning operation to avoid the expected collision, and the system automatically updates the predicted impact zone to compensate for the correction.

[0071] With continued reference to the general process in Figure 6, the Figures 7 and 8 illustrate the detailed process for generating the estimated impact zone 410. As described above, initially the prediction system 40 receives the trailer angle and trailer end detection in a step Receive trailer angle and trailer end detection 510. The trailer angle and end detection can be performed using any conventional detection, including image-based detections, sensor-based detections and / or any other existing detection system.

[0072] Information about the angle and end of the trailer is then used by the CMS controller to identify multiple detection points or positions along the inner edge 14' of the trailer 14 in a 520 step of Setting Detection Points Along the Trailer in 3D. As used herein, 3D refers to positioning in a real three-dimensional space relative to the trailer and 2D refers to a position in an image frame along a two-dimensional axis (EG, XY). Existing systems, and particularly vehicle camera monitoring systems, may utilize various established processes or methodologies to con Petition 870250077905, dated 01 / 09 / 2025, page 29 / 68 22 / 27 to convert a 3D position into a 2D position of a given camera view.

[0073] Multiple detection points 430-438 are distributed along the 14' side of the trailer 14, within the curve. In the illustrated example, detection points 430-438 are evenly distributed along the 14' side. In alternative examples, a uniform distribution may not be necessary and detection points 430-438 may be concentrated near the end point (end point of trailer 430), with detection points near the end of the tractor being more spaced out. In one example, a fixed number (e.g., five) of detection points is used and the points are distributed along the 14' side. In another example, the number of detection points 430-438 is determined based on the trailer length and a desired distribution of detection points.

[0074] After defining the initial detection points 430-438 along the side of the trailer 14, the impact area prediction system 40 applies the steering angle, trailer angle, trailer angle change rate, vehicle speed, yaw rate and / or any similar parameters known to the CMS 15 to a kinematic model to predict a three-dimensional position of each detection point at a future time (t1) with a predetermined duration in the future (e.g., 1 second) in a 3D Detection Point Future Position Prediction step 530. Each predicted point at t1 is stored, and step 520 is reiterated 522 using the t1 position of detection point 430'-438' as the starting point, generating a new predicted position of detection point 430'-438' at t2. In the example illustrated in Figures 7 and 8, the forecast is iterated twice, generating two predicted future positions (t1, t2).It is recognized that, in alternative examples, the number of iterations may be increased when a longer duration forecast is required. (From the mes.) Petition 870250077905, dated 01 / 09 / 2025, p. 30 / 68 23 / 27 In this way, the number of iterations can be increased with a shorter duration between prediction points (e.g., t0 to t1, t1 to t2, etc.), resulting in a larger number of prediction points, generating the geometry of the predicted impact area.

[0075] After iteration 522, the process aggregates detection points 430-438 and converts the 3D positions of the prediction points into two-dimensional positions within an image plane of a Class II / IV image to which the overlay is being applied, in step 540 of Aggregate Detection Points and Convert 3D Position to 2D Image Point. After converting the image points into two-dimensional image points, the aggregated image points are converted into an impact area 410, defining a boundary in 2D space, including all predicted positions 430-438 from t0 to tn, in step 550 of Convert 2D Detection Points to Impact Area in Class II / IV Images. The limit is defined as the minimum space required to include all predicted positions 430438. The impact area 410 is then aligned with the side of the truck in the Class II / IV images and shaded as an overlay.

[0076] The trailer impact area is also useful in a potential corner-cutting scenario when vehicle 10 is traveling on a curved roadway in traffic (Figure 10) with other vehicles 700 and 702. On a winding roadway, it becomes more likely that the trailer end will cross the lane markers 706 that indicate the boundaries of adjacent lanes 704, creating a potentially dangerous situation. To resolve this, the CMS 15 uses the lane detection module 100 to determine the lane boundary for vehicle 10, the trailer end detection module 102 to determine the trailer boundary, and the collision warning module 110 to determine an imminent intersection between the trailer boundary and the lane boundary. The Petition 870250077905, dated 01 / 09 / 2025, page 31 / 68 The 24 / 27 trailer impact area prediction module 106 can be used to predict the future trailer impact area and anticipate the imminent intersection. If the impact area is used to determine the trailer's curve cut, the kinematic model used may be different from that used to calculate the trailer angle. The trailer collision kinematic model uses the trailer angle as input, along with the other signals mentioned. However, if only the final wheel / end location of the trailer needs to be instantly checked relative to the lane marking, the kinematic model used to calculate the trailer angle may be sufficient. The alert can be based on time to cross and / or time to collision. The alert is issued at different levels depending on the severity level.

[0077] The driver and passenger displays 18a and 18b are shown in Figures 11A and 11B. Vehicle 10 and its trailer 14 (and the end of the trailer) are moving in the direction of vehicle 702, which is in an inner lane 704, where the lane boundary (markers 706) will be cut. A warning 708 is superimposed on display 18b, indicating that the turn cut is about to occur. Warning 708, which may be accompanied by an audible warning, is given in a highlight color (perhaps flashing) and does not obstruct lanes 704. Warning 708 encourages the driver to pay attention to the location and timing of the trailer.

[0078] The overlay is continuously updated as the process is iterated, thus allowing the impact area overlay to be accurate along the vehicle's curve. In one example, the overlay includes a first overlay on the displayed image corresponding to a first region encompassing a predicted trajectory of the trailer relative to the trailer's current position, the first overlay including a first boundary provided by a first line. Petition 870250077905, dated 01 / 09 / 2025, page 32 / 68 25 / 27 curve indicating an internal trailer trajectory and a trailer impact area from the first boundary to the trailer. In another example, the overlay includes a second overlay in the displayed image corresponding to a second region encompassing a predicted tractor trajectory relative to the tractor's current position, the first overlay including a second boundary provided by a second curved line indicating an internal tractor trajectory and a tractor impact area from the second boundary to the tractor.

[0079] Examples of overlaps are illustrated in the Figures 12A-13C. Figures 12A (outer turning radius) and 12B (inner turning radius) represent, respectively, displays 18a and 18b on the driver's and passenger's sides during a turning maneuver similar to that shown in Figure 5C. The overlay representing the trailer impact area includes a first boundary 804, provided by a curved line indicating an inner trailer trajectory. The trailer impact area 806 extends from the first boundary 804 to the trailer 14, particularly to a second boundary 808, for example, a straight line projected from the side of the trailer 14 to the ground. In the example shown in Figure 12B, the trailer impact area 806 is hatched to provide better visibility of the impact area for the driver and to allow the captured image to still be visible. An overlay 802 may be provided on the curb 800, if desired, in a color contrasting with the trailer impact overlay.

[0080] Additional examples of impact area overlays are shown in Figures 13A-13C, which may be available to the driver in a display customization menu. In the example in Figure 13A, the first boundary 804' is outlined with a colored line and the trailer impact area 806' displays only the captured image, without overlay. In the example in Figure 13B, the first Petition 870250077905, dated 01 / 09 / 2025, p. 33 / 68 In Figure 13C, the first 804'' boundary is outlined with a line of a different color, and the trailer impact area 806'', and the trailer impact area 806'', is shaded with a prominent shaded region and bounded by the first 806'' boundary and the second 808'' boundary. In the example in Figure 13C, the first 804''' boundary is outlined with a darker colored line, and the trailer impact area 806'', and the trailer impact area 806'', is lightly shaded and bounded by the first 806''' boundary and the second 808''' boundary. Other display schemes may be used in addition to those shown.

[0081] The CMS 15 includes a Decision Manager or Collision Alert Module 110, shown in Figure 9, which communicates with modules 101 to 108 to assess the proximity between the predicted trajectories of the tractor and / or trailer 112 and 114 (i.e., the impact areas of the tractor and trailer) and one or more objects (e.g., predicting an imminent collision with the curb, corner cut, object collision, etc.). The decision manager considers the estimated time to the event, the severity (which object it is), the rate of approach between the objects, etc. The decision manager issues a soft alert 602 or a sharp alert 604 via one or more output devices 39. In the example of a displayed alert (e.g., an overlay signal), the severity can be communicated to the driver based on the signal's flash frequency or the signal's color change. In the example of an audible alert, the sound level or pattern can be altered based on the severity.

[0082] Although described above in relation to a commercial tractor pulling a trailer, it is understood that the wide turning requirement is present in any similar vehicle. Thus, the features, systems and apparatus of the invention described herein are applicable to any configurations of similar vehicles and are not limited to configurations of commercial tractors with trailers. Petition 870250077905, dated 01 / 09 / 2025, page 34 / 68 27 / 27

[0083] Although an example of embodiment has been disclosed, a professional with common knowledge in this area would recognize that certain modifications would fall within the scope of the claims. For this reason, the following claims should be studied to determine their true scope and content. Petition 870250077905, dated 01 / 09 / 2025, page 35 / 68

Claims

1 / 6 CLAIMS 1. A camera monitoring system (CMS) for a vehicle, characterized in that: a camera configured to face backward and provide a captured image of a field of view including at least part of a trailer; a display in communication with the camera and configured to represent a displayed image comprising at least part of the captured image, including part of the trailer; a controller in communication with the camera and the display, the controller comprising a collision warning module configured to provide an overlay on the displayed image corresponding to a region encompassing a predicted path for the trailer relative to a current position of the trailer, the overlay including a first boundary provided by a curved line indicative of an inner path of the trailer and an impact area from the first boundary to the trailer;wherein the controller includes a trailer impact area prediction module configured to define an impact area geometry and wherein the controller includes a collision warning module configured to cause the controller to identify at least one object, including an object within an image, compare the location of the object within the image with the impact area geometry, and issue a collision warning in response to the object overlapping with the impact area geometry, wherein the object includes a curb and the controller is configured to provide another overlap on the curb.

2. CMS, according to claim 1, characterized in that the controller includes a memory and a processor; Petition 870250077905, dated 01 / 09 / 2025, p.44 / 68 2 / 6 the controller being connected to multiple cameras, including the camera, the multiple cameras arranged around a vehicle and configured to receive a video feed from each of the cameras in the multiple cameras, the controller including at least one side camera configured to define a rear side view and at least one rear camera configured to generate a rear-facing view; the memory storing a trailer end detection module configured to identify a trailer end in at least one image generated by the multiple cameras; and the memory further storing the trailer impact area prediction module configured to define the impact area geometry using a set of predicted future positions of prediction points in a prediction set, the prediction points being defined along an edge of the trailer.

3. CMS, according to claim 2, characterized in that the impact area prediction module defines the impact area geometry by a process that includes: identifying a current location (t0) of a set of trailer prediction points along an inner edge of the trailer and storing the current location (t0) of the prediction point set in a prediction set; identifying a first predicted future position of each prediction point at a time (t1) based on a set of parameters including at least one vehicle trailer angle, a vehicle steering angle, and the current location (t0) of a corresponding prediction point and storing the first predicted future position (t1) in the prediction set; identifying at least one future prediction position... Petition 870250077905, dated 01 / 09 / 2025, p.45 / 68 3 / 6 tional of each prediction point at a time (tn) based on a second set of parameters, including at least the vehicle's towing angle, the vehicle's steering angle, and a location at a previous time (tn-1) of the corresponding prediction point; and convert each location in the prediction set from a three-dimensional real-world position to a two-dimensional position within a rear-view display image, generating a geometry including each two-dimensional position and representing the geometry on the captured image as an overlay on the display.

4. CMS, according to claim 3, characterized in that the controller is configured to iterate the process defined in the trailer impact area prediction module during a turning operation.

5. CMS, according to claim 3, characterized in that the parameter set includes at least one vehicle towing angle, a vehicle steering angle, the current location (t0) of the corresponding prediction point, the rate of change of the towing angle, the vehicle speed, and the yaw rate.

6. CMS, according to claim 3, characterized in that the first predicted future position of each prediction point in time (t1) and of each additional predicted future position is determined by applying the parameter set to a kinematic model.

7. CMS, according to claim 1, characterized in that the set of prediction points is unevenly distributed along one side of the trailer.

8. CMS, according to claim 7, characterized in that the set of prediction points is concentrated at or near the identified end of the trailer.

9. CMS, according to claim 1, characterized by the fact that the overlay is at least one between shaded and hatched, with the captured image visible through the overlay.

10. A method for displaying a potential trailer impact area to a vehicle operator, characterized in that it comprises: predicting a trailer impact area by defining an impact area geometry using a set of predicted future positions of prediction points in a prediction set, the prediction points being defined along an edge of a trailer; converting the impact area geometry into a two-dimensional overlay; applying the two-dimensional overlay to a display of an image captured from a camera during a rotation operation; identifying at least one object within the captured image, wherein the object includes a curb; comparing the location of the object within the image with the impact area geometry; applying another overlay to the curb; and issuing a collision warning in response to the object overlaying the impact area geometry.

11. Method according to claim 10, characterized in that predicting the impact area of ​​a trailer, defining the impact area geometry using a set of predicted future positions of prediction points in a prediction set, comprises: identifying a current location (t0) of a set of prediction points of the trailer along an inner edge of the trailer and storing the current location (t0) of the prediction point set in a prediction set; Petition 870250077905, dated 01 / 09 / 2025, p.47 / 68 5 / 6 identify a first predicted future position of each prediction point at a time (t1) based on a set of parameters including at least one vehicle towing angle, a vehicle steering angle, and the current location (t0) of a corresponding prediction point and store the first predicted future point (t1) in the prediction set; and identify at least one additional predicted future position of each prediction point at a time (tn) based on a second set of parameters, including at least the vehicle towing angle, the vehicle steering angle, and a location at a previous time (tn-1) of the corresponding prediction point.

12. Method, according to claim 11, characterized in that it further comprises iteration of the method during the curve operation.

13. Method, according to claim 11, characterized in that the parameter set includes at least one vehicle towing angle, a vehicle steering angle, the current location (t0) of the corresponding prediction point, the rate of change of the towing angle, the vehicle speed, and the yaw rate.

14. Method, according to claim 11, characterized in that the first predicted future position of each prediction point in time (t1) and of each additional predicted future position is determined by applying the parameter set to a kinematic model.

15. Method according to claim 10, characterized in that the conversion of the impact area geometry into a two-dimensional overlay comprises converting each location in the prediction set from a three-dimensional real-world position to a two-dimensional position within a rear-view display image and generating a geometry including each two-dimensional position, with the impact area geometry defining the overlay.

16. Method, according to claim 10, characterized in that the overlay is at least one of shaded and hatched, with an image captured by the camera visible through the overlay. Petition 870250077905, dated 01 / 09 / 2025, pp. 49 / 68

Images