Ultrasonic detection-based reversal docking support system

Abstract

A method for assisting in the reverse operation of a tractor-trailer includes determining a predicted trailer path based at least partially on the steering angle of the tractor, identifying the location of adjacent objects adjacent to the trailer using ultrasonic sensors, and identifying the state of the predicted path as one of the following: correct, incorrect but correctable, and uncorrectable. The state is obtained by determining whether the predicted path would cause a portion of the trailer to intersect with the location of at least one adjacent object. After determining the state, the method identifies steering angle corrections that can adjust the predicted path so that a portion of the trailer does not intersect with an adjacent object.

Description

Technical Field

[0001] This disclosure relates to docking assistance for commercial vehicles.

Background Art

[0002] Mirror replacement systems, camera systems for supplementing mirror views, and other camera monitoring systems are utilized in commercial vehicles to enhance a vehicle operator's ability to view the surrounding environment. A camera monitoring system (CMS) utilizes one or more cameras to provide an enhanced field of view to a vehicle operator. In some examples, a mirror replacement system covers a wider field of view than a conventional mirror or includes views that are not fully obtainable via a conventional mirror.

[0003] Driver assistance systems, semi-autonomous driver assistance systems, and other vehicle systems use or require knowledge of the relative positions of a trailer and objects the trailer is passing by to determine an estimated path of the trailer and provide warnings when the trailer is about to collide with an adjacent object. In addition to the information required from the relative positions, similar information regarding direction of movement, speed of movement, and vehicle operation is necessary to accurately predict the physical movement of a tractor and trailer.

Summary of the Invention

[0004] An exemplary method for assisting in the reverse operation of a tractor-trailer includes the steps of: determining a predicted trailer path of the trailer based at least partially on the steering angle of the tractor; identifying the location of at least one adjacent object relative to the trailer using a set of ultrasonic sensors positioned around the trailer; determining whether the predicted path causes a portion of the trailer to intersect with the location of the at least one adjacent object, and identifying steering angle modifications that, in response to detection that a portion of the trailer is predicted to intersect with the location of the at least one adjacent object without modification, would allow the predicted path to be adjusted so that no portion of the trailer intersects with the location of the at least one adjacent object, thereby identifying the state of the predicted path as correct, incorrect but correctable, and uncorrectable; and displaying one of the following messages based on the state of the predicted path: no path adjustment needed, path adjustment needed, and path correction impossible.

[0005] In another example of the above method for assisting in the reverse operation of a truck trailer, the path adjustment message includes a steering angle adjustment command.

[0006] In another example of any of the above methods for assisting the reverse operation of a truck trailer, the steering angle adjustment includes at least one of an icon indicating the direction and magnitude of the adjustment, and a text description of the direction and angle of the steering angle adjustment.

[0007] In another example of any of the above methods for assisting in the reverse operation of a truck trailer, the adjustment message includes color coding to indicate the magnitude of the necessary correction.

[0008] In another example of any of the above methods for assisting in the reverse operation of a truck trailer, the predicted trailer path is determined using a kinematic model.

[0009] In another example of any of the above methods for assisting the reversing operation of a truck trailer, the step of identifying the position of at least one adjacent object relative to the trailer using a set of ultrasonic sensors positioned around the trailer includes generating an ultrasonic point cloud that defines the relative position of each object adjacent to the trailer.

[0010] Another example of any of the above methods for assisting in the reverse operation of a truck trailer further includes the step of displaying distance markers indicating the shortest distance between the trailer and adjacent objects on each side of the trailer.

[0011] Another example of any of the above methods for assisting in the reverse operation of a truck trailer further includes the step of displaying a distance marker indicating the shortest distance from the rear of the trailer to an object behind the trailer.

[0012] In another example of any of the above methods for assisting in the reverse operation of a truck trailer, one of the following messages—no route adjustment required, route adjustment required, and route correction impossible—based on the predicted route status, is displayed as an overlay on the mirrored replacement image.

[0013] In another example of any of the above methods for assisting the reversing operation of a truck trailer, the mirror replacement image is a stitched image from a rear-facing camera on the driver's side and a rear-facing camera on the passenger's side.

[0014] Another example of any of the above methods for assisting in the reverse operation of a truck trailer further includes the step of generating a visual overlay of the predicted trailer path and applying the visual overlay to the mirror replacement image.

[0015] In another example of any of the above methods for assisting the reverse operation of a truck trailer, the visual overlay of the predicted trailer path includes shading the area of ​​the image that the trailer is expected to traverse.

[0016] In another example of any of the above methods for assisting in the reverse operation of a truck trailer, the step of displaying the route adjustment message includes changing the shading to a first color, and the step of displaying the route correction impossible message includes changing the shading to a second color.

[0017] Another example of any of the above methods provided to assist in the reverse operation of a truck trailer further includes the step of generating and displaying a top-down view of the trailer, the top-down view of the trailer including the predicted trailer path and one of the following based on the status of the predicted path message: the no-path-adjustment message, the path-adjustment message, and the path-correction-impossible message.

[0018] In one exemplary embodiment, the vehicle controller comprises a processor and memory, the memory storing instructions for the processor to determine a predicted trailer path of the trailer based at least partially on the steering angle of the tractor; to identify the location of at least one adjacent object relative to the trailer using a pair of ultrasonic sensors positioned around the trailer; to determine whether the predicted path intersects a portion of the trailer with the location of the at least one adjacent object; and to identify steering angle modifications that, in response to detection that a portion of the trailer is predicted to intersect with the location of the at least one adjacent object without modification, allow the predicted path to be adjusted so that no portion of the trailer intersects with the location of the at least one adjacent object, thereby identifying the state of the predicted path as correct, incorrect but correctable, and uncorrectable; and to display on a screen one of a no-route adjustment message, a route adjustment message, and a route correction impossible message based on the state of the predicted path.

[0019] Another example of the vehicle controller described above further includes a plurality of ultrasonic sensors disposed around the trailer, and each of the ultrasonic sensors is connected to the processor such that the processor receives the sensor output of the ultrasonic sensor.

[0020] In another example of any of the vehicle controllers described above, the vehicle controller is a component of a camera monitoring system.

Brief Description of the Drawings

[0021] The present disclosure can be further understood by referring to the following detailed description in conjunction with the accompanying drawings.

[0022] [Figure 1A] It is a schematic front view of a commercial truck equipped with a camera monitoring system (CMS) used to provide at least Class II and Class IV views.

[0023] [Figure 1B] It is a schematic top view of a commercial truck equipped with a camera monitoring system that provides views of Class II, Class IV, Class V, and Class VI.

[0024] [Figure 2] It is a schematic upper perspective view of a vehicle cab including a display and an in-vehicle camera.

[0025] [Figure 3] An example of a reverse operation to the cargo entrance of a commercial vehicle is shown.

[0026] [Figure 4] An exemplary reverse operation at a correct trailer angle using the first display configuration is shown.

[0027] [Figure 5] An exemplary reverse operation at a modifiable trailer angle using the first display configuration is schematically shown.

[0028] [Figure 6] A schematic diagram illustrates an exemplary reverse operation at an uncorrectable trailer angle using the first display configuration.

[0029] [Figure 7] This demonstrates an exemplary reverse operation at the correct trailer angle using the second display configuration.

[0030] [Figure 8] A schematic diagram illustrates an exemplary reverse operation with a modifiable trailer angle using the second display configuration.

[0031] [Figure 9] A schematic diagram illustrates an exemplary reverse operation at an uncorrectable trailer angle using the second display configuration.

[0032] [Figure 10] This shows the processor that operates the reverse assist system.

[0033] The embodiments, examples, and substitutes, claims, or the following descriptions and drawings in the paragraphs above may be adopted independently or in any combination, including any of their various aspects or their respective individual features. Features described in relation to one embodiment are applicable to all embodiments unless such features are incompatible. [Modes for carrying out the invention]

[0034] A schematic diagram of a commercial vehicle 10 is shown in Figures 1A and 1B. Figure 2 schematically shows the interior of the vehicle's cab. The vehicle 10 includes a vehicle cab or tractor 12 for towing a trailer 14. Although this disclosure envisions a commercial truck, the present invention can be applied to other types of vehicles and is not limited to commercial vehicles. The vehicle 10 incorporates a camera surveillance system (CMS) 15 (shown in Figure 2), which includes driver's and passenger's side camera arms 16a, 16b mounted on the outside of the vehicle cab 12. If necessary, the camera arms 16a, 16b may also include conventional mirrors integrated with them, but the CMS 15 can also completely replace the mirrors. In additional examples, multiple camera arms may be included on each side, each arm housing one or more cameras and / or mirrors.

[0035] Each camera arm 16a, 16b includes a base fixed to, for example, the driver's cab 12. A swivel arm is supported by the base and may be articulated thereto. At least one rear-facing camera 20a, 20b is positioned within each camera arm. The external cameras 20a, 20b each provide external field views FOVEX1, FOVEX2, each including at least one of Class II and Class IV views (Figure 1b), which are legally defined views in the commercial truck industry. Multiple cameras may be used in each camera arm 16a, 16b to provide these views as needed. Each arm 16a, 16b may also provide a housing enclosing electronics configured to provide various features of the CMS 15.

[0036] The first and second video displays 18a and 18b are located on or near the A-pillars 19a and 19b in the driver's seat 12, on the driver's side and passenger's side, respectively, and display Class II and Class IV views on each side of the vehicle 10, which provide a rearward view along the vehicle 10 captured by the external cameras 20a and 20b.

[0037] If images of Class V and Class VI views are also required, a camera housing 16c and a camera 20c may be positioned in front of or near the front of the vehicle 10 to provide these views (Figure 1b). A third display 18c, located in the driver's cab 12 near the upper center of the windshield, can be used to display Class V and Class VI views forward of the vehicle 10 to the driver.

[0038] If a Class VIII view is required, camera housings can be positioned on the sides and rear of the vehicle 10 to provide a field of view that includes part or all of the Class VIII zone of the vehicle 10. In such an example, the third display 18c may include one or more frames displaying the Class VIII view. Alternatively, additional displays can be added near the first, second, and third displays 18a, 18b, and 18c to provide dedicated displays for providing the Class VIII view. The displays 18a, 18b, and 18c are oriented towards the driver area 24 within the driver's cab 22 where the driver is seated in the driver's seat 26.

[0039] Trailer 14 includes several ultrasonic sensors 102, 104, 106, and 108. Ultrasonic sensors 102-108 enable the detection of short-range point clouds that identify objects adjacent to trailer 14 and determine the distance between those objects and trailer 14. These detections are provided to a camera monitoring system (CMS15) and / or another vehicle controller, which communicates with CMS15 via any conventional sensor communication method. CMS15 and / or the vehicle controller communicating with CMS15 include a docking assistance program that determines the ideal reverse route for backing trailer 16 into the corresponding loading dock. In other examples, other reverse operations may use the same process based on ultrasonic sensor position detection, and the assistance system is not limited to docking operations. The docking assistance program displays prompts and / or instructions above the view on display panels 18a, 18b, which help the vehicle operator correctly perform the reverse operation.

[0040] Referring to Figures 1A and 2, Figure 3 schematically illustrates an exemplary reversing operation 200 in which the tractor 210 reverses the trailer 220 into the loading dock 230. A specific space 232 corresponding to the loading dock 230 is located between two adjacent parked trailers 240. In a different environment, the trailers 240 may be replaced with other types of obstacles, and the system will function in the same way.

[0041] To properly fit trailer 220 into the loading dock 230, the controller operating the docking assistance program determines that trailer 220 should follow a path 222, which can be achieved by moving the hitch point connecting tractor 210 to trailer 222 along a hitch point path 212. The path can be determined by the controller using any kinematic path estimation model. The hitch point path 212 is fixed relative to tractor 210 and is therefore directly controlled by the steering of tractor 210. In contrast, trailer path 222 includes some variation due to the trailer 220's ability to turn relative to tractor 210 at hitch point 214.

[0042] Instructions can be given to the vehicle operator regarding how to follow the trailer path 222, and these instructions may include requested modifications to operations to shift from the current path to the correct path 222. In one example, the requested steering modification may be specified by the steering wheel, and in another example, the requested steering modification can be made relative to the direction of the trailer. In the former, the driver is instructed to turn the steering wheel to the left, and as a result the trailer is steered to the left or right. This example is called a steering wheel instruction. In the latter, the driver is instructed to steer the trailer to the left, which requires the driver to turn the steering wheel to the left or right. This example is called a trailer steering instruction.

[0043] To assist the driver / operator of tractor 210 in properly docking trailer 220 into loading dock 230, the docking assistance program uses ultrasonic sensors 202, 204, 206, and 208 located on the sides and ends of trailer 220 to determine the distance 250 between sensors 202-208 and an object adjacent to the elongated compartment 232 where trailer 220 will dock (such as trailer 240). This information is used to create a point cloud that identifies the position of the object adjacent to trailer 220. In addition to the information from the ultrasonic sensors, the controller operating the docking assistance program receives operational information from steering angle sensors and other vehicle information available via the connection to the vehicle controller. The received information is used in combination with a kinematic model to determine whether the current operation keeps trailer 220 along the trailer path 222. If the operational parameters are incorrect, the driver determines a corrective action and provides the operator with the corrective action via one or more displays 18a, 18b, and 18c in the cab.

[0044] During a reverse docking operation following route 222, the vehicle can be in three possible states: the correct route, the incorrect but correctable route, and the incorrect and uncorrectable route.

[0045] The correct path is a state in which the relative positions of the trailer 220 and adjacent objects 240, combined with the current steering angle from the driver's cab, result in a reverse path that correctly fits the trailer 220 into the corresponding dock 230.

[0046] An incorrect but correctable path is a condition in which the relative positions of the trailer 220 and adjacent objects 240, combined with the current steering angle, result in a reverse path that would collide with one of the adjacent objects 240, but the path can be corrected while maintaining the reverse operation through a steering angle correction or another maneuver correction.

[0047] An incorrect but uncorrectable path is one in which continuing the reverse maneuver would result in a collision with one or both of the adjacent objects 240, and there are no steering corrections available to correct the path. In such a situation, the docking assistance program instructs the driver to exit the lane and resume steering.

[0048] Referring to Figures 1A–3, Figure 4 schematically illustrates an example of a correct path state via a top-down image 310 and corresponding (multiple) displays 320 shown to the vehicle operator. In the example in the figure, the two images are shown to be close to each other, but it should be understood that the (multiple) displays 320 could be separate display monitors, separate images within a single display monitor, or any other configuration capable of displaying images. In the correct path state, the trailer 312 is reversing toward the dock 311, and ultrasonic sensors (shown in Figures 1A–2) determine the distances 314,316 between the trailer and each adjacent object 318 and the dock 311. Adjacent objects 318 could be additional trailers, cargo loading docks, loading docks, or any other structural features that could potentially collide with the trailer 312.

[0049] The ultrasonic sensor determines that the left side of the trailer is 7 feet away from an adjacent object, the right side of the trailer is 8 feet away from an adjacent object, and the rear of the trailer is 34 feet away from dock 311. Based on these determinations and the steering angle in the cab, the docking assistance system determines that the reverse operation can be continued without assistance. After determining that the operation can be continued without assistance, the docking assistance program provides (multiple) overlay displays 320 on the Class IV view on displays 18a, 18B. The overlays include distance indicators 322, 324 showing the distance from each side of the trailer to an adjacent object 318, and a dock distance indicator 326 showing the distance to dock 311. In some examples, the indicators may be displayed using color coding, where the color of the display indicates that the operation can proceed without modification.

[0050] Continuing with reference to Figure 4 and using similar figures to indicate similar elements, Figure 5 schematically illustrates an example of an incorrect but correctable path state in the same display system as Figure 4. Unlike the state in Figure 4, the incorrect but correctable state identifies that the rear left side of trailer 312 is too close to the left object 318, and the front right side of trailer 312 is too close to the right object 318. Once it is determined that a point on trailer 312 is too close to the corresponding object 318, the controller determines that the vehicle operator can correct the action by adjusting the steering to the left. This instruction is displayed to the operator via a warning prompt 330 that is prominently displayed on the screen. The illustrated warning prompt 330 is a text instruction. In another example, the warning prompt can take any form that clearly and uniquely identifies that the current path is incorrect and correctable, and identifies the vehicle action (e.g., steering adjustment) required to correct the path.

[0051] Continuing with reference to Figures 4 and 5, and using similar figures to indicate similar elements, Figure 6 shows what happens if the driver or vehicle operator fails to follow the above prompts and / or if an incorrect and uncorrectable steering action occurs. When the vehicle enters an uncorrectable state, prompt 330 changes from an instruction to correct the steering to an instruction to pull out of the elongated compartment 232 and redo the reverse operation. In some examples, the change in prompt can be further emphasized by changing the color of the prompt (e.g., from yellow to red), by visual effects (e.g., flashing), by accompanying audio signals, and / or by other means.

[0052] Continuing to refer to Figures 4, 5, and 6, it is understood that in some examples, a top-down image 310 may be generated and displayed to the vehicle operator on a separate screen during operation. In such examples, prompts 330 and information displays may also be provided to the user on the top-down image.

[0053] In some examples, the information provided via text prompts and text displays 322, 324, 330 can be supplemented with additional shaded overlays 450 showing the desired and estimated paths. Referring to Figures 4–6, Figures 7–9 show the same state, with Figure 7 corresponding to Figure 4, Figure 8 to Figure 5, and Figure 9 to Figure 6. The examples in Figures 7–9 add an additional prediction screen 410 and an additional overlay 450 on (multiple) displays 320. The additional overlay 450 provides predictions to the image of where the trailer will pass along the current path. The overlay 450 is color-coded in some cases to indicate when the trailer 312 is approaching an object (by changing to yellow) and when the trailer 312 is crossing or passing over an object (by changing to red).

[0054] Referring to all the figures above, Figure 10 shows a method 900 for operating the system described herein. First, the controller operating the system receives distance measurements from an ultrasonic sensor in step 910 “receive ultrasonic sensor data”. The distance is used by the controller to determine a set of distances between the trailer and an adjacent object in step 920 “determine distance to adjacent object”. Based on the determined set of distances, a parking action that can position the trailer correctly is determined in step 930 “determine parking action”.

[0055] After determining the parking maneuver that results in the correct positioning of the trailer, the driver assistance system determines, in the “Identify Steering Corrections” step 940, any steering corrections (if any) that would enable the trailer to follow the determined parking maneuver. The identified steering corrections are output to the display section of the camera monitoring system and displayed to the user in the “Display Steering Corrections” step 950. The displayed steering corrections may include one, more, or all of the following: distance markers, dock markers, specific steering commands, and shaded overlays. Each element is created in substeps 952, 954, 956, and 958, which are operated within the “Display Steering Corrections” step 950.

[0056] For example, Figures 4, 5, and 6 include distance indicators 322, 324, a dock distance indicator 326, and a trailer steering command 330. Similarly, examples in Figures 7, 8, and 9 include a steering command 330, a dock distance indicator 326, and a shaded overlay 450. The process is then repeated during the parking operation via an iterative loop 960.

[0057] As mentioned above regarding docking operations, it should be understood that the process and system can be used in any reverse operation, including parking, docking, reversing from a parking position, or any similar reverse operation.

[0058] While exemplary embodiments have been disclosed, those skilled in the art will recognize that certain modifications fall within the scope of the claims. Therefore, the following claims should be considered in order to determine their true scope and content.

Claims

1. A method for assisting the reverse operation of a tractor trailer, A step of determining the predicted trailer path of the trailer based at least partially on the steering angle of the tractor, The steps include: identifying the position of at least one adjacent object relative to the trailer using a set of ultrasonic sensors arranged around the trailer; The steps include determining whether the predicted path causes a portion of the trailer to intersect with the location of at least one adjacent object, and, in response to detection that a portion of the trailer is predicted to intersect with the location of at least one adjacent object without modification, identifying steering angle modifications that would allow the predicted path to be adjusted so that no portion of the trailer intersects with the location of at least one adjacent object, thereby identifying the state of the predicted path as one of correct, incorrect but correctable, and uncorrectable. The steps include displaying one of the following messages based on the predicted path status: a message indicating no path adjustment is needed, a path adjustment message, and a message indicating that path correction is not possible. Methods that include...

2. The method according to claim 1, wherein the route adjustment message includes a steering angle adjustment instruction.

3. The method according to claim 2, wherein the adjustment of the steering angle includes at least one of an icon indicating the direction and magnitude of the adjustment, and a text description of the direction and angle of the steering angle adjustment.

4. The method according to claim 3, wherein the adjustment message includes color coding to indicate the magnitude of the necessary correction.

5. The method according to claim 1, wherein the predicted trailer path is determined using a kinematic model.

6. The method according to claim 1, wherein the step of identifying the position of at least one adjacent object relative to the trailer using a set of ultrasonic sensors arranged around the trailer includes generating an ultrasonic point cloud that defines the relative position of each object adjacent to the trailer.

7. The method according to claim 6, further comprising the step of displaying distance markers indicating the shortest distance between the trailer and adjacent objects on each side of the trailer.

8. The method according to claim 6, further comprising the step of displaying a distance marker indicating the shortest distance from the rear of the trailer to an object behind the trailer.

9. The method according to claim 1, wherein one of the following messages—a message indicating no route adjustment is needed, a route adjustment message, and a message indicating that route correction is not possible—based on the predicted route status is displayed as an overlay on the mirror replacement image.

10. The method according to claim 9, wherein the mirror replacement image is an image stitched together from a rear-facing camera on the driver's side and a rear-facing camera on the passenger's side.

11. The method according to claim 9, further comprising the steps of generating a visual overlay of the predicted trailer path and applying the visual overlay to the mirror replacement image.

12. The method according to claim 11, wherein the visual overlay of the predicted trailer path includes shading of the region of the image that the trailer is expected to traverse.

13. The method according to claim 12, wherein the step of displaying the route adjustment message includes changing the shading to a first color, and the step of displaying the route correction impossible message includes changing the shading to a second color.

14. The process further includes the step of generating and displaying a top-down view of the trailer, The method according to claim 1, wherein the top-down view of the trailer includes the predicted trailer route and one of the following based on the status of the predicted route message: the no-route adjustment message, the route adjustment message, and the route correction impossible message.

15. It comprises a processor and memory, and the memory is controlled by the processor, Determining the predicted trailer path of the trailer based at least partially on the tractor's steering angle, Using a set of ultrasonic sensors positioned around the trailer, the position of at least one adjacent object relative to the trailer is identified. The process involves determining whether the predicted path causes a portion of the trailer to intersect with the location of at least one adjacent object, and, in response to detection that a portion of the trailer is predicted to intersect with the location of at least one adjacent object without modification, identifying steering angle modifications that would allow the predicted path to be adjusted so that no portion of the trailer intersects with the location of at least one adjacent object, thereby identifying the state of the predicted path as one of the following: correct, incorrect but correctable, and uncorrectable. Based on the predicted path status, one of the following messages will be displayed on the screen: a message indicating no path adjustment is needed, a message indicating path adjustment is needed, and a message indicating that path correction is not possible. A vehicle controller that stores the commands to perform certain actions.

16. The vehicle controller according to claim 15, further comprising a plurality of ultrasonic sensors arranged around the trailer, each of the ultrasonic sensors being connected to the processor so that the processor receives the sensor output of the ultrasonic sensor.

17. The vehicle controller according to claim 15, wherein the vehicle controller is a component of a camera monitoring system.

Images