Reversing and parking auxiliary system using ultrasonic detection
By using ultrasonic sensors and kinematic models in commercial vehicles, identifying and adjusting the predicted path of the trailer, the problem of collisions being difficult to avoid during reversing operation is solved, and the safety and accuracy of the operation are improved.
Patent Information
- Application Number
- CN202280101128.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-19
- Publication Date
- 2025-06-13
AI Technical Summary
In commercial vehicles, it is difficult for the prior art to accurately predict the expected path and relative position of the trailer, resulting in the difficulty of avoiding collisions with adjacent objects during reverse control.
By providing ultrasonic sensors around the tractor and trailer, the location of the adjacent object is identified and the predicted path of the trailer is determined based on the steering angle of the tractor. Use kinematic models to determine the state of the prediction path and adjust the steering angle according to the state to avoid the trailer intersecting with adjacent objects.
Accurate identification and adjustment of trailer prediction paths is achieved, safety and accuracy during reversing operation is improved, and the risk of collision with neighboring objects is reduced.
Smart Images
Figure CN120152898A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to commercial vehicle docking assistance. Background Art
[0002] In commercial vehicles, mirror replacement systems, camera systems for supplementing mirror views, and other camera monitoring systems are used to enhance the vehicle operator's ability to observe the surrounding environment. A camera monitoring system (CMS) uses one or more cameras to provide an enhanced field of view to the vehicle operator. In some examples, the mirror replacement system covers a larger field of view than a traditional mirror or includes views that cannot be fully obtained via a traditional mirror.
[0003] Driver assistance systems, semi-automatic driver assistance systems, and other vehicle systems use or require knowledge of the relative positions of the trailer and the objects the trailer is navigating past to determine the trailer's predicted path and provide warnings when the trailer is about to collide with a neighboring object. In addition to the required relative position information, information such as the direction of travel, speed of travel, and similar information about vehicle operation is needed to accurately predict the physical movement of the tractor and trailer. Summary of the Invention
[0004] An exemplary method for assisting in the backing maneuver of a tractor-trailer includes: determining a predicted trailer path of the trailer based at least in part on the steering angle of the tractor; using a set of ultrasonic sensors disposed around the trailer to identify the position of at least one neighboring object relative to the trailer; identifying the status of the predicted path as one of a correct status, an incorrect but correctable status, and an uncorrectable status by determining whether the predicted path causes a portion of the trailer to intersect the position of at least one neighboring object, and in response to detecting that a portion of the trailer is expected to intersect the position of at least one neighboring object without correction, identifying a steering angle correction that can adjust the predicted path such that no portion of the trailer intersects the position of at least one neighboring object; and displaying one of a no path adjustment message, a path adjustment message, and a path cannot be corrected message based on the status of the predicted path.
[0005] In another example of the method for assisting in the backing maneuver of a truck-trailer described above, the path adjustment message includes a steering angle adjustment instruction.
[0006] In another example of any of the methods for assisting in the backing maneuver of a truck-trailer described above, the steering angle adjustment includes at least one of: an icon indicating the adjustment direction and magnitude and a text descriptor of the direction and angle of the steering angle adjustment.
[0007] In another example of any of the methods for assisting in the backing maneuver of a truck-trailer described above, the adjustment message includes a color coding indicating the magnitude of the correction required.
[0008] In another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer, the predicted trailer path is determined using a kinematic model.
[0009] In another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer, identifying the position of at least one neighboring object relative to the trailer using a set of ultrasonic sensors disposed around the trailer includes generating an ultrasonic point cloud that defines the relative positions of each object close to the trailer.
[0010] Another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer further includes displaying distance markers that indicate the shortest distance between the trailer and neighboring objects on each side of the trailer.
[0011] Another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer further includes displaying distance markers that indicate the shortest distance from the rear of the trailer to an object behind the trailer.
[0012] In another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer, one of a no path adjustment message, a path adjustment message, and a non - correctable path message based on the state of the predicted path is displayed as an overlay on the mirror replacement image.
[0013] In another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer, the mirror replacement image is an image stitched together from a driver - side rear - facing camera and a passenger - side rear - facing camera.
[0014] Another example of any of the above methods for ensuring the reverse maneuvering of a truck trailer further includes generating a visual overlay of the predicted trailer path and applying the visual overlay to the mirror replacement image.
[0015] In another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer, the visual overlay of the predicted trailer path includes shading an image area that the trailer is expected to pass through.
[0016] In another example of any of the above methods for ensuring the reverse maneuvering of a truck trailer, displaying the path adjustment message includes changing the shading to a first color, and displaying the non - correctable path message includes changing the shading to a second color.
[0017] Another example of any of the above methods for assisting in the reverse maneuvering of a truck trailer further includes generating and displaying a top view of the trailer, where the top view of the trailer includes the predicted trailer path and one of a no path adjustment message, a path adjustment message, and a non - correctable path message based on the state of the predicted path message.
[0018] In an exemplary embodiment, a vehicle controller includes a processor and a memory that stores instructions for causing the processor to determine a predicted trailer path of a trailer based at least in part on a steering angle of a tractor; identify a position of at least one neighboring object relative to the trailer using a set of ultrasonic sensors disposed around the trailer; identify a status of the predicted path as one of a correct status, an incorrect but correctable status, and an uncorrectable status by determining whether the predicted path causes a portion of the trailer to intersect with the position of at least one neighboring object, and identify a steering angle correction that can adjust the predicted path such that no portion of the trailer intersects with the position of at least one neighboring object in response to detecting that a portion of the trailer is predicted to intersect with the position of at least one neighboring object without correction; and cause a screen to display one of a no path adjustment message, a path adjustment message, and an uncorrectable path message based on the status of the predicted path.
[0019] Another example of the vehicle controller described above further includes a plurality of ultrasonic sensors disposed around the trailer, each ultrasonic sensor being connected to the processor such that the processor receives sensor outputs of the ultrasonic sensors.
[0020] In another example of any of the vehicle controllers described above, the vehicle controller is part of a camera monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The present disclosure can be further understood with reference to the following detailed description considered in conjunction with the accompanying drawings, in which:
[0022] Figure 1A is a schematic front view of a commercial truck having a camera monitoring system (CMS) for providing at least Class II and Class IV views.
[0023] Figure 1B is a schematic top plan view of a commercial truck having a camera monitoring system for providing Class II, Class IV, Class V, and Class VI views.
[0024] Figure 2 is a schematic top perspective view of a vehicle cab including a display and an interior camera.
[0025] Figure 3 illustrates an exemplary backing maneuver of a commercial vehicle entering a loading area.
[0026] Figure 4 illustrates an exemplary backing maneuver at a correct trailer angle using a first display configuration.
[0027] Figure 5 schematically illustrates an exemplary backing maneuver at a correctable trailer angle using a first display configuration.
[0028] Figure 6 Schematically shows an exemplary reverse maneuver performed at an uncorrectable trailer angle using a first display configuration.
[0029] Figure 7 Shows an exemplary reverse maneuver performed at a correct trailer angle using a second display configuration.
[0030] Figure 7 Schematically shows an exemplary reverse maneuver performed at a correctable trailer angle using a second display configuration.
[0031] Figure 9 Schematically shows an exemplary reverse maneuver performed at an uncorrectable trailer angle using a second display configuration.
[0032] Figure 10 Shows a processor for operating a reverse assist system.
[0033] The embodiments, examples, and alternatives (including any one of their aspects or individual features) in the preceding paragraphs, claims, or the following description and drawings can be taken individually or in any combination. Features described in connection with one embodiment apply to all embodiments unless they are incompatible. Detailed Description
[0034] Figure 1A and Figure 1B Shows a schematic diagram of a commercial vehicle 10. Figure 2 Schematically shows the interior of a vehicle cab. The vehicle 10 includes a vehicle cab or tractor 12 for towing a trailer 14. Although commercial trucks are contemplated in the present disclosure, the present invention can also be applied to other types of vehicles and is not limited to commercial vehicles. The vehicle 10 includes a camera monitoring system (CMS) 15 ( Figure 2 as shown), which has a driver-side camera arm 16a and a passenger-side camera arm 16b mounted outside the vehicle cab 12. If desired, the camera arms 16a, 16b can also include conventional mirrors integrated therewith, although the CMS 15 can be used to completely replace the mirrors. In other examples, each side can include multiple camera arms, each arm accommodating one or more cameras and / or mirrors.
[0035] Each camera arm 16a, 16b includes a base fixed to, for example, the cab 12. The pivot arm is supported by the base and is pivotable relative to the base. At least one rearward camera 20a, 20b is respectively disposed within the camera arm. The external cameras 20a, 20b respectively provide an external field of view FOV EX1 , FOV EX2, each of these external fields of view includes at least one of Class II views and Class IV views( Figure 1B ), and these views are the legally required views in the commercial trucking industry. If desired, multiple cameras can also be used in each camera arm 16a, 16b to provide these views. Each arm 16a, 16b can also provide a housing that encapsulates electronic devices configured to provide various features of the CMS 15.
[0036] The first video display 18a and the second video display 18b are arranged on each of the driver side and the passenger side within the vehicle cab 12, on or near the A-pillars 19a, 19b, to display the Class II views and the Class IV views of the respective sides of the vehicle 10, and these views provide rearward side views of the vehicle 10 captured by the external cameras 20a, 20b.
[0037] If video of Class V and Class VI views is also needed, a camera housing 16c and a camera 20c can be arranged at or near the front of the vehicle 10 to provide these views( Figure 1B ). A third display 18c arranged within the cab 12 near the center of the top of the windshield can be used to display the Class V and Class VI views towards the front of the vehicle 10 to the driver.
[0038] If video of Class VIII views is needed, camera housings can be provided on the sides and rear of the vehicle 10 to provide a field of view including part or all of the Class VIII area of the vehicle 10. In such an example, the third display 18c can include one or more frames for displaying the Class VIII views. Alternatively, additional displays can be added near the first display 18a, the second display 18b, and the third display 18c, and these additional displays provide displays dedicated to providing the Class VIII views. The displays 18a, 18b, 18c face the driver area 24 within the cab 22, where the operator sits on the driver seat 26.
[0039] Trailer 14 includes a plurality of ultrasonic sensors 102, 104, 106, 108. The ultrasonic sensors 102 - 108 provide short - range point cloud detection that identifies objects adjacent to the trailer 14 and the distances between these objects and the trailer 14. These detections are provided to the camera monitoring system (CMS 15) and / or another vehicle controller in communication with the CMS 15 via any conventional sensor communication method. The CMS 15 and / or the vehicle controller in communication with the CMS 15 includes a docking assistance program that determines an ideal reverse path for backing the trailer 16 into a corresponding loading dock. In other examples, the same process can be used for other reverse maneuvers based on ultrasonic sensor position detection, and the assistance system is not limited to docking maneuvers. The docking assistance program causes prompts and / or instructions to be displayed above the view in display panels 18a, 18b, and these instructions assist the vehicle operator in correctly performing the reverse maneuver.
[0040] Continuing to refer Figures 1A to 2 , Figure 3 FIG. schematically illustrates an exemplary reverse operation 200, where a tractor 210 is backing a trailer 220 into a loading dock 230. A specific space 232 corresponding to the loading dock 230 is provided between two adjacent parked trailers 240. In an alternative environment, the trailers 240 can be replaced by any other type of obstacle, and the system operates in the same manner.
[0041] To correctly insert the trailer 220 into the loading dock 230, the controller operating the docking assistance program determines that the trailer 220 should follow path 222 and that this trailer path 222 can be achieved by moving the hitch point connecting the tractor 210 and the trailer 222 along the hitch point path 212. The controller can use any kinematic path estimation model to determine this path. Since the hitch point path 212 is fixed relative to the tractor 210, the hitch point path 212 is directly controlled by the steering of the tractor 210. In contrast, since the trailer 220 is able to rotate relative to the tractor 210 at the hitch point 214, the trailer path 222 includes some variations.
[0042] Instructions on how to follow the trailer path 222 can be provided to the vehicle operator, and these instructions can include a request for corrective action to transform from the current path to the correct path 222. In one example, the requested steering correction can be specified on the steering wheel, while in another example, the requested steering correction can be relative to the trailer orientation. In the former, the driver is commanded to turn the steering wheel to the left, which causes the trailer to turn left or right. This example is referred to as a steering wheel instruction. In the latter, the driver is commanded to turn the trailer to the left, which requires the driver to turn the steering wheel left or right. This example is referred to as a trailer steering instruction.
[0043] To assist the driver / operator of the towing vehicle 210 in correctly inserting the trailer 220 into the loading dock 230, the docking assist program uses ultrasonic sensors 202, 204, 206, 208 disposed on the sides and ends of the trailer 220 to determine the distance 250 between the sensors 202 - 208 and an object (such as trailer 240) adjacent to the slot 232 where the trailer 220 is to be parked. This information is used to create a point cloud that identifies the position of the adjacent object relative to the trailer 220. In addition to the information from the ultrasonic sensors, the controller operating the docking assist program also receives operating information from a steering angle sensor and other vehicle information obtained through a connection to the vehicle controller. The received information is used in conjunction with a kinematic model to determine whether the current operation will keep the trailer 220 along the trailer path 222. If the operating parameters are incorrect, the driver determines the corrective measures and provides them to the operator via one or more displays 18a, 18b, 18c in the cab.
[0044] During a reverse docking maneuver, such as a maneuver along path 222, the vehicle can be in one of three possible states: a correct path, an incorrect but correctable path, and an incorrect and uncorrectable path.
[0045] A correct path is a state in which the relative positions of the trailer 220 and the adjacent object 240, combined with the current steering angle of the cab, form a reverse path that correctly inserts the trailer 220 into the corresponding dock 230.
[0046] An incorrect but correctable path is a state in which the relative positions of the trailer 220 and the adjacent object 240, combined with the current steering angle, form a reverse path that will collide with one of the adjacent objects 240, but the path can be corrected via a steering angle correction or via another maneuver while maintaining the reverse maneuver.
[0047] An incorrect and uncorrectable path is a state in which continuing the reverse maneuver will result in a collision with one or both of the adjacent objects 240 and there is no steering correction that can correct the path. In this state, the docking assist program instructs the driver to exit the lane and start the maneuver over.
[0048] Continuing with reference to Figures 1A to 3 , Figure 4 An example correct path state is schematically illustrated via an overhead image 310 and a corresponding display 320 presented to the vehicle operator. Although two images are shown close to each other in the example figure, it should be understood that the display 320 can be a separate display monitor, separate images within a single display monitor, or any other configuration capable of displaying images. In the correct path state, the trailer 312 reverses into the dock 311, and the ultrasonic sensors ( Figures 1A to 2Determine the distances 314, 316 between the trailer and each neighboring object 318 and the docking station 311 (as shown). The neighboring object 318 can be another trailer, a loading area, a loading dock, or any other structural feature that may collide with the trailer 312.
[0049] The ultrasonic sensors determine that the left side of the trailer is 7 feet away from the neighboring object, the right side of the trailer is 8 feet away from the neighboring object, and the rear of the trailer is 34 feet away from the docking station 311. Based on these determinations and the steering angle of the cab, the docking assist system determines that the reverse maneuver can continue without assistance. After determining that the maneuver can continue without assistance, the docking assist program provides an overlay display 320 on the Class IV view on the displays 18a, 18B. The overlay includes distance indicators 322, 324 indicating the distances from each side of the trailer to the neighboring object 318 and a docking station distance indicator 326 indicating the distance to the docking station 311. In some examples, color coding can be used to display the indicators, where the color shown indicates that the maneuver can proceed without change.
[0050] Continuing to refer to Figure 4 where like numerals represent like elements, Figure 5 schematically shows an exemplary incorrect but correctable path state in the same display system as Figure 4 In contrast to the state of Figure 4 this incorrect but correctable state identifies the left rear side of the trailer 312 as being too close to the left object 318 and the right front side of the trailer 312 as being too close to the right object 318. Once it is determined that a point on the trailer 312 is too close to the corresponding object 318, the controller determines that the vehicle operator can correct the operation by adjusting the steering to turn left. This instruction is displayed to the operator via a warning prompt 330 highlighted on the overlay screen. The illustrated warning prompt 330 is a text instruction. In alternative examples, the warning prompt can take any form that clearly and unambiguously identifies that the current path is incorrect and correctable and identifies the vehicle operation (e.g., steering adjustment) required to correct the path.
[0051] Continuing to refer to Figure 4 and Figure 5 where like numerals are used to represent like elements, Figure 6Illustrates what happens when the driver or vehicle operator fails to follow the above prompts and / or otherwise causes incorrect and uncorrectable maneuvering operations. When the vehicle enters an uncorrectable state, the prompt 330 changes from an instruction to correct the maneuver to an instruction to drive out of the gap 232 and restart the reverse maneuver. In some examples, the change in the prompt can be further emphasized by changing the color of the prompt (e.g., from yellow to red), by a visual effect (e.g., flashing), by an accompanying audio signal, and / or by any other means.
[0052] Continuing to refer to Figure 4 、 Figure 5 and Figure 6 , it can be understood that in some examples, an overhead image 310 can be generated on a separate screen and displayed to the vehicle operator during operation. In such examples, the prompt 330 and the information display can also be provided to the user on the overhead image.
[0053] In some cases, an additional shaded overlay 450 showing the desired path and the projected path can be used to supplement the information provided by the text prompts and text displays 322, 324, 330. Referring to Figures 4 to 6 , Figures 7 to 9 shows the same state, where Figure 7 corresponds to Figure 4 , Figure 8 corresponds to Figure 5 , Figure 9 corresponds to Figure 6 。 Figures 7 to 9 's example adds an additional projection screen 410 and an additional overlay 450 to the display 320. The additional overlay 450 provides a projection in the image of the position where the trailer will pass on the current path. In some cases, the overlay 450 uses color coding to indicate when the trailer 312 will approach an object (by turning yellow) and when the trailer 312 will intersect or pass through the object (by turning red).
[0054] Referring to all the previous figures, Figure 10 illustrates a method 900 for operating the system described herein. Initially, the controller of the operating system receives distance measurements from the ultrasonic sensors in the "Receive ultrasonic sensor data" step 910. In the "Determine distance to neighboring object" step 920, the controller uses this distance to determine a set of distances between the trailer and neighboring objects. Based on the determined set of distances, in the "Determine parking operation" step 930, a parking operation that can place the trailer in the correct position is determined.
[0055] After determining the parking operation that will result in the correct placement of the trailer, the driver assistance system will determine the steering correction (if any) that will allow the trailer to follow the determined parking operation in step 940, "Identify Steering Correction". The identified steering correction is output to the display portion of the camera monitoring system and is displayed to the user in step 950, "Display Steering Correction". The displayed steering correction may include one, more, or all of distance markers, docking markers, specific steering instructions, and a shadow overlay. Each element is created in sub-steps 952, 954, 956, 958 that operate within the Display Steering Correction step 950.
[0056] For example, Figure 4 , Figure 5 and Figure 6 include distance indicators 322, 324, docking distance indicator 326, and trailer steering instruction 330. Similarly, Figure 7 , Figure 8 and Figure 9 examples of include steering wheel instruction 330, docking distance indicator 326, and shadow overlay 450. Then, this process is repeated during the parking operation by repeating loop 960.
[0057] Although described above for a docking maneuver, it should be understood that the process and system can be used for any reverse maneuver, including parking, docking, backing out of a parking position, or any similar reverse maneuver.
[0058] Although example embodiments have been disclosed, those of ordinary skill in the art will recognize that certain modifications will fall within the scope of the claims. Accordingly, the appended claims should be studied to determine their true scope and content.
Claims
1. A method for assisting in the reverse maneuver of a tractor-trailer, which comprises: Determining a predicted trailer path of the trailer based at least in part on the steering angle of the tractor; Identifying the position of at least one neighboring object relative to the trailer using a set of ultrasonic sensors disposed around the trailer; Identifying the state of the predicted path as one of a correct state, an incorrect but correctable state, and an uncorrectable state by determining whether the predicted path causes a part of the trailer to intersect the position of the at least one neighboring object, and in response to detecting that a part of the trailer is expected to intersect the position of the at least one neighboring object without correction, identifying a steering angle correction that can adjust the predicted path so that no part of the trailer intersects the position of the at least one neighboring object; And Displaying one of a no path adjustment message, a path adjustment message, and an uncorrectable path message based on the state of the predicted path.
2. The method according to claim 1, wherein the path adjustment message includes a steering angle adjustment instruction.
3. The method according to claim 2, wherein the steering angle adjustment includes at least one of: an icon indicating the adjustment direction and amplitude and a text descriptor of the direction and angle of the steering angle adjustment.
4. The method according to claim 3, wherein the adjustment message includes a color coding indicating the amplitude of the required correction.
5. The method according to claim 1, wherein the predicted trailer path is determined using a kinematic model.
6. The method according to claim 1, wherein identifying the position of at least one neighboring object relative to the trailer using a set of ultrasonic sensors disposed around the trailer includes generating an ultrasonic point cloud that defines the relative position of each object near the trailer.
7. The method according to claim 6, further comprising displaying distance markers that indicate the shortest distance between the trailer and neighboring objects on each side of the trailer.
8. The method according to claim 6, further comprising displaying distance markers that indicate the shortest distance from the rear of the trailer to an object behind the trailer.
9. The method according to claim 1, wherein one of a no path adjustment message, a path adjustment message, and an uncorrectable path message based on the state of the predicted path is displayed as an overlay on top of a mirror replacement image.
10. The method according to claim 9, wherein the mirror replacement image is an image stitched together from a driver-side rear-facing camera and a passenger-side rear-facing camera.
11. The method according to claim 9, further comprising generating a visual overlay of the predicted trailer path and applying the visual overlay to the mirror replacement image.
12. The method according to claim 11, wherein the visual overlay of the predicted trailer path includes shading an image area that the trailer is expected to pass through.
13. The method according to claim 12, wherein displaying the path adjustment message includes changing the shade to a first color, and displaying the uncorrectable path message includes changing the shade to a second color.
14. The method according to claim 1, further comprising generating and displaying a top view of the trailer, wherein the top view of the trailer includes the predicted trailer path and one of the no path adjustment message, the path adjustment message, and the uncorrectable path message based on the state of the predicted path message.
15. A vehicle controller, which comprises: a processor and a memory, the memory storing instructions for causing the processor to determine a predicted trailer path of the trailer based at least in part on a steering angle of a towing vehicle; identifying a position of at least one neighboring object relative to the trailer using a set of ultrasonic sensors disposed around the trailer; identifying a state of the predicted path as one of a correct state, an incorrect but correctable state, and an uncorrectable state by determining whether the predicted path causes a portion of the trailer to intersect the position of the at least one neighboring object, and identifying a steering angle correction that can adjust the predicted path so that no portion of the trailer intersects the position of the at least one neighboring object in response to detecting that a portion of the trailer is expected to intersect the position of the at least one neighboring object without correction; and causing a screen to display one of a no path adjustment message, a path adjustment message, and an uncorrectable path message based on the state of the predicted path.
16. The vehicle controller according to claim 15, further comprising a plurality of ultrasonic sensors disposed around the trailer, each ultrasonic sensor connected to the processor such that the processor receives sensor outputs of the ultrasonic sensors.
17. The vehicle controller according to claim 15, wherein the vehicle controller is a component of a camera monitoring system.