METHOD AND SYSTEMS FOR ALIGNING A VEHICLE ONTO A TRAILER

Abstract

System (200) for aligning a vehicle (100) with a trailer (105), wherein the system (200) comprises: an image sensor (215) designed to collect multiple images; and an electronic processor (220) that is set up: to receive the multiple images from the image sensor (215), to determine a three-dimensional model of an area around the vehicle (100) using the multiple images, for automatically performing a vehicle maneuver to align the vehicle (100) with the trailer (105) based on the three-dimensional model of the area around the vehicle (100), to control the vehicle (100), to determine when the vehicle (100) is aligned with the trailer (105), to prevent the vehicle (100) from automatically performing the vehicle maneuver when the vehicle (100) is aligned with the trailer (105), Based on the three-dimensional model and vehicle odometry information acquired by at least one other vehicle sensor, one or more objects can be identified within the multiple images, and to determine whether the vehicle (100) is located within a starting area with respect to the trailer (105).

Description

FIELD OF INVENTION

[0001] Embodiments relate to methods and systems for aligning a vehicle with a trailer. BRIEF PRESENTATION

[0002] Backing a vehicle up to a trailer can be a tedious process for a driver, as it requires precise alignment of the vehicle's tow ball under a trailer hitch. Existing and proposed methods for trailer hitch guidance using a rearview camera rely either on the driver selecting the trailer hitch location in an image or on using a classifier to identify the trailer hitch. The first method may perform poorly because the driver does not select the correct location in the image. The second method fails if the trailer hitch is not detected by the classifier or is not correctly located in the image. For example, there are a number of distinct trailer hitch types, as well as a wide variation in trailer hitch appearance.A classifier approach may not be able to identify all variations of trailer hitches or may require significant computational power and large training datasets. Furthermore, the location of the trailer hitch in the image does not determine the distance to the hitch or the height of the hitch. Additional methods must be applied to obtain the distance for proper alignment and to check for a potential collision with the tow ball due to a difference in height. Methods and systems for aligning a vehicle with a trailer are disclosed, for example, in German patent documents DE 10 2004 029 130 A1, DE 11 2014 004 554 T5, DE 103 02 545 A1, DE 101 11 529 A1, and DE 10 2004 029 129 B4.

[0003] The above-described object is achieved according to the invention with the features of the independent patent claims. Advantageous developments of the invention are the subject of the dependent claims. Accordingly, one embodiment provides a system for aligning a vehicle with a trailer according to claim 1.

[0004] A further embodiment provides a method for aligning a vehicle with a trailer according to claim 9.

[0005] Other aspects of various embodiments will become apparent upon consideration of the dependent claims, the detailed description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a vehicle aligned with a trailer. Fig. 2 shows a system for aligning a vehicle with a trailer according to some embodiments. Fig. 3 is a flowchart showing a method for aligning a vehicle with a trailer performed by the system of Fig. 2 according to some embodiments. DETAILED DESCRIPTION

[0006] One or more embodiments are described and illustrated in the following description and the accompanying drawings. These embodiments are not limited to the specific details provided herein and may be modified in various ways. Moreover, other embodiments may exist that are not described herein. Furthermore, functionality described herein as being performed by one component may be performed in a distributed manner by multiple components. Likewise, functionality performed by multiple components may be consolidated and performed by a single component. Similarly, a component described as performing a particular functionality may also perform additional functionality not described herein.For example, a device or structure "configured" in a certain way is configured in at least that way, but may also be configured in ways not listed. Furthermore, some embodiments described herein may include one or more electronic processors configured to perform the described functionality by executing instructions stored on a non-transitory computer-readable medium. Similarly, embodiments described herein may be implemented as a non-transitory computer-readable medium storing instructions executable by one or more electronic processors to perform the described functionality.

[0007] Furthermore, the phraseology and terminology used herein are for the purpose of description and should not be considered limiting. For example, the use of "including," "containing," "comprising," "with," and variations thereof is intended to include the items listed thereafter and equivalents thereof, as well as additional items. The terms "connected" and "coupled" are used broadly and include both direct and indirect connecting and coupling. Furthermore, "connected" and "coupled" are not limited to physical or mechanical connections or couplings and can include electrical connections or couplings, both direct and indirect.Furthermore, electronic communications and notifications may be conducted using wired connections, wireless connections, or a combination thereof, and may be transmitted directly or through one or more intermediary devices over various types of networks, communication channels, and connections. Furthermore, reference terms such as first and second, upper and lower, and the like may be used herein merely to distinguish one entity or act from another entity or act, without necessarily requiring or implying any actual such relationship or order between such entities or acts.

[0008] Fig. 1 illustrates a vehicle 100 and a trailer 105. In the example shown, the vehicle 100 is aligned with the trailer 105 such that a tow ball 110 of the vehicle is directly below a trailer hitch 115 of the trailer 105. Accordingly, when the trailer 105 is lowered, the trailer hitch 115 receives the tow ball 110. As described above, reversing the vehicle 100 toward the trailer 105 can be a tedious process for a driver because it requires precise alignment of the tow ball 110 of the vehicle 100 below the trailer hitch 115.

[0009] Fig. 2 shows a system 200 for aligning the vehicle 100 with the trailer 105 according to some embodiments. As shown in Fig. 2, the system 200 includes a controller 205, a human-machine interface 210, and an image sensor 215. In some embodiments, the system 200 includes fewer, additional, or different components than those in Fig. 2 in various configurations and may perform additional functionality to that described herein.

[0010] Controller 205 may be a pre-existing controller of vehicle 105. For example, controller 205 may be a controller associated with a braking system, an engine control system, an autonomous vehicle system, or another vehicle system of vehicle 100. However, in some embodiments, controller 205 is separate from other pre-existing vehicle systems. For example, controller 205 may be included in a complementary system (e.g., system 200) for a pre-existing vehicle system. Accordingly, in such embodiments (e.g., system 200), controller 205 functions as a central location that collects information (e.g., multiple images, vehicle odometry information, and the like) from various sources such as image sensor 215, one or more additional vehicle sensors, another vehicle system, or a combination thereof.Based on the collected information, the controller 205 may transmit control signals to the other existing vehicle systems to control the vehicle 100 to perform a vehicle maneuver, such as a vehicle maneuver to align the vehicle 100 with the trailer 105.

[0011] In the Fig. 2, the controller 205 includes an electronic processor 220 (e.g., a microprocessor, an application-specific integrated circuit, or other suitable electronic device), a memory 225 (e.g., one or more non-transitory computer-readable storage media), and an input / output interface 230. The electronic processor 220, the memory 225, and the input / output interface 230 communicate via one or more data links or buses, or a combination thereof. Fig. The controller 205 shown in Figure 2 is an example, and in some embodiments, the controller 205 includes fewer, additional, or different components in various configurations than in Fig. 2. Furthermore, in some embodiments, the controller 205 performs functionality in addition to the functionality described herein.

[0012] The electronic processor 220 is configured to retrieve instructions from the memory 225 and execute instructions to perform a set of functions, including the methods described herein. For example, in some embodiments, the electronic processor 220 executes instructions to control vehicle steering, braking, and propulsion systems to align the vehicle 100 with the trailer 105. The memory 225 may include combinations of various types of memory, such as read-only memory ("ROM"), random access memory ("RAM"), or any other non-transitory computer-readable medium. As noted above, the memory 225 stores instructions executed by the electronic processor 220. The memory 225 may also store data, such as multiple images collected by the image sensor 215.The memory 225 may also store firmware, one or more applications, program data, filters, rules, one or more program modules, and other executable instructions or data.

[0013] The input / output interface 230 allows the controller 205 to communicate with devices external to the controller 205 (e.g., directly or indirectly receiving input from and providing output to devices external to the controller 205). In one example, the controller 205 communicates with the human-machine interface 210, the image sensor 215, or a combination thereof through the input / output interface 230. In some embodiments, the input / output interface 230 includes a port for receiving a wired connection to the human-machine interface 210, the image sensor 215, or a combination thereof. Alternatively or additionally, the input / output interface 230 includes a transceiver for establishing a wireless connection to the human-machine interface 210, the image sensor 215, or a combination thereof.Alternatively or additionally, the input / output interface 230 communicates with a communication bus (e.g., a CAN (Controller Area Network)) to communicate indirectly, for example, with the human-machine interface 210, the image sensor 215, or a combination thereof.

[0014] As noted above, the system 200 includes an image sensor 215, for example, a video camera. In some embodiments, the image sensor 215 is a pre-existing rearview camera of the vehicle 100. The image sensor 215 detects a plurality of images associated with an area surrounding the vehicle 100. If the image sensor is a rearview camera, the image sensor 215 is mounted at the rear of the vehicle 100 and has a rearward-facing field of view relative to the vehicle 100. Accordingly, the image sensor 215 is configured to detect a plurality of images associated with an area directly behind the vehicle 100.

[0015] In some embodiments, the image sensor 215 includes a wide-angle lens. In such a configuration, the image sensor 215 may include image processing circuitry to convert wide-angle or omnidirectional views into rectangular images. In other embodiments, the image sensor 215 is a stereo camera that captures three-dimensional information of a scene behind the vehicle 100. In both of these embodiments, the image sensor 215 is configured to track objects within the multiple images and to transmit information regarding the position and movement of the objects within the multiple images (via the input / output interface 230) to the electronic processor 220. In other embodiments, the electronic processor 220 performs image processing, object tracking, object detection, and scene mapping in conjunction with the image sensor 215.Alternatively or additionally, the image sensor 215 operates in conjunction with other sensors, such as a ToF (Time of Flight) camera, a lidar sensor, a radar sensor, an ultrasonic detector, or a combination thereof, to determine a three-dimensional model of an area around the vehicle 100 (as described in more detail below).

[0016] The system 200 also includes a human-machine interface. The human-machine interface 210 is configured to receive input from and provide output to a user of the vehicle 100. The human-machine interface 210 includes, for example, a display, such as a touchscreen display, that displays the multiple images detected by the image sensor 215 to a user of the vehicle 100. In some embodiments, the human-machine interface 210 also displays additional information to the user of the vehicle 100, such as instructions for aligning the vehicle 100 with the trailer 105, a visual representation of a launch area relative to the trailer 105 (as described in more detail below), and the like.The human-machine interface 210 also includes one or more input mechanisms, such as one or more user-selectable buttons, knobs, sliders, and the like, for receiving input from the user of the vehicle 100. For example, a user of the vehicle 100 may use the one or more input mechanisms of the human-machine interface 210 to input a request to initiate an alignment of the vehicle 100 with the trailer 105. In some embodiments, the human-machine interface 210 is a pre-existing component of the vehicle 100. The human-machine interface 210 may, for example, be part of the infotainment system, a navigation system, or another system of the vehicle 100.

[0017] As noted above, in some embodiments, the system 200 includes additional or different components than in Fig. 2. For example, the system 200 may include one or more additional vehicle sensors associated with the vehicle 100, such as a steering angle sensor, a wheel position sensor, a wheel speed sensor, a GPS (Global Positioning System) indicator, and the like. The one or more additional vehicle sensors may be configured to provide position and motion information (e.g., vehicle odometry information) regarding the vehicle 100. The one or more additional sensors provide the vehicle odometry information to the controller 205. Based on the vehicle odometry information received from the one or more additional sensors, the electronic processor 220 of the controller 205 may determine a position, heading, and motion of the vehicle 100.Accordingly, electronic processor 205 may be configured to determine the position and a change in position over time based on the vehicle odometry information. Based at least in part on such determinations, electronic processor 220 may determine a travel path of vehicle 100 and movements of objects relative to vehicle 100 in a three-dimensional model of an area around vehicle 100.

[0018] As noted above, the electronic processor 220 of the controller 205 executes instructions for aligning the vehicle 100 with the trailer 105. In particular, the electronic processor 205 executes instructions for performing the Fig. 3 for aligning the vehicle 100 with the trailer 105. As shown in Fig. 3, the method 300 includes receiving, with the electronic processor 220, a plurality of images from the image sensor 215 (at block 305). The electronic processor 220 receives the plurality of images from the image sensor 215 via the input / output interface 230 of the controller 205. As noted above, the plurality of images are associated with an area around the vehicle 100.

[0019] The electronic processor 220 determines whether the vehicle 100 is in a starting area with respect to the trailer 105 (at block 310). The starting area represents an area where the vehicle 100 is close enough to the trailer 105 to begin aligning the vehicle 100 with the trailer 105. In other words, in some embodiments, the electronic processor is configured to align the vehicle 100 with the trailer 105 only when the vehicle 100 is within a predetermined range of the trailer 105. For example, the electronic processor 220 aligns the vehicle 100 when the vehicle 100 (e.g., the trailer hitch 110) is within a distance of approximately one to five meters and at approximately + / - 20 degrees of a centerline of the vehicle 100.

[0020] In some embodiments, the electronic processor 220 generates a visual representation of the launch area and transmits the visual representation of the launch area to the human-machine interface 210 for display to a user of the vehicle 100. For example, the human-machine interface 210 displays the visual representation as a highlighted area overlaid on top of the multiple images displayed to the user of the vehicle 100. As another example, the human-machine interface 210 displays the visual representation as a cone-shaped area defined by one to five meters at + / - 20 degrees. Based on whether the vehicle 100 is within the launch area, the electronic processor 220 can modify the visual representation of the launch area.In one example, when the vehicle 100 is within the launch area, the visual representation of the launch area is displayed in a first color, such as green. When the vehicle is not within the launch area, the visual representation of the launch area is displayed in a second color, such as red. In some embodiments, the electronic processor generates and transmits instructions to a user of the vehicle 100 via the human-machine interface 210 for positioning the vehicle 100 within the launch area. In one example, the human-machine interface 210 displays the instructions to turn a steering wheel of the vehicle 100 clockwise or counterclockwise or to continue driving the vehicle 100 back toward the trailer 105.

[0021] If the electronic processor 220 determines that the vehicle 100 is within the starting area, the electronic processor 220 then determines a three-dimensional model of an area around the vehicle 100 using the multiple images (at block 315). In some embodiments, the three-dimensional model of the area around the vehicle 100 is a three-dimensional point cloud. The electronic processor 220 may determine the three-dimensional model of the area around the vehicle 100 by analyzing the multiple images using a structure-from-motion technique. The structure-from-motion techniques match features from the multiple images taken from different locations to determine a three-dimensional location of a feature (e.g., the trailer 105) as well as camera motion.In some embodiments, the electronic processor 220 identifies one or more objects within the plurality of images based on the three-dimensional model and the vehicle odometry information.

[0022] The electronic processor 220 controls the vehicle 100 to automatically perform a vehicle maneuver based on the three-dimensional model of the area around the vehicle 100 (at block 320). The electronic processor 220 controls the vehicle by controlling vehicle steering, acceleration, braking, or a combination thereof to align the vehicle 100 with the trailer 105. For example, the electronic processor 220 may control the steering and acceleration of the vehicle 100 to align the vehicle 100 with the trailer 105.

[0023] In some embodiments, the electronic processor 220 is also configured to process a manual orientation input by the user of the vehicle 100. The manual orientation input by the user of the vehicle 100 may include a manual steering input, a manual acceleration input, a manual braking input, or a combination thereof. When the electronic processor 220 receives a manual orientation input from the user of the vehicle 100, the electronic processor 220 controls the vehicle 100 to execute the vehicle maneuver based on the three-dimensional model and the manual orientation input. In other words, a user of the vehicle 100 can override the vehicle maneuvers controlled by the electronic processor 220. For example, the user of the vehicle 100 may control the braking and acceleration of the vehicle 100 while the steering of the vehicle 100 is automatically controlled (by the electronic processor 220).

[0024] In some embodiments, the electronic processor 220 identifies one or more features or objects within the three-dimensional model. In particular, in some embodiments, the electronic processor 220 assumes that a closest object to the vehicle is the trailer 105 (e.g., the trailer hitch 115). Based on this assumption, the electronic processor 200 controls the vehicle 100 to perform a vehicle maneuver to align the vehicle 100 (e.g., the tow ball 110) with the trailer 105 (e.g., the trailer hitch 115), as shown in Fig. 1 shown.

[0025] While the vehicle 100 is controlled to automatically perform the vehicle maneuver, the electronic processor 220 receives additional images from the image sensor 215 and updates the three-dimensional model based on the additional images received from the image sensor 215. Based on the updated three-dimensional model, the electronic processor 220 modifies or alters the vehicle maneuver performed by the vehicle 100.

[0026] The electronic processor 220 also determines when the vehicle 100 is aligned with the trailer 105 (at block 325). For example, the electronic processor 220 determines that the vehicle 100 is aligned with the trailer 105 (as shown in Fig. 1), based on the updated three-dimensional model. The electronic processor 220 prevents the vehicle 100 from automatically performing the vehicle maneuver when the vehicle 100 is aligned with the trailer (at block 330).

[0027] Alternatively or additionally, the electronic processor 220 receives from the human-machine interface 210 a user input initiating alignment of the vehicle 100 with the trailer 105 prior to receiving the plurality of images from the image sensor 215. In other words, a user indicates a request to initiate alignment of the vehicle 100 with the trailer 105 using the human-machine interface 210. In response to receiving the user input to initiate alignment of the vehicle 100 with the trailer 105, the electronic processor 220 initiates alignment of the vehicle 100 with the trailer 105. In some embodiments, the electronic processor 220 provides an indication to the human-machine interface 210 that alignment of the vehicle 100 with the trailer 105 is initiated.The human-machine interface 210 displays the indication that the alignment of the vehicle 100 with the trailer 105 is initiated to a user of the vehicle 100 (for example, via a display of the human-machine interface 210).

[0028] Alternatively or additionally, the electronic processor 220 detects when a new object (e.g., a person) is present in the multiple images based on the continuously updated three-dimensional model. When the electronic processor 220 detects a new object, the electronic processor 220 generates and transmits a new object warning to the vehicle user (via the human-machine interface 210). Alternatively or additionally, the electronic processor automatically prevents the vehicle 100 from automatically performing the vehicle maneuver when a new object is identified in the multiple images.

[0029] In some embodiments, the electronic processor 220 determines a height of the trailer hitch 115. Based on the height of the trailer hitch 115 and the height of the hitch 110, the electronic processor determines whether the hitch 110 of the vehicle 100 will collide with the trailer hitch 115. If the electronic processor 220 determines that the hitch 110 will collide with the trailer hitch 115, the electronic processor 220 generates a collision warning for the user of the vehicle 100 for display to the user of the vehicle 100 via the human-machine interface 210.

[0030] Various features and advantages of certain embodiments are set forth in the following claims.

Claims

[1] A system (200) for aligning a vehicle (100) with a trailer (105), the system (200) comprising: an image sensor (215) configured to collect a plurality of images; and an electronic processor (220) configured to: to receive the multiple images from the image sensor (215), using the plurality of images to determine a three-dimensional model of an area around the vehicle (100), for automatically performing a vehicle maneuver to align the vehicle (100) with the trailer (105) based on the three-dimensional model of the area around the vehicle (100), to control the vehicle (100), to determine when the vehicle (100) is aligned with the trailer (105), to prevent the vehicle (100) from automatically performing the vehicle maneuver when the vehicle (100) is aligned with the trailer (105), to identify one or more objects within the plurality of images based on the three-dimensional model and vehicle odometry information acquired by at least one further vehicle sensor, and to determine whether the vehicle (100) is within a starting area with respect to the trailer (105). [2] The system (200) of claim 1, wherein the electronic processor (220) is further configured to generate a visual representation of the launch area for display to a user of the vehicle (100) via a human-machine interface (210), the visual representation being a highlighted area overlaid on the plurality of images. [3] The system (200) of claim 2, wherein the electronic processor (220) is further configured to modify the visual representation to indicate whether the vehicle (100) is within the launch area. [4] The system (200) of claim 1, wherein, when the vehicle (100) is not within the launch area with respect to the trailer (105), the electronic processor (220) is further configured to generate instructions for display to a user of the vehicle (100) via a human-machine interface (210) for positioning the vehicle (100) within the launch area. [5] The system (200) of claim 1, wherein the electronic processor (220) is further configured to control the vehicle (100) to align the vehicle (100) with the trailer (105) by controlling at least one selected from a group consisting of steering, accelerating, and braking the vehicle (100). [6] The system (200) of claim 1, wherein the three-dimensional model of the area around the vehicle (100) is a three-dimensional point cloud. [7] The system (200) of claim 1, wherein the electronic processor (220) is further configured to determine the three-dimensional model of the area around the vehicle (100) by analyzing the plurality of images using a structure-from-motion technique. [8] The system (200) of claim 1, wherein the electronic processor (220) is further configured to continuously update the three-dimensional model while controlling the vehicle (100) to automatically perform the vehicle maneuver. [9] A method for aligning a vehicle (100) with a trailer (105), the method comprising: Receiving, with an electronic processor (220), a plurality of images from an image sensor (215); Determining, with the electronic processor (220), whether the vehicle (100) is within a launch area with respect to the trailer (105); Determining, with the electronic processor (220), a three-dimensional model of an area around the vehicle (100) using the plurality of images; Controlling, with the electronic processor (220), the vehicle (100) to automatically perform a vehicle maneuver based on the three-dimensional model of the area around the vehicle (100); Determining, with the electronic processor (220), when the vehicle (100) is aligned with the trailer (105); Preventing, with the electronic processor (220), the vehicle (100) from automatically performing the vehicle maneuver when the vehicle (100) is aligned with the trailer (105); Identifying one or more objects within the plurality of images based on the three-dimensional model and vehicle odometry information acquired by at least one other vehicle sensor. [10] The method of claim 9, further comprising: Receiving, from a human-machine interface (210), a user input that initiates an alignment of the vehicle (100) with the trailer (105); and Delivering, to the human-machine interface (210), an indication that alignment of the vehicle (100) with the trailer (105) has been initiated. [11] The method of claim 9, further comprising: continuously updating the three-dimensional model while the vehicle (100) is controlled to automatically perform the vehicle maneuver. [12] The method of claim 11, further comprising: Detect when a new object is present in the multiple images based on the constantly updated three-dimensional model. [13] The method of claim 12, further comprising: Generating a new object warning for a user of the vehicle (100) when a new object is identified in the plurality of images. [14] The method of claim 12, further comprising: automatically preventing the vehicle (100) from automatically performing the vehicle maneuver when a new object is identified in the plurality of images. [15] The method of claim 9, further comprising: Receiving a manual alignment input from a user of the vehicle (100), wherein the manual alignment input includes at least one selected from a group consisting of a manual steering input, a manual acceleration input, and a manual braking input. [16] The method of claim 15, wherein controlling the vehicle (100) to automatically perform the vehicle maneuver includes controlling the vehicle (100) to automatically perform the vehicle maneuver based on the three-dimensional model and the manual orientation input. [17] The method of claim 9, further comprising: Determining a height of a trailer coupling (115) of the trailer (105) and a height of a trailer coupling (110) of the vehicle (100); Determining whether the trailer coupling (110) will collide with the trailer hitch (115) based on the height of the trailer hitch (115) and the height of the trailer coupling (110); and Generating a collision warning for a user of the vehicle (100) when the trailer coupling (110) is about to collide with the trailer hitch (115).

Images