Autonomous Vehicle Trailer Hitch Coupling System

Abstract

A method and system for locating and tracking trailer couplers for autonomous vehicle driving is disclosed. The system converts images from a vehicle camera into a depth map that includes a number of points indicating distances between objects in the image and reference points. The system uses the depth map to identify and track the trailer couplers.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Non-Provisional Application No. 17 / 645,578, entitled “Autonomous Vehicle Trailer Hitch Coupling System,” filed December 22, 2021, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to methods and systems for identifying and tracking trailer hitch couplers to enable autonomous and / or semi-autonomous coupling. [Background technology]

[0003] Autonomously driven or assisted vehicles utilize maps to define the vehicle's path. Autonomous driving assists or completely replaces the driver in required movements and is particularly advantageous for vehicle maneuvers that are difficult for a driver, such as parallel parking and aligning a trailer hitch coupler. Current vehicles generate large amounts of information from a variety of sensor systems. Efficiently processing useful information can be challenging. Automotive parts suppliers and manufacturers are constantly searching for improvements that will increase customer satisfaction and vehicle operation. Summary of the Invention [Problem to be solved by the invention]

[0004] The discussion of the background art provided herein is intended to provide a general context for the present disclosure. To the extent that it is described in this background art section, no work of the present inventors, nor any aspect of the description that may not otherwise be considered prior art at the time of filing, is admitted, expressly or implicitly, to be prior art to the present disclosure. [Means for solving the problem]

[0005] In a method for locating and tracking a trailer coupler for autonomous vehicle operation, a method according to a disclosed example embodiment includes, among other possible things, converting an image from a vehicle camera into a depth map, where the depth points include a plurality of points indicating distances between objects in the image and a reference point; identifying the trailer coupler as the point in the revised depth map that is closest to the reference point; and communicating the location of the identified coupler to a vehicle operation control system.

[0006] Another exemplary embodiment of the above method further includes selecting a region of interest within the image from the vehicle camera and creating a depth map object within the selected region of interest.

[0007] In another exemplary embodiment of any of the aforementioned methods, the region of interest is smaller than the image from the vehicle camera and includes the trailer coupler.

[0008] In another exemplary embodiment of any of the foregoing methods, the region of interest is selected automatically by an algorithm executed by a controller in the vehicle.

[0009] Another example embodiment of any of the aforementioned methods further includes removing from the depth map points in the depth map that indicate the ground surface.

[0010] Another exemplary embodiment of any of the aforementioned methods further includes determining a path for the coupler based on the depth map.

[0011] Another example embodiment of any of the aforementioned methods further includes determining a pose of the camera based on information from at least one sensor system in the vehicle indicative of movement of the vehicle.

[0012] Another exemplary embodiment of any of the aforementioned methods further includes generating a dynamic model of vehicle movement based on information from at least one sensor system of the vehicle, and determining a camera pose based on the dynamic model.

[0013] In another exemplary embodiment of any of the aforementioned methods, the at least one sensor system comprises at least one of an acceleration sensor, a wheel angle sensor, an inertial measurement unit, or a global positioning system.

[0014] In another exemplary embodiment of any of the aforementioned methods, the vehicle camera comprises a mono camera.

[0015] Another exemplary embodiment of any of the foregoing methods further includes creating the depth map in real time by a control device in the vehicle.

[0016] Another exemplary embodiment of any of the foregoing methods further includes identifying the trailer coupler as the object associated with the point in the depth map that indicates the shortest distance to the vehicle.

[0017] According to another exemplary embodiment, an autonomous vehicle system for locating and tracking a trailer coupler includes, among other possible things, a control device configured to convert an image from a vehicle camera into a depth map, the depth points including a plurality of points indicating a distance between an object in the image and a reference point, the control device configured to identify the trailer coupler as the point in the depth map closest to the reference point, and communicate the location of the identified coupler to a vehicle driving control system.

[0018] In another embodiment of the aforementioned autonomous vehicle system, the controller is further configured to remove points representing the ground surface from the depth map.

[0019] Another embodiment of any of the aforementioned autonomous vehicle systems further comprises a selection means for selecting an area of ​​interest within an image from the vehicle camera.

[0020] Another embodiment of any of the aforementioned autonomous vehicle systems further includes at least one sensor system of the vehicle, and the controller is further configured to determine a pose of the camera based on information indicative of vehicle odometry provided by the at least one sensor system.

[0021] Another embodiment of any of the aforementioned autonomous vehicle systems further comprises a route generation system in communication with the controller, the route generation system generating a route for the vehicle relative to the coupler.

[0022] In another embodiment of any of the aforementioned autonomous vehicle systems, the controller is further configured to identify the coupler as the object corresponding to the point in the depth map that indicates the shortest distance to the vehicle.

[0023] A computer-readable medium comprises instructions executable by a control device to locate and track a trailer coupler, the instructions according to another disclosed exemplary embodiment including, among other possible things, instructions to cause the control device to convert an image from a vehicle camera into a depth map, the depth points including a plurality of points indicating a distance between an object in the image and a reference point; instructions to cause the control device to identify the trailer coupler as the point in the depth map closest to the reference point; and instructions to cause the control device to identify the coupler and communicate the position of the identified coupler to a vehicle driving control system.

[0024] Another embodiment of the aforementioned computer-readable medium further comprises instructions for selecting a region of interest within an image from a vehicle camera and creating a depth map of objects within the selected region.

[0025] Although the various examples have specific components shown in the figures, embodiments of the present disclosure are not limited to those specific combinations. Some of the components or features from one of the examples can be used in combination with features or components from another of the examples.

[0026] These and other features disclosed herein can be best understood from the following specification and drawings, the following of which is a brief description. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a schematic diagram of a vehicle including a system for aligning a hitch with a coupler. [Figure 2] FIG. 1 is a flow diagram illustrating an exemplary process for identifying and tracking trailer couplings. [Figure 3] 1 is an exemplary image captured by a camera mounted on a vehicle. [Figure 4] 1 is an exemplary depth map of objects surrounding a vehicle. [Figure 5] 1 is a schematic diagram of an exemplary area of ​​interest including a trailer coupling; [Figure 6] FIG. 2 is a schematic plan view of a vehicle-mounted camera. DETAILED DESCRIPTION OF THE INVENTION

[0028] 1, a vehicle 22 and a trailer 24 are shown schematically. The vehicle includes a system 20 for identifying and tracking the trailer's coupler 26 such that the vehicle 22 can align its hitch 28 with the coupler 26. The exemplary system 20 utilizes images generated from a camera 46 and vehicle odometry to identify and track the relative orientation between the vehicle 22 and the trailer 24, enabling the coupling operation.

[0029] The exemplary vehicle 22 includes a controller 30 having a processor 32 and a memory device 34. The memory device 34 stores software instructions 36 that cause the processor 32 to perform functions that enable the system 22 to identify and track the couplers 26 of the trailers 24. The instructions 36 may be stored on a computer-readable medium 35.

[0030] The exemplary controller 30 may be a separate controller dedicated to the control system 20 or may be part of an overall vehicle control system. Thus, the exemplary controller 30 relates to devices and systems for performing the necessary calculations and / or operational operations of the control system 20. The controller 30 may be specially constructed for the operation of the control system 20, or may comprise at least a general-purpose computer selectively activated or reconfigured by software instructions 36 stored in a memory device 34. The computing system may also consist of a network of (different) processors.

[0031] Instructions 36 for configuring and operating controller 30, control system 20, and processor 32 are embodied in software instructions 36 that may be stored on computer-readable medium 35. Computer-readable medium 35 may be embodied in any type of structure, including, but not limited to, a floppy disk, optical disk, CD-ROM, magneto-optical disk, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card, application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus. The disclosed computer-readable medium may be non-transitory, such as the examples provided.

[0032] Additionally, the software instructions 36 may be stored in a memory device 34. The disclosed memory device 34 may include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, VRAM, etc.)) and / or non-volatile memory elements (e.g., ROM, hard disk, tape, CD-ROM, etc.). The software instructions 36 in the memory device 34 may include one or more separate programs, each of which includes an ordered list of executable instructions to implement a logical function. The disclosed controller 30 is configured to execute the software instructions 36 stored in the memory device 34, communicate data to and from the memory device 34, and generally control operations in accordance with the software. The software in the memory is read, in whole or in part, into the processor 32, possibly buffered within the processor, and then executed.

[0033] System 20 utilizes information from sensors mounted within vehicle 22 to determine vehicle odometry. In one disclosed exemplary embodiment, the sensors include an inertial measurement unit (IMU) 38, a global positioning system (GPS) 40, an accelerometer 42, a wheel speed sensor 44, and a wheel angle sensor 50. While several sensor systems are disclosed by way of example, it should be recognized that other sensor systems that provide information indicative of the environment surrounding vehicle 22 may be utilized and are within the scope of the present disclosure.

[0034] The autonomous features of the vehicle 22 are provided to assist the operator when performing difficult vehicle maneuvers and actions. One such maneuver is aligning the hitch 28 with the coupler of the trailer 24. The exemplary system 20 uses information obtained from the camera 46 and information indicative of vehicle odometry to identify and track the trailer coupling 26. The tracking of the coupler 26 in the image is then used by the navigation system 48 of the vehicle 22 to plot a map aligning the coupler 26 and the hitch 28. While one camera 46 is disclosed as an example, it should be recognized that more cameras pointed around the vehicle may be used and are within the scope and contemplation of the present disclosure. The exemplary camera 46 is a mono camera 46, but other camera configurations may be used within the scope and contemplation of the present disclosure.

[0035] 2 with continued reference to FIGURE 1, an exemplary system accumulates information indicative of vehicle operation from exemplary vehicle sensor systems 38, 40, 42, 44, and 50. Information obtained by each of these sensor systems is accumulated within vehicle navigation system 48. Vehicle navigation system 48 outputs information indicative of vehicle odometry.

[0036] The information indicative of vehicle odometry may be generated by a dynamic model of vehicle operation 45. The vehicle dynamic model 45 provides information indicative of vehicle movement. The dynamic model 45 may be a separate algorithm executed by the controller 30 according to software instructions 36 stored in the memory device 34.

[0037] Information from the camera 46 and the navigation system 48 is provided to a depth map generator 52. The exemplary depth map generator 52 is part of the controller 30 that operates to define and generate a depth map from images captured by the camera 46. The depth map generator 52 is embodied in the controller 30 as software instructions that are executed by the processor 32. The memory device 34 contains various instructions 36 that cause the processor 32 to create a depth map based on information provided by the navigation system 48 and the camera 46.

[0038] The maps referenced in this example disclosure are not necessarily generated for viewing by a vehicle operator. Instead, each of the disclosed maps is generated for use by control system 20 to provide for navigation of the vehicle through an environment, autonomously and / or semi-autonomously. Thus, the maps are generated to provide a means of compiling data related to the location within the environment surrounding vehicle 22. Furthermore, each of the maps described in this disclosure describes an organization of information and relationships between the organized information indicative of the environment surrounding the vehicle.

[0039] 3 and 4, with continued reference to FIG. 2, depth map generator 52 takes image 60 (FIG. 3) generated by camera 46 and converts the image into a depth map, such as that shown at 72 in FIG. 4. The depth map converts an object, such as a vehicle, shown at 74 and image 60 into a number of points indicating the distance between camera 46 and the vehicle. In this example, vehicle 74 is represented by a series of generally identical points of a common color, each representing a different distance from vehicle 22 and object 74.

[0040] It should be appreciated that the pose of the camera 46 on the vehicle 22 is determined and used in creating the depth map 72. The pose of the camera 46 is its position and orientation relative to the origin of a coordinate system that may be established using the initial pose of the camera in the surrounding environment. The position and orientation of the vehicle, or vehicle pose, is understood as the relative position between the vehicle and the origin of the coordinate system, which may be determined with knowledge of the camera pose, i.e., the position of the camera 46 and the particular orientation of the image 60.

[0041] 5 with continued reference to FIGS. 2-4, a region of interest (ROI) filter 54 selects a space 58 within the depth map 72 in which the coupler 26 is located. The region of interest filter 54 is used to specify a region within the image 60 in which the coupler 26 is identified and tracked as indicated at 56. Areas outside the region of interest 58 are ignored.

[0042] The exemplary region of interest 58 is a square space defined around the object of interest, in this exemplary embodiment, the trailer coupler 26. The box 58 may be selected automatically by a separate algorithm executed by the controller 30, or may be selected manually by an operator in the vehicle 22.

[0043] Although the exemplary region of interest 58 is a square, it should be appreciated that other shapes, such as a circle, may be utilized and are within the contemplation of the present disclosure.

[0044] Objects within the selected region of interest 58 are analyzed to identify couplings 26. Couplings 26 are identified as features within the region of interest 58 that are closest to the camera 46.

[0045] 6, those points within the region of interest 58 that are portions of the ground are eliminated to reduce unnecessary processing effort. Relative movement between the vehicle 22 and the trailer 24 occurs substantially in a direction 64 that is perpendicular to the plane of the camera 62 as shown in FIG. 6. Movement of the vehicle 22 relative to the trailer 24 is indicated by a change in the distance between the position of the coupler 26 and the camera plane 62.

[0046] Once the ground points 68 are filtered from the region of interest 58, the system 20 identifies the couplers 26 as the portions of the depth map that represent the objects and features closest to the vehicle 22. The points closest to the vehicle 22 that are part of the trailer 24 will be the couplers 26. These points are identified and labeled as couplers 26, and then tracked to align the hitch 28 with the couplers 26.

[0047] Once the coupler 26 is identified, the vehicle navigation system 48 generates a route to align the hitch 28 with the coupler 26 of the trailer 24. The navigation system 48 may also automatically track the region of interest 58 such that only those points in the depth map 72 that indicate the distance between the vehicle 22 and the coupler 26 are tracked. The vehicle navigation system 48 defines the route to reduce the lateral and longitudinal distance between the hitch 28 and the coupler 26. The resulting route generated by the navigation system 48 provides a substantially direct and linear path.

[0048] The navigation system 48 may generate a route to align the hitch 28 and coupler, without further tracking the hitch 28. The tow vehicle 22 simply travels the generated route until the hitch 28 reaches the coupler 26. An end point that provides the desired alignment may be preselected and / or automatically set by the navigation system 48.

[0049] Although various non-limiting embodiments are shown with particular components or steps, embodiments of the present disclosure are not limited to those particular combinations. Some of the components or features from any of the non-limiting embodiments can be used in combination with features or components from any of the other non-limiting embodiments.

[0050] It should be understood that like reference numerals identify corresponding or similar elements throughout the several views. Although particular component arrangements are disclosed and illustrated in these exemplary embodiments, it should be understood that other arrangements may benefit from the teachings of the present disclosure.

[0051] The foregoing description is intended to be illustrative and not in any limiting sense. Those skilled in the art will recognize that certain modifications may fall within the scope of the present disclosure. For these reasons, the following claims should be studied to determine the true scope and content of the present disclosure.

Claims

1. 1. A method for locating and tracking a trailer coupler for autonomous vehicle operation, comprising: converting an image acquired from a vehicle camera into a depth map such that the depth points indicate distances between objects in the image and a reference point; identifying a trailer coupler as the point in the revised depth map that is closest to said reference point; communicating the identified coupler position to a vehicle operation control system; A method comprising:

2. The method of claim 1 , further comprising selecting a region of interest within an image from the vehicle camera and creating the depth map within the selected region of interest.

3. The method of claim 2 , wherein the region of interest is smaller than the image from the vehicle camera and includes the trailer coupler.

4. The method of claim 2 , wherein the region of interest is selected automatically by an algorithm executed by a control unit in the vehicle.

5. The method of claim 1 , further comprising removing from the depth map points in the depth map that indicate a ground surface.

6. The method of claim 1 , further comprising determining a path for the coupler based on the depth map.

7. The method of claim 1 , further comprising determining a pose of the vehicle camera based on information from at least one sensor system within the vehicle indicative of movement of the vehicle.

8. generating a dynamic model of vehicle movement based on information from at least one sensor system of the vehicle; determining the pose of the vehicle camera based on the dynamic model; The method of claim 7 further comprising:

9. The method of claim 8 , wherein the at least one sensor system comprises at least one of an acceleration sensor, a wheel angle sensor, an inertial measurement unit, or a global positioning system.

10. The method of claim 1 , wherein the vehicle camera comprises a mono camera.

11. The method of claim 1 , further comprising generating the depth map in real time by a control device in the vehicle.

12. The method of claim 1 , further comprising identifying the trailer coupler as the object associated with a point in the depth map that indicates a shortest distance to the vehicle.

13. 1. An autonomous vehicle system for locating and tracking a trailer coupler, comprising: converting an image from a vehicle camera into a depth map, wherein depth points include a plurality of points indicating distances between objects in the image and a reference point; identifying a trailer coupler as the point in the depth map closest to said reference point; communicating the identified position of the coupler to a vehicle driving control system; 1. An autonomous vehicle system comprising: a control device configured to:

14. The autonomous vehicle system of claim 13 , wherein the controller is further configured to remove points from the depth map that represent ground planes.

15. 15. The autonomous vehicle system of claim 14, further comprising a selection means for selecting an area of ​​interest in an image from the vehicle camera.

16. 16. The autonomous vehicle system of claim 15, further comprising at least one sensor system of the vehicle, wherein the controller is further configured to determine a pose of the vehicle camera based on information indicative of vehicle odometry provided by the at least one sensor system.

17. 17. The autonomous vehicle system of claim 16, further comprising a route generation system in communication with the controller, the route generation system generating a route for the vehicle relative to the coupler.

18. The autonomous vehicle system of claim 13 , wherein the controller is further configured to identify the coupler as the object corresponding to a point in the depth map that indicates a shortest distance to the vehicle.

19. 1. A computer-readable medium comprising instructions executable by a controller for locating and tracking a trailer coupler, the instructions comprising: instructions to the controller to convert an image acquired from a vehicle camera into a depth map including a plurality of depth points indicating a distance between an object in the image and a reference point; instructions to cause the controller to identify a trailer coupler as the point in the depth map that is closest to the reference point; instructions for causing the controller to identify a coupler and communicate the identified location of the coupler to a vehicle driving control system; 1. A computer-readable medium comprising:

20. 20. The computer-readable medium of claim 19, further comprising instructions for selecting a region of interest within an image from the vehicle camera and creating the depth map of an object within the selected region.

Images