Trailer detection and autonomous bolting

Abstract

A method for autonomously maneuvering a towing vehicle (100) toward a trailer (200) positioned behind the towing vehicle is provided. The method includes receiving one or more images (400) from one or more cameras (142), the one or more cameras (142) being positioned on a rear portion of the towing vehicle. The method also includes identifying a trailer representation within the one or more images. The trailer representation indicates a trailer positioned behind the towing vehicle. The method also includes setting a vertical center of the trailer representation as a target. The method also includes determining a first steering wheel angle to steer the towing vehicle (100) so that the vehicle autonomously maneuvers in a direction toward the target. The method also includes transmitting instructions to a driving system (110), the instructions causing the towing vehicle (100) to maneuver based on the first steering wheel angle.

Description

Technical Field

[0001] The present disclosure relates to methods and apparatus for determining the position of a trailer positioned behind a vehicle and autonomously maneuvering toward the trailer for hitching. Background Art

[0002] Trailers are usually unpowered vehicles pulled by powered traction vehicles. Trailers can be general trailers, pop-up campers, travel trailers, livestock trailers, flatbed trailers, enclosed dispatch winches, and boat trailers, etc. The traction vehicle can be a car, a cross-border vehicle, a truck, a van, a sports utility vehicle (SUV), a recreational vehicle (RV), or any other vehicle configured to be attached to the trailer and pull the trailer. The trailer can be attached to the powered vehicle using a trailer hitch. The receiver hitch is installed on the traction vehicle and is connected to the trailer hitch to form a connection. The trailer hitch can be a ball socket, a spare wheel and a gooseneck or a trailer jack. Other attachment mechanisms can also be used. In some examples, in addition to the mechanical connection between the trailer and the powered vehicle, the trailer is also electrically connected to the traction vehicle. In this way, the electrical connection allows the trailer to obtain the feed from the rear light circuit of the powered vehicle, allowing the trailer to have taillights, turn signals, and brake lights that are synchronized with the lights of the powered vehicle.

[0003] Recent advances in sensor technology have led to improved safety systems for vehicles. As such, it is desirable to provide a system capable of determining the position of a trailer relative to a towing vehicle, which system allows autonomous steering of the towing vehicle towards the trailer. Summary of the invention

[0004] One aspect of the present disclosure provides a method for autonomously maneuvering a towing vehicle toward a trailer positioned behind the towing vehicle. The method includes receiving one or more images from one or more cameras at data processing hardware, the one or more cameras being positioned on a rear portion of the towing vehicle and communicating with the data processing hardware. The method also includes identifying a trailer representation within the one or more images by the data processing hardware. The trailer representation indicates a trailer positioned behind the towing vehicle. The method also includes setting the vertical center of the trailer representation as a target by the data processing hardware. The method also includes determining a first steering wheel angle by the data processing hardware to turn the towing vehicle so that the vehicle is autonomously maneuvered in a direction toward the target. The method also includes transmitting instructions from the data processing hardware to a driving system that communicates with the data processing hardware, the instructions causing the towing vehicle to maneuver based on the first steering wheel angle.

[0005] Embodiments of the present disclosure may include one or more of the following optional features. In some embodiments, the method further includes determining, by data processing hardware, a pixel distance between a vertical center of the trailer representation and a vertical center of one or more images. When the pixel distance between the vertical center of the trailer representation and the vertical center of the one or more images is less than a threshold, the method includes: setting, by the data processing hardware, the vertical center of the trailer representation as a target; and determining, by the data processing hardware, a first steering wheel angle to steer the towing vehicle so that the vehicle is autonomously maneuvered in a direction toward the target.

[0006] In some examples, the method further includes: determining, by the data processing hardware, a pixel distance between the vertical center of the trailer representation and the vertical center of the one or more images. When the pixel distance between the vertical center of the trailer representation and the vertical center of the one or more images is greater than a threshold, the method may further include: setting, by the data processing hardware, a predefined lateral offset from the vertical center of the image as a target; and determining, by the data processing hardware, a second steering wheel angle to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

[0007] In some embodiments, the threshold comprises a plurality of pixels.The second steering wheel angle for steering the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target may be a maximum steering wheel angle that prevents the towing vehicle and trailer from V-bending.

[0008] In some examples, the method further includes defining the trailer representation using a bounding box. The vertical center of the trailer representation includes a line that vertically divides the trailer representation into two equal sections. The vertical center of the one or more images may include a line that vertically divides each of the one or more images into two equal sections, each section having an equal number of pixels. In some examples, the one or more cameras include a monocular camera and / or a fisheye camera.

[0009] Another aspect of the present disclosure provides a system for autonomously maneuvering a towing vehicle toward a trailer positioned behind the towing vehicle. The system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include receiving one or more images from one or more cameras, the one or more cameras being positioned on a rear portion of the towing vehicle and communicating with the data processing hardware. The operations also include identifying a trailer representation within the one or more images. The trailer representation indicates a trailer positioned behind the towing vehicle. The operations also include setting the vertical center of the trailer representation to a target. The operations also include determining a first steering wheel angle to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target. The operations also include transmitting instructions to a driving system in communication with the data processing hardware, the instructions causing the towing vehicle to maneuver based on the first steering wheel angle.

[0010] Embodiments of this aspect of the present disclosure may include one or more of the following optional features. In some embodiments, the operation further includes: determining a pixel distance between a vertical center of the trailer representation and a vertical center of one or more images. When the pixel distance between the vertical center of the trailer representation and the vertical center of the one or more images is less than a threshold, the operation includes: setting the vertical center of the trailer representation as a target; and determining, by data processing hardware, a first steering wheel angle to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

[0011] In some embodiments, the operations further include: determining a pixel distance between a vertical center of the trailer representation and a vertical center of the one or more images. When the pixel distance between the vertical center of the trailer representation and the vertical center of the one or more images is greater than a threshold, the operations include: setting a predefined lateral offset from the vertical center of the image as a target; and determining, by the data processing hardware, a second steering wheel angle to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

[0012] In some examples, the threshold comprises a plurality of pixels.The second steering wheel angle for steering the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target may be a maximum steering wheel angle that prevents the towing vehicle and trailer from V-bending.

[0013] In some examples, the operations further include defining the trailer representation using a bounding box, wherein a vertical center of the trailer representation includes a line that vertically divides the trailer representation into two equal sections. The vertical center of the one or more images may include a line that vertically divides each of the one or more images into two equal sections, each section having an equal number of pixels. In the system, the one or more cameras include a monocular camera and / or a fisheye camera.

[0014] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a schematic top view of an exemplary towing vehicle spaced apart from a trailer positioned behind the towing vehicle.

[0016] Figure 2 is a schematic illustration of an exemplary towing vehicle with a user interface and a sensor system.

[0017] Figure 3 is a schematic illustration of an exemplary arrangement for autonomously maneuvering a towing vehicle toward a trailer positioned behind the towing vehicle.

[0018] Figure 4A – Figure 4G It is executing Figure 3 Schematic view of an image from a rear camera during operation.

[0019] Figure 5 is a schematic illustration of an exemplary arrangement for autonomously maneuvering a vehicle hitch ball of a towing vehicle toward operation of a trailer hitch coupling of a trailer positioned behind the towing vehicle.

[0020] Fig. 6A – Figure 6C It is executing Figure 5 Schematic view of an image from a rear camera during operation.

[0021] Figure 7 is a schematic illustration of an exemplary arrangement for autonomously steering a towing vehicle toward a target.

[0022] In the various drawings, like reference numerals indicate like elements. DETAILED DESCRIPTION

[0023] A towing vehicle, such as, but not limited to, a car, a crossover vehicle, a truck, a van, a sport utility vehicle (SUV), and a recreational vehicle (RV), may be configured to tow a trailer. The towing vehicle is connected to the trailer by way of a trailer hitch. It is desirable to enable the towing vehicle to autonomously back up toward the trailer, which is identified from one or more trailer representations of the trailer displayed on a user interface (e.g., a user display). In addition, it is also desirable to enable the towing vehicle to estimate the trailer position, such as the angle and position of the trailer relative to the towing vehicle, based on one or more images received from a camera positioned on the rear portion of the towing vehicle. In addition, it is desirable to enable the towing vehicle to autonomously maneuver toward the trailer and align with the trailer's trailer hitch connector for bolting between the vehicle hitch ball of the towing vehicle and the trailer hitch connector.

[0024] refer to Figure 1 and Figure 2 In some embodiments, the driver of the towing vehicle 100 wishes to tow a trailer 200 positioned behind the towing vehicle 100. The towing vehicle 100 may be configured to receive an indication of a trailer selection 134 from the driver, which is associated with the selected trailer 200, 200a-c. In some examples, the driver steers the towing vehicle 100 toward the selected trailer 200, 200a-c, while in other examples, the towing vehicle 100 autonomously drives toward the selected trailer 200, 200a-c. The towing vehicle 100 may include a driving system 110, for example, the driving system 110 steers the towing vehicle 100 across the road surface based on a driving command having x, y, and z components. As shown, the driving system 110 includes wheels 112, i.e., a right front wheel 112a, a left front wheel 112b, a right rear wheel 112c, and a left rear wheel 112d. The driving system 110 may also include other wheel configurations. The driving system 110 may also include a braking system 114 including brakes associated with each wheel 112 and an acceleration system 116 configured to adjust the speed and direction of the towing vehicle 100. In addition, the driving system 110 may include a suspension system 118 including tires associated with each wheel 112, tire air, springs, shock absorbers, and linkages that connect the towing vehicle 100 to its wheels 112 and allow relative movement between the towing vehicle 100 and the wheels 112. The suspension system 118 may be configured to adjust the height of the towing vehicle 100, allowing the towing vehicle hitch 120 (e.g., vehicle hitch ball 122) to align with the trailer hitch 210 (e.g., trailer hitch coupling or trailer hitch cup 212), which allows an autonomous connection between the towing vehicle 100 and the trailer 200.

[0025] The towing vehicle 100 can move across the road surface by various combinations of movement relative to three mutually perpendicular axes defined by the towing vehicle 100 (lateral axis X, front-to-back axis Y, and central vertical axis Z). The lateral axis X extends between the right and left sides of the towing vehicle 100. The forward driving direction along the front-to-back axis Y is represented as F, also referred to as forward movement. In addition, the rearward or rearward driving direction along the front-to-back direction Y is represented as R, also referred to as rearward movement. When the suspension system 118 adjusts the suspension of the towing vehicle 100, the towing vehicle 100 can tilt about the X-axis and / or the Y-axis, or move along the central vertical axis Z.

[0026] The towing vehicle 100 may include a user interface 130, such as a display 132. The user interface 130 receives one or more user commands from the driver via one or more input mechanisms or a touch screen display 132, and / or displays one or more notifications to the driver. The user interface 130 communicates with a vehicle controller 150, which in turn communicates with a sensor system 140. In some examples, the user interface 130 displays an image 400 of the environment of the towing vehicle 100, resulting in one or more commands being received by the user interface 130 (from the driver), the one or more commands initiating the performance of one or more actions. In some examples, the user display 132 displays one or more trailer representations 136, 136a-c of a trailer 200 positioned behind the towing vehicle 100. In this case, the driver makes a trailer selection 134 of the trailer representations 136, 136a-c of the trailer 200.

[0027] The towing vehicle 100 may include a sensor system 140 to provide reliable and robust driving. The sensor system 140 may include different types of sensors that may be used alone or in conjunction with each other to form a sense of the environment of the towing vehicle 100, which is used for the towing vehicle 100 to drive and help the driver make intelligent decisions based on objects and obstacles detected by the sensor system 140. The sensor system 140 may include one or more cameras 142, 142a-d. In some embodiments, the towing vehicle 100 includes a rear camera 142, 142a, which is installed to provide a view of the rear driving path of the towing vehicle 100. The rear camera 142a may include a fisheye lens, which includes an ultra-wide-angle lens that produces strong visual distortion, which is intended to form a wide panoramic or hemispherical image. The fisheye camera captures images with extremely wide-angle views. In addition, the image 400 captured by the fisheye lens 142a has a typical convex non-linear representation. Other types of cameras may also be used to capture images behind the towing vehicle 100 .

[0028] In some embodiments, the sensor system 140 may also include an IMU (Inertial Measurement Unit) 144 configured to output IMU data 145, including the linear acceleration (using one or more accelerometers) and rotation rate (using one or more gyroscopes) of the vehicle. In some examples, the IMU 144 also determines a heading reference for the towing vehicle 100. Thus, the IMU 144 determines the pitch, roll, and yaw of the towing vehicle 100. The sensor system 140 may include other sensors, such as, but not limited to, radar, sonar, LIDAR (Light Detection and Ranging, which may require optical remote sensing, which measures the properties of scattered light to obtain the distance and / or other information of distant targets), LADAR (Laser Detection and Ranging), and the like.

[0029] The vehicle controller 150 includes an operating device (or processor) / data processing hardware 152 (e.g., a central processing unit having one or more operating processors) in communication with a non-volatile memory 154 (e.g., a hard disk, flash memory, random access memory, memory hardware), wherein the non-volatile memory 154 is capable of storing instructions that can be executed on (multiple) data processing hardware 152.

[0030] The vehicle controller 150 executes the driving assistance system 160. The driving assistance system 160 receives the image 400 from the camera 142 and executes actions 162-166 which send commands 170 to the driving system 110 resulting in autonomous driving of the towing vehicle 100 in the rearward direction R towards the selected trailer 200.

[0031] The driving assistance system 160 includes a braking behavior 162, a speed behavior 164, and a steering behavior 166. In some examples, the driving assistance system 160 also includes a hitch connection behavior (allowing the vehicle hitch ball 122 to connect to the trailer hitch cup 212) and a suspension adjustment behavior (causing the vehicle suspension to be adjusted to allow the hitch connection of the vehicle hitch ball 122 and the trailer hitch cup 212). Each behavior 162-166 causes the towing vehicle 100 to take an action, such as driving backward, turning at a specific angle, braking, accelerating, decelerating, etc. The vehicle controller 150 can steer the towing vehicle 100 across the road surface in any direction by controlling the driving system 110 (more specifically, by issuing commands 170 to the driving system 110). In addition, the driving assistance system 160 determines the steering wheel angle and speed of the towing vehicle 100 based on the received image 400.

[0032] The braking action 162 may be performed to stop the towing vehicle 100 or to slow the towing vehicle 100 based on the determined steering wheel angle and the determined speed of the towing vehicle 100. The braking action 162 sends a signal or command 170 to the driving system 110 (e.g., the braking system 114) to stop the towing vehicle 100, reduce the speed of the towing vehicle 100, or cease applying brakes.

[0033] Speed ​​behavior 164 may be executed to change the speed of towing vehicle 100 by accelerating or decelerating based on the determined steering wheel angle and the determined speed of towing vehicle 100. Speed ​​behavior 164 sends a signal or command 170 to braking system 114 for deceleration or to acceleration system 116 for acceleration.

[0034] Steering behavior 166 may be executed to change the direction of towing vehicle 100 based on the determined steering wheel angle and the determined speed of towing vehicle 100. As such, steering behavior 166 sends a signal or command 170 to acceleration system 116 indicating a steering angle, causing driving system 110 to change direction.

[0035] In some examples, the steering behavior 166 includes a proportional-integral-derivative controller 168 (PID controller). The PID controller 168 is a control loop feedback mechanism that is widely used in industrial control systems and various other applications that require continuous modulation control. The PID controller 168 continuously calculates an error value as the difference between the desired set point and the measured process variable, and applies corrections based on proportional, integral, and derivative terms. The PID controller 168 determines the steering wheel angle based on the horizontal pixel distance D between the image vertical centerline 402 and the target 416.

[0036] Figure 3 as well as Figure 4A – Figure 4G The method 300 executed by the vehicle controller 150 is shown for determining the position P of the trailer 200 within the image 400. T , and autonomously maneuvering the towing vehicle 100 toward the trailer 200. The method 300 is based on analyzing the received image 400 and determining the trailer position P based on the analyzed image 400. T. In other words, as the towing vehicle 100 drives in a rearward direction R toward the trailer 200, the method 300 analyzes the images 400 received from the rear camera 142a. The controller 150 analyzes one or more of the images 400 and instructs the driving system 110 to maneuver the towing vehicle 100 so that within each image 400, the pixel distance D between the center of the target (i.e., the trailer representation 136 of the towing vehicle 100) and the vertical centerline 402 of the image 400 is minimized. During execution of the method 300 (by the controller 150), the towing vehicle 100 autonomously approaches the trailer 200 in a manner similar to that of a human driver, such as by using single or double arc maneuvers. In some examples, each image 400, 400A–400G includes a representation of the rear portion of the towing vehicle 100 and a tow hitch hook ball representation 422, which may be bounded by a bounding box or circle 423.

[0037] At box 301, the driving assistance system 160 receives one or more images 400 from the rear camera 142a. In some examples, at box 302, the driving assistance system 160 determines the location of the trailer 200 within the image 400 by analyzing the one or more images 400. When the image 400 includes more than one trailer 200, the driving assistance system 160 instructs the display 132 to request a trailer selection 134 associated with one of the trailers 200. In other examples, the driver selects the trailer 200 (i.e., the representation 136 of the trailer 200) from the image 400. In some examples, once the trailer representation 136 is selected or identified, the driving assistance system 160 defines the identified trailer representation 136 by a bounding box (also referred to as a region of interest (ROI) 410). The driving assistance system 160 then determines a vertical centerline 412 of the ROI 410. The vertical centerline 412 represents the centerline of the trailer representation 136 in the image 400. At box 302, the driver assistance system 160 determines a vertical centerline 402 of the image 400 and a horizontal centerline 404 of the image 400. The vertical centerline 402 and the horizontal centerline 404 intersect at a center C of the image.

[0038] At box 304, the driver assistance system 160 determines the horizontal pixel distance D between the image vertical centerline 402 and the ROI vertical centerline 412 (i.e., the center of the trailer bounding box). At decision box 306, the driver assistance system 160 determines whether the horizontal pixel distance D is less than a threshold value, which is a number of pixels. In some examples, the threshold value may be half the width of the ROI 410. If the driver assistance system 160 determines that the horizontal distance D is less than the threshold value, then at box 308, the driver assistance system 160 sets the ROI vertical centerline 412 (i.e., the trailer vertical centerline 412) as the target 418, as shown in FIG. Figure 4D – Figure 4F412. After the driving assistance system 160 sets the target 418 as the vertical centerline 412 of the ROI 410 and the towing vehicle 100 begins autonomous maneuvering toward the target 418, then, at box 310, the driving assistance system 160 determines the distance D between the vertical centerline 402 of the image and the target 418 (i.e., the vertical centerline 412). Thereafter, at box 312, the PID controller 168 analyzes the horizontal distance D between the vertical centerline 402 of the image and the target 418, and at box 314, the PID controller 168 determines the steering wheel angle to steer and maneuver the towing vehicle 100 toward the target vertical centerline 412 in real time. At box 316, after the driving assistance system 160 issues a command 170 to the driving system 110 to maneuver the towing vehicle 100 based on the determined steering wheel angle, the driving assistance system 160 updates the ROI vertical centerline 412. At decision box 318, the driver assistance system 160 determines whether the vertical centerline 402 of the image has passed through the ROI vertical centerline 412. In other words, at decision box 318, the driver assistance system 160 determines whether the horizontal distance D is equal to or less than zero. If the driver assistance system 160 determines that the vertical centerline 402 of the image has not passed through the ROI vertical centerline 412, the driver assistance system 160 repeats the steps of boxes 308-318. The horizontal distance D being less than zero indicates that the towing vehicle 100 is moving away from the trailer 200, because the horizontal distance D being equal to zero indicates that the towing vehicle 100 and the trailer 200 are aligned. At decision box 318, if the driver assistance system 160 determines that the vertical centerline 402 of the image has passed through the ROI vertical centerline 412, the driver assistance system 160 stops its calculations because the towing vehicle 100 has reached a predetermined pixel distance separated from the horizontal centerline 404 of the image 400.

[0039] Figure 4A Then, the towing vehicle 100 autonomously turns at the maximum steering wheel angle, such as Figure 4B Then, during the second turn, the towing vehicle 100 tracks the ROI vertical centerline 412, as shown in FIG. Figure 4C – Figure 4D ​​ Figure 3 At block 320, the driver assistance system 160 uses the offset 416 from the ROI vertical centerline 412 as a target ( Figure 4A ). Offset 416 is a vertical line within image 400 that is on the same side of the image as trailer 200. As shown, trailer 200 is on the right side of image 400, and therefore, offset 416 is also on the right side of image 400. At box 322, driver assistance system 160 determines a maximum steering wheel angle to steer towing vehicle 100 toward target 418 because horizontal distance D was greater than the threshold as determined at decision box 306. Once towing vehicle 100 has completed the maximum steering wheel angle, Figure 4B The captured image 400 shown in FIG. 4 includes the trailer 200 at the opposite side of the vertical centerline 402 of the image from its initial position. Figure 4A In the maneuver, the towing vehicle 100 and the offset 416 are on the same side of the image 400 relative to the vertical centerline 402; and after the maneuver, the towing vehicle 100 is on the left side and the offset 416 remains on the right side. At box 324, due to the movement of the towing vehicle 100 in box 322, the driving assistance system 160 updates the target 418 within the image 400. Then, at decision box 326, the driving assistance system 160 determines whether the vertical centerline 402 of the image has passed the target 418 while the towing vehicle 100 moves autonomously in the rearward direction R. In other words, at decision box 326, the driving assistance system 160 determines whether the horizontal distance D is equal to or less than zero. If the driving assistance system 160 determines that the image center 414 has not passed the target 418, the driving assistance system 160 repeats the steps of boxes 322, 324, and 326. However, if the driving assistance system 160 determines that the image center 414 has passed the target 418 (such as Figure 4D ), then at box 308, the driving assistance system 160 sets the ROI vertical centerline 412 as the new target 418.

[0040] refer to Figure 5 as well as Fig. 6A – Figure 6C In some embodiments, the controller 150 performs a process similar to that described with respect to Figure 3 The method 300 is described to identify the trailer hitch cup 212 of the trailer 200 and autonomously drive to the trailer hitch cup 212 and connect the vehicle hitch ball 122 to the trailer hitch cup 212 .

[0041] In some embodiments, at box 502, the driving assistance system 160 receives one or more images 400 from the rear camera 142a. The driving assistance system 160 determines the location of the trailer hitch cup 212 within the image 400 by analyzing the one or more images 400. In some examples, the driving assistance system 160 instructs the display 132 to request a selection of a hitch representation associated with the trailer 200 within the image 400 from the driver. The driver selects the trailer hitch cup 212 (i.e., the representation 136 of the trailer hitch cup 212) from the image 400. In some examples, once the hitch representation 424 is identified, the driving assistance system 160 defines the identified hitch representation 424 by a bounding box or circle (as shown), also referred to as a region of interest (ROI) 410. Then, at box 504, the driving assistance system 160 determines the vertical centerline 412 of the ROI 410. The vertical centerline 412 represents the centerline of the hitch representation 424 in the image 400. Also, at block 504 , the driver assistance system 160 determines the vertical centerline 402 of the image 400 .

[0042] At box 506, the driver assistance system 160 sets the ROI vertical centerline 412 (ie, the coupling vertical centerline 412) as the target 418, as shown in FIG. Fig. 6A – Figure 6C410 and the towing vehicle 100 begins autonomously maneuvering toward the target 418. Then, at box 508, the driving assistance system 160 determines the distance D between the vertical centerline 402 of the image and the target 418 (i.e., the vertical centerline 412). Thereafter, at box 510, the PID controller 168 analyzes the horizontal distance D between the vertical centerline 402 of the image and the target 418, and at box 512, the PID controller 168 determines the steering wheel angle to steer and maneuver the towing vehicle 100 toward the target vertical centerline 412 in real time. At box 514, after the driving assistance system 160 issues a command 170 to the driving system 110 to maneuver the towing vehicle 100 based on the determined steering wheel angle, the driving assistance system 160 updates the ROI vertical centerline 412. At decision box 516, the driver assistance system 160 determines whether the vertical centerline 402 of the image has passed through the ROI vertical centerline 412. In other words, at decision box 516, the driver assistance system 160 determines whether the horizontal distance D is equal to or less than zero. If the driver assistance system 160 determines that the vertical centerline 402 of the image has not passed through the ROI vertical centerline 412, the driver assistance system 160 repeats the steps of boxes 506-box 516. The horizontal distance D being less than zero indicates that the towing vehicle 100 is moving away from the trailer 200, because the horizontal distance D being equal to zero indicates that the towing vehicle 100 and the trailer 200 are aligned. At decision box 318, if the driver assistance system 160 determines that the vertical centerline 402 of the image has passed through the ROI vertical centerline 412, the driver assistance system 160 stops its calculations because the towing hitch ball 122 is aligned with the trailer hitch cup 212.

[0043] As previously discussed, the driver assistance system 160 requires low computing resources and can achieve real-time performance on low-cost hardware. Since no markings are required on the trailer, the operational and computational complexity is reduced, and thus the production cost of the system is also reduced.

[0044] Figure 7 An exemplary arrangement of operations of method 700 is provided for using Figure 1 – Figure 6C The method 700 autonomously maneuvers the towing vehicle 100 toward the trailer 200 positioned behind the towing vehicle 100 using the system described in the drawings. At block 702, the method 700 includes receiving, at data processing hardware 152 (of a controller 150), one or more images 400 from one or more cameras 142, 142a-d positioned on a rear portion of the towing vehicle 100 and in communication with the data processing hardware 152.

[0045] At block 704 , the method 700 includes identifying, by the data processing hardware 152 , within the one or more images 400 , a trailer representation 136 indicative of the trailer 200 positioned behind the towing vehicle 100 .

[0046] At block 706 , the method 700 includes setting, by the data processing hardware 152 , the vertical centerline 412 of the trailer representation 136 as the target 418 .

[0047] At block 708 , the method 700 includes determining, by the data processing hardware 152 , a first steering wheel angle to steer the towing vehicle 100 such that the towing vehicle 100 autonomously maneuvers in a direction toward the target 418 (ie, the vertical centerline 412 ).

[0048] At block 710 , the method 700 includes transmitting a command 170 from the data processing hardware 152 to the driving system 110 in communication with the data processing hardware 152 , the command 170 causing the towing vehicle 100 to maneuver based on the first steering wheel angle.

[0049] In some examples, the method 700 includes determining, by the data processing hardware 152, a pixel distance D between the vertical centerline 412 of the trailer representation 136 and the vertical centerline 402 of the one or more images 400. When the pixel distance D between the vertical centerline 412 of the trailer representation 136 and the vertical centerline 402 of the one or more images 400 is less than a threshold, the method includes: setting, by the data processing hardware 152, the vertical centerline 412 of the trailer representation 136 as the target 418; and determining, by the data processing hardware 152, a first steering wheel angle to steer the towing vehicle 100 so that the vehicle autonomously maneuvers in a direction toward the target 418 (i.e., the vertical centerline 412 of the trailer representation 136).

[0050] In some embodiments, the method 700 includes determining, by the data processing hardware 152, a pixel distance D between the vertical centerline 412 of the trailer representation 136 and the vertical centerline 402 of the one or more images 400. When the pixel distance D between the vertical centerline 412 of the trailer representation 136 and the vertical center of the one or more images 400 is greater than a threshold, the method 700 includes: setting, by the data processing hardware 152, a predefined lateral offset 416 from the vertical centerline 402 of the image 400 as a target 418; and determining, by the data processing hardware 152, a second steering wheel angle to turn the towing vehicle 100 so that the vehicle autonomously maneuvers in a direction toward the target 418. In some examples, the threshold includes a plurality of pixels. The second steering wheel angle for turning the towing vehicle 100 so that the vehicle autonomously maneuvers in a direction toward the target 418 is a maximum steering wheel angle that prevents the towing vehicle 100 and the trailer 200 from V-bending.

[0051] In some embodiments, the method 700 also includes defining the trailer representation 136 using the bounding box 410. The vertical centerline 412 of the trailer representation 136 includes a line that vertically divides the trailer representation 136 into two equal sections. In some examples, the vertical centerline 402 of the one or more images 400 includes a line that vertically divides each image 400 in the one or more images 400 into two equal sections, wherein each section has an equal number of pixels. The one or more cameras can be a monocular camera and / or a fisheye camera.

[0052] Various embodiments of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0053] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language and / or an object-oriented programming language and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device and / or means (e.g., disks, optical disks, memories and programmable logic devices (PLDs)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0054] The subject matter and implementation scheme of functional operation described in this specification may be implemented in a digital electronic circuit system or in computer software, firmware or hardware, including the structure disclosed in this specification and its structural equivalent or one or more combinations thereof. In addition, the subject matter described in this specification may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device, or to control the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material composition affecting a machine-readable propagation signal, or a combination of one or more thereof. The terms "data processing device", "computing device" and "computing processor" cover all devices, devices and machines for processing data, including programmable processors, computers or multiple processors or computers by way of example. In addition to hardware, the device may also include code, which forms an execution environment for the computer program involved, for example, code constituting a processor firmware, a protocol stack, a database management system, an operating system or one or more combinations thereof. A propagation signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0055] Similarly, although operations are depicted in the accompanying drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may be integrated together in a single software product, or packaged into multiple software products as a whole.

[0056] A number of embodiments have been described. However, it will be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A method for autonomously steering a towing vehicle towards a trailer positioned behind the towing vehicle, the method include: receiving, at data processing hardware, one or more images from one or more cameras positioned on a rear portion of the towing vehicle and in communication with the data processing hardware; identifying, by the data processing hardware, within the one or more images, a trailer representation indicative of the trailer positioned behind the towing vehicle; setting, by the data processing hardware, a vertical centerline of the trailer representation as a target; determining, by the data processing hardware, a first steering wheel angle based on a vertical centerline represented by the trailer to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target; transmitting instructions from the data processing hardware to a steering system in communication with the data processing hardware, the instructions causing the towing vehicle to steer based on the first steering wheel angle; determining, by the data processing hardware, a pixel distance between the vertical centerline of the trailer representation and a vertical centerline of the one or more images; When the pixel distance between the vertical centerline of the trailer representation and the vertical centerline of the one or more images is greater than a threshold: setting, by the data processing hardware, a predefined lateral offset from the vertical centerline of the image as the target; as well as A second steering wheel angle is determined by the data processing hardware to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

2. The method according to claim 1, further comprising: include: determining, by the data processing hardware, a pixel distance between the vertical centerline of the trailer representation and a vertical centerline of the one or more images, the vertical centerline of the one or more images passing through a center defined by the vertical centerline of the one or more images and a horizontal centerline of the one or more images; When the pixel distance between the vertical centerline of the trailer representation and the vertical centerline of the one or more images is less than a threshold: setting, by the data processing hardware, the vertical centerline of the trailer representation as the target; and The first steering wheel angle is determined by the data processing hardware to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

3. The method according to claim 1, in, The threshold includes a plurality of pixels.

4. The method according to claim 1, in, The second steering wheel angle for steering the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target is a maximum steering wheel angle that prevents the towing vehicle and the trailer from V-bending.

5. The method according to claim 1, further comprising: include: The trailer representation is bounded by a bounding box, wherein the vertical centerline of the trailer representation comprises a line that vertically divides the trailer representation into two equal sections.

6. The method according to claim 1, in, The vertical centerline of the one or more images comprises a line that vertically divides each of the one or more images into two equal segments, each segment having an equal number of pixels.

7. The method according to claim 1, in, The one or more cameras include a monocular camera.

8. The method according to claim 1, in, The one or more cameras include a fisheye camera.

9. A system for autonomously steering a towing vehicle towards a trailer positioned behind the towing vehicle, the system include: Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations, the operations comprising: receiving one or more images from one or more cameras positioned on a rear portion of the towing vehicle and in communication with the data processing hardware; identifying within the one or more images a trailer representation indicative of the trailer positioned behind the towing vehicle; setting a vertical centerline of the trailer representation as a target; determining a first steering wheel angle based on a vertical centerline of the trailer representation to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target; transmitting instructions to a steering system in communication with the data processing hardware, the instructions causing the towing vehicle to steer based on the first steering wheel angle; determining a pixel distance between the vertical centerline of the trailer representation and a vertical centerline of the one or more images; When the pixel distance between the vertical centerline of the trailer representation and the vertical centerline of the one or more images is greater than a threshold: setting, by the data processing hardware, a predefined lateral offset from the vertical centerline of the image as the target; and A second steering wheel angle is determined by the data processing hardware to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

10. The system according to claim 9, in, The operations also include: determining a pixel distance between the vertical centerline of the trailer representation and a vertical centerline of the one or more images that passes through a center defined by the vertical centerline of the one or more images and a horizontal centerline of the one or more images; When the pixel distance between the vertical centerline of the trailer representation and the vertical centerline of the one or more images is less than a threshold: setting, by the data processing hardware, the vertical centerline of the trailer representation as the target; and The first steering wheel angle is determined by the data processing hardware to steer the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target.

11. The system according to claim 9, in, The threshold includes a plurality of pixels.

12. The system according to claim 9, in, The second steering wheel angle for steering the towing vehicle so that the vehicle autonomously maneuvers in a direction toward the target is a maximum steering wheel angle that prevents the towing vehicle and the trailer from V-bending.

13. The system according to claim 9, further comprising: include: The trailer representation is bounded by a bounding box, wherein the vertical centerline of the trailer representation comprises a line that vertically divides the trailer representation into two equal sections.

14. The system according to claim 9, in, The vertical centerline of the one or more images comprises a line that vertically divides each of the one or more images into two equal segments, each segment having an equal number of pixels.

15. The system according to claim 9, in, The one or more cameras include a monocular camera.

16. The system according to claim 9, in, The one or more cameras include a fisheye camera.

Images