Obstacle detection for trailer turning

Abstract

The device may include a camera positioned at the rear of the vehicle, configured to capture images of the trailer and images of a scene containing objects, and further configured to generate and output image data corresponding to the scene. The device may also include a controller mounted on the vehicle, which includes an input / output interface, memory, and an electronic processor, the electronic processor being configured to receive image data from the camera, analyze objects in the scene, calculate the relative positions of the trailer and the objects, determine that an object is an obstacle using an obstacle detection algorithm, and determine if the obstacle exceeds a proximity threshold, and in response to the determination that the obstacle exceeds the proximity threshold, the controller controls the vehicle.

Description

Technical Field

[0001] Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 520,016, filed Aug. 16, 2023, the entire content of which is incorporated herein by reference.

[0002] Technical Field Each embodiment described herein relates to an obstacle detection system for a trailer.

Background Art

[0003] Summary Many vehicles include a radar system for detecting and monitoring blind spots of the vehicle. Additionally, many vehicles include the ability to carry a trailer. The radar capabilities of a vehicle do not always cover the additional blind spots introduced by the trailer. It would be desirable for a vehicle to include additional extended details for these blind spots. Thus, the embodiments described herein provide systems and methods for detecting, among other things, blind spots of a vehicle.

Summary of the Invention

Means for Solving the Problems

[0004] In some embodiments, the technology described herein relates to an obstacle detection system for a trailer connected to and towed by a vehicle, the system comprising a camera positioned at the rear of the vehicle, the camera configured to capture images of the trailer and images of a scene containing objects, the camera further configured to generate and output image data corresponding to the scene, the system comprising a controller mounted on the vehicle, the controller comprising an input / output interface, memory and an electronic processor, the electronic processor configured to receive image data from the camera, analyze objects in the scene, calculate the relative positions of the trailer and the objects, determine that an object is an obstacle using an obstacle detection algorithm, and determine that the obstacle exceeds a proximity threshold, and in response to the determination that the obstacle exceeds a proximity threshold, the controller controls the vehicle.

[0005] In some embodiments, the technology described herein relates to a system in which an electronic processor further analyzes image data to determine object data points and background data points. In some embodiments, the technology described herein relates to a system in which an electronic processor determines whether an object is an obstacle based on object data points and whether an object is not an obstacle based on background data points. In some embodiments, the technology described herein relates to a system in which an electronic processor further calculates the 3D position of a trailer, the instantaneous joint angle of the trailer, and the position of an object relative to the position of the trailer.

[0006] In some embodiments, the technology described herein relates to a system in which a proximity threshold is a distance of less than 1 meter, and in response to a determination that an obstacle exceeds the proximity threshold, the controller stops the vehicle. In some embodiments, the technology described herein relates to a system in which an electronic processor further calculates the instantaneous articulation angles of multiple corners of the trailer relative to an object. In some embodiments, the technology described herein relates to a system in which the proximity threshold is a range of distances between an object and the trailer, and the range of distances includes a range between 5 centimeters and 1 meter.

[0007] In some embodiments, the technology described herein relates to a system in which an electronic processor further determines that an obstacle exceeds a first proximity threshold and a second proximity threshold, the second proximity threshold being the closer proximity between the trailer and the obstacle, and in response to the obstacle exceeding the first proximity threshold, the controller generates an alarm, and in response to the obstacle exceeding the second proximity threshold, the controller controls the vehicle.

[0008] In some embodiments, the techniques described herein relate to a system in which an obstacle detection algorithm includes creating a coordinate plot of image data that includes object data points and background data points.

[0009] In some embodiments, the technique described herein is an obstacle detection method for a trailer connected to and towed by a vehicle, the method comprising: generating an image of the trailer and an image of a scene containing an object using a camera positioned at the rear of the vehicle; outputting image data corresponding to the scene using the camera; receiving the image data using an electronic processor; analyzing the object in the scene using the electronic processor; calculating the relative positions of the trailer and the object using the electronic processor; determining that the object is an obstacle using an obstacle detection algorithm using the electronic processor; determining that the obstacle exceeds a proximity threshold using the electronic processor; and controlling the vehicle in response to the determination that the obstacle exceeds a proximity threshold using the electronic processor.

[0010] In some embodiments, the techniques described herein further include a method by which an electronic processor analyzes image data to determine object data points and background data points. In some embodiments, the techniques described herein further include a method by which an electronic processor calculates instantaneous joint angles of multiple corners of a trailer relative to an object.

[0011] In some embodiments, the technique described herein further includes using an electronic processor to calculate a range of distances between an object and a trailer, wherein the range of distances includes a range between 5 centimeters and 1 meter.

[0012] In some embodiments, the technique described herein further includes determining by an electronic processor that an obstacle exceeds a first proximity threshold and a second proximity threshold, the second proximity threshold being the closer proximity between the trailer and the obstacle; generating an alarm by the electronic processor in response to the obstacle exceeding the first proximity threshold; and controlling the vehicle by the electronic processor in response to the obstacle exceeding the second proximity threshold.

[0013] In some embodiments, the techniques described herein further include generating a coordinate plot of image data containing object data points and background data points using an electronic processor.

[0014] In some embodiments, the technology described herein is an obstacle detection system for a trailer connected to and towed by a vehicle, the system including a camera positioned at the rear of the vehicle, the camera configured to capture images of the trailer and images of a scene containing objects, the camera further configured to generate and output image data corresponding to the scene, the system including a controller mounted on the vehicle, the controller including an input / output interface, memory, and an electronic processor, the electronic processor receiving image data from the camera, generating object data points and background data points from the image data, and including object data points and background data points The system generates a coordinate plot of the image data, uses object data points and background data points to calculate the positions of multiple corners of the trailer, calculates the instantaneous joint angles of the multiple corners of the trailer relative to the object, uses the instantaneous joint angles to calculate the 3D position of the trailer relative to the object, uses an obstacle detection algorithm to determine if the object is an obstacle, and is configured to determine if the obstacle exceeds a first proximity threshold and a second proximity threshold, where the second proximity threshold is the closer proximity between the trailer and the obstacle, and in response to the obstacle exceeding the first proximity threshold, the controller generates an alarm, and in response to the obstacle exceeding the second proximity threshold, the controller controls the vehicle.

[0015] In some embodiments, the technology described herein relates to a system in which an electronic processor determines whether an object is an obstacle based on object data points and whether an object is not an obstacle based on background data points. In some embodiments, the technology described herein further relates to a system in which an electronic processor calculates the instantaneous articulation angle of a trailer corner relative to an object, based on the dimensions of the trailer and the position of the trailer corner.

[0016] In some embodiments, the technology described herein relates to a system in which a first proximity threshold is a range of distances between an object and a trailer, the range of distances includes a range between 5 centimeters and 1 meter, but is greater than a second proximity threshold, and a second proximity threshold is a range of distances between an object and a trailer, the range of distances includes a range between 5 centimeters and 1 meter, but is smaller than a first proximity threshold.

[0017] In some embodiments, the technology described herein relates to a system in which, in response to a determination that an obstacle exceeds a first proximity threshold, the controller generates a warning to the vehicle driver indicating that the trailer and the obstacle are at risk of collision, and in response to a determination that the obstacle exceeds a second proximity threshold, the controller stops the vehicle.

[0018] Other aspects, features, and embodiments will become apparent from the detailed description and the accompanying drawings. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram of an obstacle detection system for trailer turning in several configurations. [Figure 2] These are images of the system shown in Figure 1 in several different configurations. [Figure 3] This is a block diagram of the system in Figure 1 in several different configurations. [Figure 4] This is a graph of objects detected by obstacle detection systems for trailer turning in various configurations. [Figure 5] This is a flowchart of the obstacle detection process for trailer turning in several configurations. [Modes for carrying out the invention]

[0020] Detailed explanation FIG. 1 is a diagram of an obstacle detection system for trailer turning according to some embodiments. System 100 includes vehicle 105 and trailer 110 attached to vehicle 105 by a hitch. Vehicle 105 has an onboard controller 115. In the illustrated example, controller 115 includes electronic processor 120, input / output interface 125, and memory 130. In some examples, electronic processor 120 is implemented as a microprocessor having a separate memory, such as memory 130. In other examples, electronic processor 120 may be implemented as a microcontroller (having memory 130 on the same chip). In other examples, electronic processor 120 may be implemented using multiple processors. Additionally, electronic processor 120 may be implemented partially or wholly as, for example, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and the like, and accordingly, memory 130 may be unnecessary or may be changed.

[0021] In some examples, memory 130 includes a non - transient computer - readable memory that stores instructions received and executed by electronic processor 120 to implement the methods described in this specification that include an obstacle detection method. Memory 130 may include, for example, a program storage area and a data storage area. The program storage area and the data storage area may each include a combination of different types of memory, such as read - only memory and random access memory. Input / output interface 125 may include one or more input mechanisms and one or more output mechanisms (e.g., general - purpose input / output (GPIO), controller area network (CAN) bus interface, analog input digital input, and the like).

[0022] Software used during the operation of the vehicle 105 is stored in the memory 130. For example, an obstacle detection algorithm 135 (also referred to as algorithm 135) can be stored in the memory 130 or at a separate memory location. In some examples, the illustrated components can be combined or divided into separate software, firmware, and / or hardware. For example, instead of placing logic and processing within a single electronic processor and performing them by a single electronic processor, they can be distributed among multiple electronic processors. Hardware and software components may be arranged as being on the same computing device regardless of how they are combined or divided, or may be distributed among different computing devices connected by one or more networks or other suitable communication links.

[0023] The vehicle 105 also includes a sensor 140. The sensor 140 may be a speed sensor, an accelerometer, a radar sensor, a LIDAR sensor, or the like. The vehicle 105 also includes a camera 145 configured to capture an image of a trailer 110 connected to the vehicle 105. For example, in one embodiment, the camera is assembled near the rear trailer hitch of the vehicle and is angled downward to capture a wide view of the area behind the vehicle. The vehicle CAN bus 150 electronically and communicatively connects the camera 145, the sensor 140, and the vehicle controller 115 to each other.

[0024] Figure 2 shows image 200 from camera 145 connected to vehicle 105. As previously mentioned, camera 145 is angled downwards to capture a wide field of view. Image 200 also shows trailer 110, obstacle 205, road surface 210, trailer chassis 215, vehicle hitch 220, and trailer corner 225. The size of the trailer, including the position of the trailer corner 225 relative to the trailer chassis 215 and vehicle hitch 220, will be used later by algorithm 135. The camera records image 200 and stores the image data in controller 115, where the obstacle detection algorithm 135 analyzes the image data using Structure from Motion (SfM) technique to pinpoint the exact location of the obstacle 205 relative to trailer 110. Objects detected by the camera may include fences, cones, curbs, other vehicles, road surfaces, or any other objects.

[0025] In image 200, object data points 230 and background data points 235 can be seen in addition. Object data points 230 indicate objects that the obstacle detection algorithm 135 has determined to be within a dangerous proximity threshold relative to the trailer 110. For example, if an object is determined to be beyond the dangerous proximity threshold, the obstacle detection algorithm 135 classifies that object as an obstacle 205. However, if the object is not beyond the dangerous proximity threshold, the algorithm 135 does not classify that object as an obstacle, as indicated by the background data points 235. In some examples, when the algorithm 135 detects an obstacle 205, the controller 115 is instructed to warn the driver of the proximity to the obstacle 205, or to control the vehicle 105 to avoid a collision between the obstacle 205 and the trailer 110. This method will be described in more detail below and shown in Figure 5.

[0026] Figure 3 is a block diagram 300 of top views of the vehicle 105 and trailer 110 in several embodiments. Figure 300 includes similar configurations of the vehicle 105, trailer 110, and obstacle 205 captured in image 200. As the vehicle 105 turns around the detected obstacle 205, the joint angle Θ (theta) changes. For example, when the vehicle 105 and the trailer are parallel, such as when the vehicle is moving in a straight line, the joint angle is approximately 0 degrees. The joint angle Θ is used by algorithm 135 to identify the proximity of objects and further determine whether an object is an obstacle 205. For example, algorithm 135 can use known dimensions of the trailer 110 (e.g., trailer length, trailer height, chassis length, and similar) to calculate the instantaneous joint angle Θ and determine the real-world 3D position of the trailer's corners. This identification helps algorithm 135 calculate the proximity of the obstacle 205 to the trailer 110.

[0027] Figure 4 is a graph 400 showing the 2D coordinates of an object detected in a camera image in several embodiments. Graph 400 is a coordinate plot of the image 200 shown in Figure 2 and includes the same data points as the image 200. Graph 400 includes the X-axis 305 and the Y-axis 310, along with object data points 230 and background data points 235. For example, a Y-coordinate of 0 along the Y-axis 310 corresponds to the center line of the image 200. Similarly, obstacles 205 and object data points 230 in the image 200 are represented on graph 400. Certain patterns of data points on graph 400 can indicate the distance between the object and the trailer 110, or that the object is an obstacle 205. For example, object data points 230 are clustered on graph 400, while background data points 235 are not clustered on graph 400. Another image captured by camera 145 generates a different graph containing different data points. As camera 145 captures images over time and vehicle 105 tows trailer 110, the data points generate different patterns, which are used by algorithm 135 for obstacle detection and proximity determination.

[0028] Figure 5 is a flowchart of the obstacle detection process for trailer turning in several embodiments. Process 500 begins in step 505 when vehicle 105 tows trailer 110. The process continues to step 510 when a camera 145 mounted on the vehicle records a wide-angle view of the trailer 110. Camera 145 captures multiple image frames and generates image data corresponding to the image scene. In step 515 of process 500, the image data is transferred from camera 145 to controller 115 and processed by electronic processor 120. The process continues to step 520 when electronic processor 120 executes obstacle detection algorithm 135. Each image captured by camera 145 is passed through algorithm 135 so that each frame is analyzed individually. Algorithm 135 uses methods such as Structure from Motion (SfM) to determine the 3D position of objects captured by the 2D images. Furthermore, algorithm 135 analyzes the image data and categorizes the object data points 230 and background data points 235. Algorithm 135 additionally calculates the 3D position of the trailer corners 225 and the instantaneous joint angles of the trailer. Using these calculated positions together with the known dimensions and image data of the trailer as described above, algorithm 135 proceeds to step 525 of process 500 to determine whether any of the detected objects are obstacles such as obstacle 205. If algorithm 135 determines that none of the objects in image 200 are obstacles 205, the process returns to step 510 where a new camera image is acquired. On the other hand, if algorithm 135 determines that an object is an obstacle 205, the process proceeds to step 530 where the algorithm calculates the proximity of the obstacle 205 to the trailer 110. Process 500 continues to step 535 where algorithm 135 determines whether the detected obstacle 205 exceeds a dangerous proximity threshold. The dangerous proximity threshold may be, for example, a distance range between 5 centimeters and 1 meter. If the object does not exceed the dangerous proximity threshold, the process returns to step 510, in which a new camera image is captured.However, if a dangerous proximity threshold is exceeded, the process proceeds to step 540, in which the controller 115 is configured to control one aspect of the vehicle. The controller 115 can generate a warning output by the input / output interface 125, thereby alerting the driver of the vehicle that the trailer 110 and the obstacle 205 are at risk of collision. In some examples, the controller 115 can control the vehicle to slow down, stop, or otherwise avoid a collision between the trailer 110 and the obstacle 205. In some examples, multiple proximity thresholds are provided. For example, there may be a first proximity threshold and a second proximity threshold different from the first proximity threshold, where exceeding the first proximity threshold generates a warning to the driver, and exceeding the second proximity threshold causes the controller to control the vehicle. In some examples, the second proximity threshold is a closer distance between the obstacle and the trailer than the first proximity threshold.

[0029] Accordingly, various embodiments of the systems and methods described herein provide, in particular, techniques for detecting obstacles around a trailer. Other features and advantages of the present invention are described in the following claims.

[0030] The above specification has described specific examples. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of the invention as described in the following claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive, and all such modifications are intended to be within the scope of this teaching.

[0031] No benefit, advantage, solution to a problem, or any element that could produce or enhance any benefit, advantage, or solution should be construed as an essential, necessary, or indispensable feature or element of any or all claims. The present invention is defined solely by the appended claims, which include any modifications made during the pendency of this application and all equivalents of those claims issued.

[0032] Furthermore, terms used herein to express relationships such as "first" and "second," "above" and "below," and similar terms may be used solely to distinguish one entity or action from another entity or action, and do not necessarily require or suggest any actual such relationship or order between such entities or actions. The terms "equip," "have," "possess," "include," "contain," "contain," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that equips, has, includes, or contains a list of elements may not only include those elements but also other elements not explicitly listed, or other elements inherent to such process, method, article or apparatus.

[0033] The element preceding “equipped with,” “having,” “containing,” or “containing” does not preclude the existence of additional identical elements in a process, method, article, or apparatus that equips, has, includes, or contains that element, unless further constraints apply. The indefinite articles “a” and “an” are defined as one or plural unless otherwise specified herein. The terms “substantially,” “essentially,” “almost,” “about,” “approximately,” or any other variation thereof are defined as approximate as understood by those skilled in the art, and in one non-limiting example, the term is defined as no more than 10%, in another no more than 5%, in yet another no more than 1%, and in yet another no more than 0.5%. As used herein, the term “combined” is defined as being connected, but not necessarily direct or mechanical. An apparatus or structure “configured” by a particular method is configured by at least such method, but may be configured by methods not listed.

Claims

1. An obstacle detection system for a trailer connected to and towed by a vehicle, The system comprises a camera positioned at the rear of the vehicle, the camera configured to capture images of the trailer and images of a scene containing objects, and the camera is further configured to generate image data corresponding to the scene and output the image data. The system comprises a controller mounted on the vehicle, the controller including an input / output interface, memory, and an electronic processor. The aforementioned electronic processor, The image data is received from the aforementioned camera. Analyze the object in the aforementioned scene, The relative positions of the trailer and the object are calculated, An obstacle detection algorithm is used to determine that the object is an obstacle. Determining whether the obstacle exceeds the proximity threshold It is configured in such a way, In response to the determination that the obstacle exceeds the proximity threshold, the controller controls the vehicle.

2. The electronic processor further analyzes the image data to determine object data points and background data points. The system according to claim 1.

3. The electronic processor determines whether the object is an obstacle based on the object data points and whether the object is not an obstacle based on the background data points. The system according to claim 2.

4. The electronic processor further calculates the 3D position of the trailer, the instantaneous joint angle of the trailer, and the position of the object relative to the position of the trailer. The system according to claim 1.

5. The aforementioned proximity threshold is a distance of less than 1 meter. In response to the determination that the obstacle exceeds the proximity threshold, the controller stops the vehicle. The system according to claim 1.

6. The electronic processor further calculates the instantaneous joint angles of multiple corners of the trailer relative to the object. The system according to claim 1.

7. The proximity threshold is a range of distances between the object and the trailer, and the range of distances includes a range between 5 centimeters and 1 meter. The system according to claim 1.

8. The electronic processor further determines that the obstacle exceeds a first proximity threshold and a second proximity threshold, the second proximity threshold being the closer proximity between the trailer and the obstacle. In response to the obstacle exceeding the first proximity threshold, the controller generates an alarm. In response to the obstacle exceeding the second proximity threshold, the controller controls the vehicle. The system according to claim 1.

9. The obstacle detection algorithm includes creating a coordinate plot of the image data, which includes object data points and background data points. The system according to claim 1.

10. An obstacle detection method for a trailer connected to and towed by a vehicle, The aforementioned method, A camera positioned at the rear of the vehicle generates an image of the trailer and an image of the scene containing the object. The camera outputs image data corresponding to the scene, The electronic processor receives the image data, The electronic processor analyzes the objects in the scene, The electronic processor calculates the relative positions of the trailer and the object, The aforementioned electronic processor determines that the object is an obstacle using an obstacle detection algorithm, The electronic processor determines that the obstacle exceeds the proximity threshold, Equipped with, A method for controlling the vehicle in response to the determination by the electronic processor that the obstacle exceeds the proximity threshold.

11. The method further comprises analyzing the image data using the electronic processor to determine object data points and background data points. The method according to claim 10.

12. The method further comprises using the electronic processor to calculate the instantaneous joint angles of multiple corners of the trailer relative to the object. The method according to claim 10.

13. The method further comprises calculating a range of distances between the object and the trailer using the electronic processor, The aforementioned distance range includes the range between 5 centimeters and 1 meter. The method according to claim 10.

14. The aforementioned method, The electronic processor determines that the obstacle exceeds a first proximity threshold and a second proximity threshold, wherein the second proximity threshold is the closer proximity between the trailer and the obstacle. The electronic processor generates an alarm in response to the obstacle exceeding the first proximity threshold, The electronic processor controls the vehicle in response to the obstacle exceeding the second proximity threshold, Furthermore, The method according to claim 10.

15. The method further comprises generating a coordinate plot of the image data, which includes object data points and background data points, using the electronic processor. The method according to claim 10.

16. An obstacle detection system for a trailer connected to and towed by a vehicle, The system comprises a camera positioned at the rear of the vehicle, the camera configured to capture images of the trailer and images of a scene containing objects, and the camera is further configured to generate image data corresponding to the scene and output the image data. The system comprises a controller mounted on the vehicle, the controller including an input / output interface, memory, and an electronic processor. The aforementioned electronic processor, The image data is received from the aforementioned camera. Object data points and background data points are generated from the aforementioned image data. A coordinate plot of the image data, which includes object data points and background data points, is generated. Using the object data points and background data points, the positions of multiple corners of the trailer are calculated. The instantaneous joint angles of the multiple corners of the trailer relative to the object are calculated. Using the instantaneous joint angle, the 3D position of the trailer relative to the object is calculated. An obstacle detection algorithm is used to determine that the object is an obstacle. It is determined that the obstacle exceeds the first proximity threshold and the second proximity threshold. The system is configured such that the second proximity threshold is the closer proximity between the trailer and the obstacle. In response to the obstacle exceeding the first proximity threshold, the controller generates an alarm. A system in which the controller controls the vehicle in response to the obstacle exceeding the second proximity threshold.

17. The electronic processor determines whether the object is an obstacle based on the object data points and whether the object is not an obstacle based on the background data points. The system according to claim 16.

18. The electronic processor further calculates the instantaneous joint angle of the corner of the trailer relative to the object, based on the dimensions of the trailer and the position of the corner of the trailer. The system according to claim 16.

19. The first proximity threshold is a range of distances between the object and the trailer, and this range includes a range between 5 centimeters and 1 meter, but is greater than the second proximity threshold. The second proximity threshold is a range of distances between the object and the trailer, the range of distances includes a range between 5 centimeters and 1 meter, but is smaller than the first proximity threshold. The system according to claim 16.

20. In response to the determination that the obstacle exceeds the first proximity threshold, the controller generates a warning to the vehicle driver indicating that the trailer and the obstacle are at risk of collision. In response to the determination that the obstacle exceeds the second proximity threshold, the controller stops the vehicle. The system according to claim 16.

Images