Panning camera with trailer length and object detection for mirror-less vehicles

Abstract

Presented herein are embodiments of pannable cameras with trailer length and object detection for camera-equipped vehicles to supplement or replace mirrors commonly used nowadays. In one embodiment, a vehicle controller is provided, comprising: one or more processors; a non-transitory computer readable medium; and program instructions stored on the non-transitory computer readable medium. The program instructions, when executed by the one or more processors, cause the one or more processors to: cause a side view camera on a towing vehicle to capture an image (s) of a front edge, a lower edge, an upper edge, and / or a rear edge of a trailer coupled to the towing vehicle; and estimating the length of the trailer based on the image (s) captured by the side view camera. Other embodiments are provided.

Description

Background Technology

[0001] Some heavy commercial vehicles configured for towing trailers may be equipped with one or more cameras. For example, cameras may be used as a supplement to or alternative to front and / or side mirrors to allow the driver to see the sides of the vehicle. Attached Figure Description

[0002] Figure 1 This is a block diagram of the components of the vehicle in the embodiment.

[0003] Figure 2 This is a flowchart of a method for an embodiment of trailer length and object detection.

[0004] Figure 3 This is an illustration of a display screen showing an image from a side-view camera. Summary of the Invention

[0005] The following embodiments generally relate to vehicles equipped with cameras to supplement or replace mirrors commonly used today. In one embodiment, a vehicle controller is provided, comprising: one or more processors; a non-transitory computer-readable medium; and program instructions stored on the non-transitory computer-readable medium. The program instructions, when executed by the one or more processors, cause the one or more processors to: cause a side-view camera on a tractor unit to capture images of one or more of the front, lower, upper, and / or rear edges of a trailer coupled to the tractor unit; and to estimate the length of the trailer based on the images captured by the side-view camera(s).

[0006] In another embodiment, a method is provided to be performed in a vehicle including a tractor, a trailer towed by the tractor, and at least one image capture device. The method includes: capturing one or more images of the trailer(s) using the at least one image capture device; and analyzing the images of the trailer(s) to estimate the length of the trailer(s).

[0007] In yet another embodiment, a tractor unit is provided, comprising: at least one image capturing device; and a component for estimating the length of a trailer towed by the tractor unit based on images of the front edge, lower edge, upper edge, and / or rear edge of the trailer captured by the at least one image capturing device.

[0008] Other embodiments are possible, and each of the embodiments can be used alone or in combination. Detailed Implementation

[0009] Now turn to the attached image. Figure 1This is a block diagram of example components of a vehicle 100 according to an embodiment. In one embodiment, vehicle 100 is a highway / off-highway vehicle as described in NHTSA Title 49, Part B, Chapter V, Section 571, Subsection A, Section 571.3, wherein the vehicle may be one or more of the following in a reasonable combination: “truck,” “truck tractor,” “trailer,” “full trailer,” “semi-trailer tractor,” “semi-trailer,” and / or “long cargo trailer.” The term “cab” is sometimes used herein, referring to the part of the tractor in which the vehicle operator typically controls the vehicle. Further details of the typical or necessary inputs and outputs provided to the vehicle operator are not the subject of this patent.

[0010] like Figure 1 As shown, the vehicle 100 in this example includes one or more of the following: a vehicle controller 101, which includes one or more processors 102; one or more outward-facing image capture devices (e.g., one or more cameras) 104; one or more driver-facing image capture devices 106; a transceiver 108; user input / output (I / O) devices 110 (e.g., microphone, speaker, touchscreen, indicator lights, etc.); a braking system 112; an engine system 114; a Global Positioning System (GPS) 116 capable of identifying the vehicle's location; and one or more memories 118. The image capture device can be any device capable of digitizing images (e.g., camera, LiDAR, radar, ultrasound, etc., or any combination thereof). In one embodiment, the outward-facing image capture device 104 is an alternative / complement to a forward-facing mirror and / or one or more side mirrors. Furthermore, the vehicle 100 may have other devices, including but not limited to forward-facing cameras, radar, LiDAR, and environmental sensors.

[0011] These various components can communicate with each other directly or indirectly (through components that may or may not be shown or described herein) via wired (e.g., Controller Area Network (CAN), Ethernet, Automotive Ethernet, Power Line Communication (PLC), etc.) or wireless (e.g., WiFi, Bluetooth, Cellular Network, etc.) connections. Figure 1 In the example, the components are connected directly or indirectly via bus 150, which can be in the form of a controller area network (CAN).

[0012] One or more memories 118 (e.g., one or more non-transitory computer-readable media) store computer-readable program code 120. These one or more memories 118 may be of the same or different types, and may be part of the same memory device or different memory devices. For example, some or all of the memories 118 may be volatile or non-volatile non-transitory memories, solid-state memories, flash memory, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and variations and combinations thereof.

[0013] One or more processors 102 can execute computer-readable program code 120, which may have instructions (e.g., modules, routines, subroutines, programs, applications, etc.) that, when executed by one or more processors 102, cause one or more processors 102 to perform certain functions, such as those discussed herein, and other functions not described herein. It should be noted that one or more processors 102 may have different functions (e.g., a first subset of one or more processors may be used for certain functions, while a second subset of one or more processors may be used for other functions). One or more processors 102 may also be implemented entirely in hardware (e.g., application-specific integrated circuits (ASICs)).

[0014] As described above, some heavy commercial vehicles configured for towing trailers may be equipped with one or more cameras. For example, one or more outward-facing cameras 104 may serve as an alternative to one or more side mirrors to allow the driver to see the sides of the vehicle(s), or may enhance the side mirrors. The following embodiments utilize this technological shift from traditional mirrors to camera-based systems to provide additional functionality previously unavailable. Such functionality may include trailer length and object detection, as described below.

[0015] Currently, when a driver is in the cab using only traditional mechanical side mirrors, it is difficult to determine the length of the trailer. Furthermore, when the driver is in the cab, it may be difficult to adjust the mechanical mirrors. Therefore, if the vehicle configuration changes due to adding or removing trailers, the driver may find it difficult to maintain optimal visibility. In addition, traditional mirrors do not allow any kind of additional detail to be displayed on their surface to inform the driver of important potential events.

[0016] The following embodiments utilize a camera as a side mirror, positioned at a fixed point on the side of the cab. In one embodiment, depending on the technology used in the camera system, the side mirror has two different operating modes for determining the trailer length. For example, if the camera has a wide viewing angle, it can digitally pan to show only the area of ​​interest to the driver in the cab (e.g., on a screen mounted on the A-pillar). However, if the camera does not have a wide viewing angle, it can mechanically pan to the same point of interest, thus showing that area to the driver in the cab. In a dual-camera system (e.g., one on the driver's side and one on the passenger's side), as the cameras pan, they know their position on the side of the truck, and once they have observed the rear edge of the trailer, they will be able to determine their panning angle.

[0017] Then, processor(s) 102 can use trigonometric calculations to determine the trailer length in the field of view. If multiple trailers are towed, processor(s) 102 can repeat this task, stopping each time to record the observed trailer length. Once all trailers have been taken into account, the camera can position itself in a location (e.g., an ideal location) to maintain as much road visibility as possible, thus adapting to changes in the traffic system as needed.

[0018] In one embodiment, when the vehicle is started with a key or when a detected change in the vehicle system indicates the addition or removal of a trailer, the side-view mirror camera can pan from its stationary position until a first rear edge is detected. One or more processors 102 can calculate the length of one trailer and then continue calculating its length if a second trailer is present. Once all trailers have been measured, the camera can (e.g., as commanded by one or more processors 102) position itself at a viewing angle (e.g., an ideal viewing angle) and monitor that area, thereby transmitting relevant information to the driver.

[0019] Figure 2 This is a flowchart 200 of an embodiment of a method for trailer length and object detection. (See flowchart 200 for example.) Figure 2As shown, after the vehicle is ignited (action 205), the cameras on the vehicle (e.g., one camera as the driver's side mirror and another as the passenger's side mirror) are activated from their initial (e.g., innermost) position (action 210). The cameras then pan outwards until the end of (one or more) trailers is detected (action 215) (for high-resolution cameras, panning can be mechanical or digital). Image processing performed by (one or more) processors 102 can identify key features, which may include, but are not limited to: the front vertical edge of the trailer, the rear vertical edge of the trailer, the lower horizontal edge of the trailer marked with reflective tape, the lower horizontal edge of the trailer marked with "scale" reflective tape, the lower horizontal edge of the trailer without reflective tape, optical character recognition (OCR) of trailer dimensions marked at predetermined locations on the trailer (e.g., the front and / or sides of the trailer), quick response (QR) codes or barcodes describing trailer dimensions, and / or QR codes or barcodes identifying the trailer in a database.

[0020] Next, processor(s) 102 may determine whether more than one trailer has been detected (action 220). If more than one trailer is detected, the panning process is repeated to calculate the length of each trailer and the total number of trailers (action 225). If no more than one trailer is detected, processor(s) 102 may estimate the trailer length based on the camera's viewing position, the turning angle, and the viewing arc between the front and rear edges of the trailer(s) in the field of view (action 230). The turning angle can be determined by configuration and sensor inputs, which may include, but are not limited to, steering angle sensors, wheel speeds, vehicle gradient, wheelbase, and / or track width.

[0021] Next, the camera is positioned to facilitate driver interaction (e.g., the optimal position) (Action 235). Then, processor(s) 102 can determine whether the object of interest (e.g., a person, car, motorcycle, debris, etc.) is within the camera's viewing angle (Action 240). The object size, position, and relative speed can be scaled relative to the trailer size, position, and speed. If the object of interest is not within the camera's viewing angle, processor(s) 102 continues to monitor the camera's viewing angle (Action 245). However, if the object of interest is within the camera's viewing angle, processor(s) 102 can display additional boxes, outlines, or other markings around or near the object of interest on one or more screens in the vehicle's cab (Action 250).

[0022] Then, processor(s) 102 can determine whether the object is detected as being too close to the vehicle (e.g., within a certain threshold distance) (action 255). If processor(s) 102 determines that the object is not too close to the vehicle, processor(s) 102 can continue monitoring the process (action 260). However, if processor(s) determine that the object is too close to the vehicle, processor(s) 102 can transmit a possible collision to the driver's dashboard (action 265).

[0023] The following is an example of a method for measuring trailer length using a single camera. It should be noted that this is merely an example, and other implementations can be used. Furthermore, while this example is described using a single camera, other methods can utilize more than one camera.

[0024] This example method will combine Figure 3 To discuss, Figure 3 A display screen 300 is shown, displaying the output of a driver-side camera. The display screen 300 is located in the vehicle's cab, and the displayed output shows the trailer 310 of the vehicle. The lower edge of the trailer 310 is marked with reflective tape 320 (e.g., DOT C2 reflective tape, which alternates between red and white and appears silver during the day). Reflective tape 320 is typically used for trailer classification of commercial vehicles weighing over 10,000 pounds and exceeding 80 inches in width. (DOT C2 tape has a standard height of two inches, and this standard height is used to establish trailer length; this would not be the case if the tape had an unknown and non-standard height).

[0025] One or more processors 102 can analyze the displayed image for the pixel height of the reflective strip 320 at its closest and farthest points in the image. In this example, there is a two-inch transition ratio when processing the image along the lower edge of the trailer 310. In the first pass, the total number of pixels from the beginning to the end of the trailer edge in the field of view is counted and recorded as key pixels for the lower trailer edge measurement. The pixel ratio factor can be determined by the following formula: .

[0026] The distance between two pixels at the nearest edge (front) of trailer 310 can be calculated and recorded using the following formula: .

[0027] In the second pass, processor(s) 102 can process the image by counting key pixels identified as the lower trailer edge, from the beginning to the end of the trailer edge in the field of view. The distance between each pair of pixels can be calculated and summed as shown in the following formula: .

[0028] It should be noted that this method assumes the image is captured with sufficient detail (e.g., number of pixels) to produce results without significant errors. Lower resolution images may require interpolation / interpretation of more pixels. Alternatively, a multi-pixel step can be used instead of a single-pixel step. Furthermore, this method only demonstrates one example technique for using a single camera to analyze an image of a trailer and determine its length using image analysis. Other techniques can also be used, as mentioned above. Additionally, the estimated length can be used in conjunction with parameters of the brake controller. Moreover, the calculated trailer length can be adjusted to take into account the pivot point of the trailer image, as this can represent the kingpin position on the trailer. The determined length can be considered as the length from the fifth wheel connection position of the tractor-trailer.

[0029] It should be understood that all embodiments provided in this detailed description are merely examples and other implementations may be used. Therefore, any components, architectures, or other details presented herein should not be interpreted as belonging to the claims unless expressly set forth in the claims. Furthermore, it should be understood that components shown or described as being “coupled” (or “communicating”) with each other may be directly coupled (or communicate) with each other, or indirectly coupled (or communicate) with each other through one or more components that may or may not be shown or described herein.

[0030] The above detailed description is intended to be understood as an illustration of selected forms that the invention may take, and not as a limitation thereof. Only the appended claims, which include all equivalents, are intended to define the scope of the claimed invention. Therefore, any components, architectures, or other details presented herein should not be interpreted as such unless expressly set forth in the claims. Finally, it should be noted that any aspect of any embodiment described herein can be used alone or in combination with each other.

Claims

1. A vehicle controller, comprising: One or more processors; Non-transitory computer-readable medium; as well as Program instructions stored on the non-transitory computer-readable medium, which, when executed by the one or more processors, cause the one or more processors to: This enables the side-view camera on the tractor to capture images of one or more of the front, lower, upper, and / or rear edges of the trailer coupled to the tractor; and The length of the trailer is estimated based on the images captured by the side-view camera(s).

2. The vehicle controller as claimed in claim 1, wherein, When the program instructions are executed by the one or more processors, the one or more processors also cause the one or more processors to: Detecting the object of interest captured by the side-view camera; and On the display device, a mark is displayed near the display of the object of interest.

3. The vehicle controller as claimed in claim 2, wherein, When the program instructions are executed by the one or more processors, the one or more processors also cause the one or more processors to: Determine whether the object of interest is within a threshold distance of the vehicle; and In response to determining that the object of interest is within the threshold distance of the vehicle, a driver alert is generated.

4. The vehicle controller as claimed in claim 1, wherein, The length of the trailer is estimated based on the observation position of the side-view camera, the turning angle, and / or the observation arc between the front edge and the rear edge of the trailer.

5. The vehicle controller as claimed in claim 4, wherein, The turning angle is determined based on the output of the steering angle sensor, the output of the wheel speed sensor, the gradient of the vehicle, the wheelbase indication, and / or the track width indication.

6. The vehicle controller as claimed in claim 1, wherein: The vehicle includes at least one auxiliary trailer coupled to the tractor; and When the program instructions are executed by the one or more processors, the one or more processors also cause the one or more processors to: The side-view camera on the tractor unit captures images of the front and rear edges(one or more) of the at least one attached trailer; and The length of the at least one additional trailer is estimated based on the images of the at least one additional trailer captured by the side-view camera.

7. The vehicle controller as claimed in claim 1, wherein, The side-view camera is configured to mechanically pan to capture images of one or more of the front and rear edges of the trailer.

8. The vehicle controller as claimed in claim 1, wherein, The side-view camera is configured to digitally pan to capture one or more images of the front and rear edges of the trailer.

9. The vehicle controller as claimed in claim 1, wherein, When the program instructions are executed by the one or more processors, the one or more processors also cause the one or more processors to: The second side-view camera on the tractor unit captures images of one or more of the front and rear edges of the trailer; and The length of the trailer is estimated based on the images(s) captured by the second side-view camera.

10. A method comprising: The following operations are performed in a vehicle comprising a tractor, a trailer towed by the tractor, and at least one image capturing device: The at least one image capturing device is used to capture one or more images of the trailer; as well as Analyze one or more of the images of the trailer to estimate the length of the trailer.

11. The method of claim 10, further comprising: On the display device, a mark is displayed near the detected object of interest.

12. The method of claim 11, further comprising: An alarm is generated in response to the object of interest being within a threshold distance of the tractor or the trailer.

13. The method of claim 12, wherein, The length of the trailer is estimated based on the markings of the front vertical edge of the trailer, the markings of the rear vertical edge of the trailer, the markings of the lower horizontal edge of the trailer marked with reflective strips, the markings of the lower horizontal edge of the trailer marked with graduated reflective strips, and / or the markings of the lower horizontal edge of the trailer without reflective strips.

14. The method of claim 12, wherein, The length of the trailer is estimated based on a mark on the trailer indicating the length of the trailer, a Quick Response (QR) code or barcode specifying the length of the trailer, and / or a QR code or barcode identifying the trailer in a database specifying the length of the trailer.

15. The method of claim 12, wherein, The length of the trailer is estimated based on the height of the nearest and farthest pixels of the reflective strip on the trailer in the image.

16. The method of claim 12, wherein, The image capture device includes a side-view camera.

17. The method of claim 17, wherein, The side-view camera is positioned on the driver's side of the vehicle, and the vehicle includes an additional side-view camera positioned on the passenger side of the vehicle.

18. A tractor unit, comprising: At least one image capture device; as well as A component for estimating the length of the trailer towed by the tractor based on images of the front and rear edges of the trailer captured by the at least one image capturing device.

19. The tractor unit as claimed in claim 18, further comprising: A component for displaying a mark on a display device near the display of a detected object of interest.

20. The tractor unit as claimed in claim 19, further comprising: A component for generating an alarm in response to the object of interest being within a threshold distance of the tractor or trailer.

Images