Image-based trailer wheelbase determination

Abstract

To accurately determine a wheelbase length of a vehicle.SOLUTION: A method of determining a trailer detection parameter of a trailer is provided, comprising receiving an image from at least one camera at a controller. The at least one camera defines a field of view including at least a portion of the vehicle trailer. The method further comprises determining a trailer angle of the vehicle trailer relative to a tractor, identifying at least one feature in an image of the trailer, determining a two-dimensional distance from the at least one feature to a predefined position on the image, and converting the two-dimensional distance to a three-dimensional distance based at least in part on the determined angle. The three-dimensional distance is the trailer detection parameter of the trailer.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present disclosure relates to camera mirror systems (CMS) for use in commercial trucks, and more particularly to a system and method for determining trailer wheelbase length using a CMS. [Background technology]

[0002] Mirror replacement systems and camera systems to supplement the mirror view are utilized in commercial vehicles to enhance the vehicle operator's ability to view the surrounding environment. Camera mirror systems (CMS) utilize one or more cameras to provide the vehicle operator with an extended field of view. In some instances, mirror replacement systems cover a wider field of view than a traditional mirror or include views that are not fully available in a traditional mirror.

[0003] In some maneuvers, such as trailer reversing maneuvers, knowledge of the true trailer angle, and the location of other trailer features, may prove particularly beneficial to the performance of automated and semi-automated vehicle systems. Purely tractor-based trailer angle positioning systems (i.e., systems for determining the relative trailer angle between the trailer and the tractor) exist, but many such systems rely on or otherwise utilize the particular length of the trailer. Connecting trailers of different lengths (e.g., connecting a 5m trailer when the system is expecting a 10m trailer) can result in inaccurate or erroneous determinations within the system, resulting in poor performance of vehicle systems that rely on the determinations.

[0004] In one particular example, a kinematic model that is dependent on trailer length becomes distorted if an incorrect trailer length is used. Summary of the Invention [Means for solving the problem]

[0005] An exemplary method for determining a trailer detection parameter of a trailer includes receiving, in a controller, an image from at least one camera defining a field of view that includes at least a portion of the vehicle trailer, determining a trailer angle of the vehicle trailer relative to the tractor, identifying at least one feature in the image of the trailer, determining a two-dimensional distance from the at least one feature to a predetermined location on the image, and converting the two-dimensional distance into a three-dimensional distance that is a trailer detection parameter of the trailer based at least in part on the identified angle.

[0006] In another example of the above method for determining a trailer detection parameter of a trailer, the determined trailer angle is in the range of 20 degrees to 70 degrees.

[0007] In another example of any of the above methods for determining a trailer sensing parameter of a trailer, the trailer sensing parameter is a wheelbase length and the at least one feature is at least one wheel.

[0008] In another example of any of the above methods for determining trailer detection parameters of a trailer, the at least one wheel includes a first wheel and a second wheel, and the two-dimensional distance from the at least one wheel to a predetermined location on the image is a distance from a midpoint between the first wheel and the second wheel on the image to the predetermined location.

[0009] In another example of any of the above methods for determining trailer detection parameters of a trailer, the at least one wheel is a single wheel and the two-dimensional distance from the at least one wheel to a predetermined location on the image is a distance from a center point of the wheel to the predetermined location on the image.

[0010] In another example of any of the above methods for determining a trailer detection parameter of a trailer, the trailer detection parameter is a trailer length from the hitch point to a rear end of the trailer, and the at least one feature is a feature located at the rear end of the trailer.

[0011] In another example of any of the above methods for identifying trailer detection parameters of a trailer, a feature located at the rear end of the trailer is the lower corner of the trailer.

[0012] In another example of any of the above methods for identifying trailer detection parameters of a trailer, converting a two-dimensional distance to a three-dimensional distance based at least in part on a specified angle involves cross-referencing the two-dimensional distance with the specified angle using a conversion. function including cross-referencing the two-dimensional distance with the specified angle using a conversion.

[0013] Another example of any of the above methods for identifying trailer detection parameters of a trailer further includes repeating the above method over a plurality of trailer angles in the range of 20 degrees to 70 degrees and storing each specified three-dimensional wheelbase length in memory.

[0014] Another example of any of the above methods for identifying trailer detection parameters of a trailer further includes filtering the three-dimensional wheelbase lengths stored using a mid-pass filter.

[0015] In another example of any of the above methods for identifying trailer detection parameters of a trailer, the memory stores the output of the mid-pass filter as the specified wheelbase length.

[0016] In another example of any of the above methods for identifying trailer detection parameters of a trailer, identifying at least one wheel within an image of the trailer includes identifying at least one wheel using an image-based object detection system stored in a controller.

[0017] In another example of any of the above methods for identifying trailer detection parameters of a trailer, the controller is one of a vehicle control unit and a camera mirror system controller.

[0018] In another example of any of the above methods for determining a trailer detection parameter of a trailer, the field of view is one of a class II view and a class IV view.

[0019] In one exemplary embodiment, a camera mirror system for a vehicle includes at least one camera defining a field of view that includes at least a portion of a trailer; a controller including a processor; and an object detection system. In and a controller including a memory that stores an image-based object detection module configured to identify at least one wheel, a trailer angle module configured to identify a trailer angle relative to the tractor, and a trailer detection parameter identification module configured to identify a trailer wheelbase length based on the trailer angle and images received from the at least one camera.

[0020] In another example of the camera mirror system for the vehicle, the field of view is one of a Class II and a Class VI view.

[0021] In another example of any of the above camera mirror systems for vehicles, the trailer detection parameter identification module includes instructions configured to cause the processor to determine a two-dimensional distance from at least one trailer parameter to a predetermined location on the image and convert the two-dimensional distance to a three-dimensional distance based at least in part on the determined angle, the three-dimensional distance being a wheelbase length of the trailer.

[0022] In another example of any of the above camera mirror systems for vehicles, converting the two-dimensional distance to a three-dimensional distance includes converting the identified trailer angle and the two-dimensional distance corresponding to the image. function This includes cross-referencing.

[0023] In another example of any of the above camera mirror systems for vehicles, the determined three-dimensional distances are stored in a memory list of determined three-dimensional distances.

[0024] In another example of any of the above camera mirror systems for vehicles, the controller further stores a filtering module configured to filter the memory list of identified three-dimensional distances.

[0025] In another example of any of the above camera mirror systems for vehicles, the filtering module is a mid-pass filter.

[0026] In one exemplary embodiment, the vehicle controller includes an image input configured to receive images from at least one camera. Department and a processor and memory, wherein the memory stores an image-based object detection module configured to identify at least one feature within a received image, a trailer angle module configured to identify a trailer angle relative to the tractor, and a trailer detection parameter identification module configured to identify trailer detection parameters based on the trailer angle and the images received from the at least one camera.

[0027] In another example of the vehicle controller, the trailer detection parameter identification module causes the processor to identify a two-dimensional distance from at least one wheel to a predetermined position on the image and convert the identified trailer angle and the two-dimensional distance corresponding to the image. function and converting the two-dimensional distance to a three-dimensional distance based at least in part on the identified angle by cross-referencing the two-dimensional distance to a three-dimensional distance, the three-dimensional distance being a wheelbase length of the trailer.

[0028] In another example of the vehicle controller, the memory further includes a filtering module configured to filter the memory list of determined three-dimensional distances with a mid-pass filter. [Brief description of the drawings]

[0029] The present disclosure can be further understood by reference to the following detailed description when considered in conjunction with the accompanying drawings. [Figure 1A] FIG. 1 is a schematic front view of a commercial truck having a camera mirror system (CMS) used to provide at least a Class II view and a Class IV view. [Figure 1B] FIG. 1 is a schematic top view of a commercial truck equipped with a camera mirror system providing Class II, Class IV, Class V and Class VI views. [Diagram 2] FIG. 1 is a schematic top perspective view of a vehicle cabin including a display and an interior camera. [Diagram 3] 1 shows a schematic of a vehicle undergoing a turning maneuver, with the tractor and trailer at a predetermined angle. [Figure 4] 1 illustrates a schematic of a process for determining wheelbase length based on an image including a trailer.

[0030] The embodiments, examples and alternatives of the preceding paragraphs, claims or the following description and drawings, including any of their various aspects or their respective individual features, can be taken independently or in any combination. Features described in relation to one embodiment are applicable to all embodiments, unless such features are incompatible. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] A schematic diagram of a commercial vehicle 10 is shown in Figures 1A and 1B. The vehicle 10 includes a vehicle cab or tractor 12 for towing a trailer 14. Although a commercial truck is contemplated in this disclosure, the invention may be applied to other types of vehicles. The vehicle 10 incorporates a camera mirror system (CMS) 15 (Figure 2) having driver and passenger side camera arms 16a, 16b mounted on the outside of the vehicle cab 12. If desired, the camera arms 16a, 16b may include conventional mirrors integrated therewith, although the CMS 15 may be used to replace mirrors entirely. In a further example, each side may include multiple camera arms 16, with each arm 16 housing one or more cameras and / or mirrors.

[0032] Each of the camera arms 16a, 16b includes a base that is fixed to, for example, the cab 12. The pivot arm is supported by and can articulate relative to the base. At least one rear-facing camera 20a, 20b is disposed within each of the camera arms. The exterior cameras 20a, 20b each provide an exterior field of view FOVEX1, FOVEX2 that each includes at least one of a Class II view and a Class IV view (FIG. 1B), which are legally defined views in the commercial truck industry. The Class II view of a particular side of the vehicle 10 is a subset of the Class IV view of the same side of the vehicle 10. If desired, multiple cameras can be used in each of the camera arms 16a, 16b to provide these views. Each of the arms 16a, 16b can also provide a housing that encases electronics configured to provide various functions of the CMS 15.

[0033] First and second video displays 18a, 18b are positioned at or near the A-pillars 19a, 19b on the driver's and passenger's sides, respectively, within the vehicle cab 12 and display Class II and Class IV views of each side of the vehicle 10, which provide rearward facing side views along the vehicle 10 captured by external cameras 20a, 20b.

[0034] If video of Class V and Class VI views is also desired, camera housing 16c and camera 20c can be positioned at or near the front of vehicle 10 to provide these views (FIG. 1B). A third display 18c located within vehicle cab 12 near the top center of the windshield can be used to display Class V and Class VI views facing forward of vehicle 10 to the driver.

[0035] If a Class VIII view video is desired, the camera housing can be disposed on the side and rear of the vehicle 10 to provide a field of view that includes some or all of the Class VIII zone of the vehicle 10. In such an example, the third display 18c can include one or more frames that display the Class VIII view. Alternatively, an additional display can be added near the first, second, and third displays 18a, 18b, 18c to provide a dedicated display for presenting the Class VIII view.

[0036] Referring particularly to FIG. 1B, the trailer 14 is coupled to the tractor 12 via a hitch 102. The trailer 14 includes a set of wheels 112, 114 and a distance 120 from the center point 116 of the set of wheels, and the hitch 102 is referred to as the wheelbase of the trailer 14. Similarly, the length from the hitch point of the trailer to the rear end of the trailer is referred to as the trailer length. A given tractor 12 can be connected to a plurality of different trailers 14, and the different trailers 14 can have different potential wheelbase lengths and different potential trailer lengths. In some examples, the wheelbase length can range from 5 m (16.4 feet) to 15 m (49.2 feet), although alternative lengths can be achieved without substantially modifying the configuration of the vehicle 10. Similar ranges can exist for the trailer length.

[0037] Some vehicle operations use the assumption of a default wheelbase length as an input parameter. As an example, semi - automatic mirror panning systems, kinematic models for automatic and / or semi - automatic driver assistance systems, and similar vehicle systems can utilize the wheelbase length to assist their operations. The default wheelbase assumption can provide an acceptable input parameter for some systems, but some systems executed by the vehicle ECU (e.g., semi - automatic driver assistance systems based on kinematic models), and some systems locally executed within the camera mirror system itself (e.g., automatic panning systems) can benefit from a more accurate specification of the actual wheelbase of the trailer 14.

[0038] Continuing to refer to FIGS. 1A, 1B, and 2, FIG. 3 schematically shows the vehicle 10 during a turning operation, where the tractor 12 and the trailer 14 are at an angle 102 within a predetermined angle range. During the turning operation, the camera arm 16 of the trailer 12 includes one or more cameras that capture a field of view 202 that includes the trailer 14, the wheels 112, 114, and other parts of the vehicle 10. The field of view 202 can be any of the class views described above, or a view provided by the camera arm 16 or another camera (or cameras) attached at a similar location.

[0039] A controller 104 connected to the camera receives the field of view 202 and performs an image - based analysis of the received image. The controller 104 can be the controller of the camera mirror system disposed within the camera arm 16, a general vehicle controller disposed within the trailer 12, or any other controller communicatively connected to the camera within the camera arm 16.

[0040] The controller 104 includes an image-based object detection system. The image-based object detection system uses software to analyze received images and identify one or more objects within the images. The object detection system can be any known object detection including rule-based object detection methods, machine learning based object detection methods, or any other known object detection methods. One set of objects that can be identified includes the rear wheels 112, 114 of the trailer 14.

[0041] The controller 104 also includes a trailer detection parameter identification software module. As used herein, trailer detection parameters refer to parameters indicative of trailer length, including the length from the hitch point to the rear end of the trailer (i.e., trailer length) and the length from the hitch point to the rear wheels of the trailer (i.e., wheelbase length). In one example, the trailer detection parameter identification software module is configured to identify the wheelbase length of the trailer 14 using the identified wheel 112, 114 positions according to a process 300 shown in FIG. 4. First, the process 300 identifies the position of each wheel of the trailer 14 in the image using an object detection system in step 310 of identifying trailer wheels. The object detection system identifies the front wheels 112 and rear wheels 114 of the set of wheels 110, as well as a midpoint 116 between the front wheels 112 and rear wheels 114.

[0042] The process 300 then determines the two-dimensional distance from the midpoint 116 to a predetermined location 118 at the edge of the field of view in a "determine 2D wheelbase" step 320. The predetermined location 118 is selected based on the expected location of the vehicle trailer 12 at the edge of the field of view 202 and may depend on the trailer angle. In another example, the predetermined location may be any other location in the image known to correlate to a hitch point. As used herein, the two-dimensional distance refers to the measured distance between the midpoint 116 and the predetermined location 118 in the image itself. The measured distance is, in one example, the number of pixels between the midpoint 116 and the predetermined location 118. As will be appreciated, the two-dimensional distance is not the same as the wheelbase length 120. However, the two-dimensional distance 120 correlates to the wheelbase length based on the trailer angle 102. By determining the two-dimensional length while the vehicle is within a predetermined angle range (e.g., between 20 degrees and 70 degrees), the controller 104 performs the conversion in a "Convert 2D Wheelbase to 3D Wheelbase" step 330, utilizing a known correlation between the measured two-dimensional length and the determined angles. function can be used to accurately determine the wheelbase length of the vehicle.

[0043] In one example, the conversion function identifies the best fit three-dimensional wheelbase length using a software module that includes multiple lookup tables. The lookup tables provide a mapping that defines the number of pixels in the two-dimensional image (two-dimensional length) to three-dimensional distance for a range of angles, where the first axis is the two-dimensional length and the second axis is the corresponding three-dimensional distance. Each lookup table corresponds to a range of trailer angles and is used if the trailer angle is within that range. If the two-dimensional length 120 is between two breakpoints on the lookup tables, the software module in the controller 104 can select the average of the possible three-dimensional wheelbases or select the wheelbase that is closest to the identified value.

[0044] In another example, the correlation between the angle and the two-dimensional wheelbase may be determined by using the trailer angle and the two-dimensional wheelbase as inputs to find the best fit. joint meeting The inputs can be converted into a regression equation, which the controller 104 uses to determine the correct output.

[0045] By performing multiple wheelbase determinations at multiple different angles and filtering the raw 3D wheelbase determinations with a mid-pass filter, noise and erroneous determinations are removed to determine the accurate 3D wheelbase, which can be stored by the controller 104 and provided to any vehicle system that requires or would benefit from a 3D wheelbase.

[0046] In yet another example, the trailer detection parameter identification software module can use substantially the same process to identify the trailer length instead of the wheelbase length, replacing the identification of the rear wheels with the identification of a rear feature of the trailer, which in one example is the bottom corner of the trailer.

[0047] While exemplary embodiments have been disclosed, a person of ordinary skill in the art would recognize that certain modifications would come within the scope of the following claims, and for that reason the following claims should be studied to determine their true scope and content.

Claims

1. 1. A method for determining trailer sensing parameters of a trailer, comprising: receiving, at a controller, images from at least one camera defining a field of view that includes at least a portion of the vehicle trailer; determining a trailer angle of the vehicle trailer relative to a tractor; Identifying at least one feature within an image of the trailer; determining a two-dimensional distance from the at least one feature to a predetermined location on the image; converting the two-dimensional distance into a three-dimensional distance that is a trailer detection parameter of the trailer based at least in part on the determined trailer angle; Including, the two-dimensional distance corresponds to a number of pixels in the image; The method of claim 1, wherein the conversion from the two-dimensional distance to the three-dimensional distance is performed by referencing at least one lookup table based on the identified trailer angle, the lookup table mapping the pixel number to the three-dimensional distance.

2. The method of claim 1 , wherein the trailer detection parameter is a wheelbase length and the at least one feature is at least one wheel.

3. 3. The method of claim 2, wherein the at least one wheel includes a first wheel and a second wheel, and the two-dimensional distance from the at least one wheel to the predetermined location on the image is a distance from a midpoint between the first wheel and the second wheel to the predetermined location on the image.

4. 3. The method of claim 2, wherein the at least one wheel is a single wheel and the two-dimensional distance from the at least one wheel to the predetermined location on the image is a distance from a center point of the wheel to the predetermined location on the image.

5. The method of claim 1 , wherein the trailer detection parameter is a trailer length from a hitch point to a rear end of the trailer and the at least one feature is a feature located at the rear end of the trailer.

6. The method of claim 5 , wherein the features located at the rear end of the trailer are lower corners of the trailer.

7. The method of claim 1 , wherein identifying at least one wheel in an image of the trailer includes identifying the at least one wheel using an image-based object detection system stored in the controller.

8. The method of claim 7 , wherein the controller is one of a vehicle control unit and a camera mirror system controller.

9. The method of claim 1 , wherein the view is one of a Class II view and a Class IV view.

10. The method of claim 1, wherein the predetermined position is a position at the edge of the field of view, and the two-dimensional distance is the measured distance between a center point of the wheel and the predetermined position within the image itself.

11. The method of claim 1, wherein converting the two-dimensional distance to the three-dimensional distance is performed at a plurality of identified trailer angles, each of the plurality of identified trailer angles providing a respective lookup table that maps the two-dimensional distance to the three-dimensional distance.

12. The method described in claim 11, wherein the converting step is performed for the identified plurality of trailer angles and includes a step of selecting a best three-dimensional distance based on a plurality of three-dimensional distances at the identified plurality of trailer angles.

13. The method described in claim 12, wherein the selecting step is performed by either finding the average of multiple three-dimensional distances or by selecting the three-dimensional distance that corresponds to the closest two-dimensional distance.

14. The method of claim 11 , wherein the identified trailer angles are in the range of 20 degrees to 70 degrees.

15. The method of claim 12, wherein the selecting step is performed using a transformation function that includes a best fit regression equation.

16. 15. The method of claim 14, further comprising repeating the method over a plurality of identified trailer angles ranging from 20 degrees to 70 degrees and storing each identified three-dimensional wheelbase length in memory.

17. The method of claim 16 , further comprising filtering the stored three-dimensional wheelbase lengths with a mid-pass filter.

18. The method of claim 17, wherein the memory stores an output of the mid-pass filter as a specified wheelbase length.

19. 1. A camera mirror system for a vehicle, comprising: at least one camera defining a field of view that includes at least a portion of the trailer; a controller including a processor and a memory that stores an image based object detection module configured to identify at least one wheel in an object detection system, a trailer angle module configured to identify a trailer angle relative to a tractor, and a trailer detection parameter identification module configured to identify a trailer wheelbase length based on the identified trailer angle and images received from the at least one camera; Equipped with the trailer detection parameter identification module includes instructions configured to cause the processor to determine a two-dimensional distance from at least one trailer parameter to a predetermined location on the image; the two-dimensional distance corresponds to a number of pixels in the image; the trailer detection parameter identification module includes instructions configured to cause the processor to convert the two-dimensional distance to a three-dimensional distance based at least in part on the determined trailer angle; A camera mirror system, wherein the processor for converting references at least one lookup table that maps the pixel number to the three-dimensional distance based on the identified trailer angle.

20. The camera mirror system of claim 19, wherein the processor for converting uses a conversion function that includes a best fit regression equation.

21. A vehicle controller, an image input configured to receive images from at least one camera; a processor and a memory, the memory storing an image based object detection module configured to identify at least one feature in a received image, a trailer angle module configured to identify a trailer angle relative to the tractor, and a trailer detection parameter identification module configured to identify a trailer detection parameter based on the identified trailer angle and images received from the at least one camera; Equipped with the trailer detection parameter identification module includes instructions configured to cause the processor to determine a two-dimensional distance from at least one trailer parameter to a predetermined location on the image; the two-dimensional distance corresponds to a number of pixels in the image; the trailer detection parameter identification module includes instructions configured to cause the processor to convert the two-dimensional distance to a three-dimensional distance based at least in part on the determined trailer angle; The processor for converting references at least one lookup table that maps the pixel number to the three-dimensional distance based on the identified trailer angle.

22. 22. The vehicle controller of claim 21, wherein the memory further comprises a filtering module configured to filter the memory list of determined three-dimensional distances with a mid-pass filter.

Images