Method and device for determining a joint angle

Abstract

Device (100) for determining a joint angle of a trailer (420; 510, 601) in relation to a vehicle, the device comprising: at least a memory, (103) comprising computer-executable instructions; and at least one processor (101) configured to read and execute the computer-executable instructions, wherein the computer-executable instructions cause the at least one processor (101) to: detect an area of ​​a trailer (420; 510; 601) based on a first image (306; 441; 501) of the trailer (420; 510; 601) captured at a first distance, and generate a first bounding frame (302, 431, 511) around the detected area of ​​the trailer (420; 510; 601) in the first image (306; 441; 501); detect the area of ​​the trailer (420; 510; 601) based on a second image (307; 442; 502) of the trailer (420; 510;601), which was recorded at a second distance, and generating a second bounding frame (303; 432, 512) around the detected area of ​​the trailer (420; 510; 601) in the second image (307; 442; 502); Determining a first set of feature points in the first bounding frame (302, 431, 511) and a second set of feature points in the second bounding frame (303; 432, 512) corresponding to the first set of feature points; Determining a correspondence between the first set of feature points and the second set of feature points and a correspondence between the corners of the first bounding frame (302, 431, 511) and the corners of the second bounding frame (303; 432, 512); Estimating at least one dimension of the trailer (420; 510; 601) by performing a least squares analysis to obtain three-dimensional coordinates (309; 407) of the real world of the first and second sets of feature points and the vertices of the first and second bounding frames (303;432, 512); wherein the computer-executable instructions cause the at least one processor (101) to determine an articulation angle of the trailer (420; 510; 601) with respect to a line through a center of the vehicle (410) based on the estimated at least one dimension and the three-dimensional coordinates (309; 407) of the real world.;

Description

INTRODUCTION The device and method relate to determining the articulation angle of trailers and assisting operators in coupling vehicle trailer hitches. In particular, the device and method, which correspond to exemplary embodiments, relate to providing a reversing video to assist a driver with a trailer hitch. US 6,172,601 B1 discloses a reversing camera system mounted at the rear of the towing vehicle that captures images of the attached trailer and provides trailer angle detection. The trailer is detected using a color pattern matching technique, in which the positions of color-patterned stickers on the trailer are detected and a relative angle of the trailer to the longitudinal axis of the vehicle is determined. Further state of the art is described in WO 2012 103 193 A1 and US 2010 / 0 324 770 A1. The object of the invention is to create a device and a method with which it is possible to determine a joint angle between a vehicle and a trailer based on images from a reversing camera. The problem is solved by the subject matter of claims 1 and 9. Advantageous embodiments of the invention are described in the dependent claims. SUMMARY One or more exemplary embodiments provide a method and a device that can determine the dimensions of a trailer based on a video and the trailer's position relative to a vehicle. In particular, one or more exemplary embodiments provide a method and a device that can determine the trailer's dimensions, the distance between the trailer and the vehicle, and the trailer's articulation angle. A method for determining the articulation angle of trailers is provided. The method includes, for example, detecting the area of ​​a trailer based on a first image of the trailer taken at a first distance, and generating a first bounding box around the detected area of ​​the trailer in the first image; detecting the area of ​​the trailer based on a second image of the trailer taken at a second distance, and generating a second bounding box around the detected area of ​​the trailer in the second image; determining a first set of feature points in the first bounding box and a second set of feature points in the second bounding box corresponding to the first set of feature points.Determining a correspondence between the first set of feature points and the second set of feature points, and a correspondence between the corners of the first bounding frame and the corners of the second bounding frame, and estimating at least one dimension of the trailer by performing a least squares analysis to determine, for three-dimensional real-world coordinates, the first and second sets of feature points and the corners of the first and second bounding frames, wherein determining an articulation angle of the trailer with respect to a line through a vehicle centerline is performed based on the estimated at least one dimension and the three-dimensional real-world coordinates. The articulation angle can be determined from the estimated minimum of one dimension and a distance between the trailer and the vehicle. The estimated minimum dimension can be the width of the trailer. The distance can be determined from information provided by at least one accelerometer, speedometer, ultrasonic sensor, odometer, radar sensor and wheel speed sensor. The estimated minimum dimension can be the height and width of the trailer. The procedure may further include receiving the first image of the trailer, taken at the first distance, and the second image of the trailer, taken at the second distance, from a camera facing the rear of the vehicle. The camera can be a camera located in a centrally mounted top brake light (CHMSL). Determining the correspondence between the first set of feature points and the second set of feature points may involve determining a rotation matrix corresponding to an articulation angle of the trailer with respect to a line through a vehicle centerline. The correspondence between the first set of feature points and the second set of feature points, as well as the correspondence between the corners of the first boundary frame and the corners of the second boundary frame, can be determined based on this rotation matrix. Furthermore, a device for determining the articulation angle of trailers is provided. The device comprises: at least one memory containing computer-executable instructions; and at least one processor configured to execute the computer-executable instructions. The computer-executable instructions can cause the at least one processor to detect an area of ​​a trailer based on a first image of the trailer at a first distance and to create a first bounding box around the detected area of ​​the trailer in the first image; to detect the area of ​​the trailer based on a second image of the trailer at a second distance and to create a second bounding box around the detected area of ​​the trailer in the second image; and to determine a first set of feature points in the first bounding box and a second set of feature points in the second bounding box.which correspond to the first set of feature points, determine a correspondence between the first set of feature points and the second set of feature points and a correspondence between the corners of the first bounding frame and the corners of the second bounding frame, and estimate at least one dimension of the trailer by performing a least squares analysis to determine, for three-dimensional real-world coordinates, the first and second sets of feature points and the corners of the first and second bounding frames, wherein the computer-executable instructions can cause the at least one processor to determine an articulation angle of the trailer with respect to a line through a center of the vehicle based on the estimated at least one dimension and on three-dimensional real-world coordinates. The computer-executable instructions can cause the at least one processor to determine the articulation angle based on the estimated at least one dimension and distance between the trailer and the vehicle. The estimated at least one dimension can be the width of the trailer. The computer-executable instructions can cause the at least one processor to determine the distance from the information provided by at least one accelerometer, speedometer, ultrasonic sensor, odometer, radar sensor, and wheel speed sensor. The estimated minimum dimension can be the height and width of the trailer. The device may further include a camera directed towards the rear of the vehicle. The computer-executable instructions can cause the at least one processor to receive the first image of the trailer, taken at the first distance, and the second image of the trailer, taken at the second distance, from the camera. The camera may be located in a centrally mounted top-mounted brake light (CHMSL). The computer-executable instructions can cause the at least one processor to determine the correspondence between the first set of feature points and the second set of feature points by determining a rotation matrix corresponding to an articulation angle of the trailer with respect to a line through a center point of the vehicle. The correspondence between the first set of feature points and the second set of feature points, as well as the correspondence between the corners of the first bounding frame and the corners of the second bounding frame, can be determined based on the rotation matrix. The computer-executable instructions can also cause the at least one processor to estimate at least one dimension of the pendant based on the focal length of a camera that took the first and second images. Further purposes, advantages and novel features of the exemplary embodiments will become apparent from the following detailed description of the exemplary embodiments and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a block diagram of a device that detects a trailer according to an exemplary embodiment; Fig. 2 shows a flowchart for a method for detecting a trailer according to an exemplary embodiment; Fig. 3 shows a diagram for detecting feature points in an image of a trailer according to one aspect of an exemplary embodiment; Fig. 4 shows diagrams for determining a boundary frame at various distances from a trailer view according to one aspect of an exemplary embodiment; Fig. 5 shows diagrams for determining a boundary frame and feature points at various distances from a trailer view according to one aspect of an exemplary embodiment; and Figs. 6A and 6B show diagrams for calculating a trailer's articulation angle with respect to a vehicle's centerline according to one aspect of an exemplary embodiment. DETAILED DESCRIPTION OF EXAMPLES OF EXECUTION A device and a method for detecting a trailer and determining a joint angle will now be described in detail with reference to Figures 1-6B of the accompanying drawings, in which the same reference numerals refer to the same elements. The following disclosure enables those skilled in the art to carry out the inventive concept. However, the exemplary embodiments disclosed herein are merely illustrative and do not limit the inventive concept of the embodiments described herein. Furthermore, descriptions of the features or aspects of each exemplary embodiment should normally be considered as available for aspects of other embodiments. It is also understood that, where indicated herein, a first element is "connected to", "formed on" or "attached" to a second element, the first element may be directly connected to, directly formed on or directly arranged on the second element, and intermediate elements may be present between the first element and the second element, unless it is indicated that a first element is "directly" connected to, attached to, formed on or arranged on the second element.Furthermore, if a first element is configured to "send" information to or "receive" information from a second element, the first element can send the information directly to or receive it from the second element, send the information via or receive it from a bus, send or receive it via a network, or send or receive it via intermediate elements, unless the first element is configured to send information "directly" to or receive it from the second element. Throughout the entire disclosure, one or more of the disclosed elements can be combined into a single device or combined into one or more devices. Additionally, individual elements can be provided on separate devices. Vehicles, such as trucks, include towing hitches that serve as attachment points for trailers that can be pulled by the vehicle. Some trailers that can be pulled by a vehicle include a fifth wheel coupling, a gooseneck coupling, or a low-profile hitch. To attach a trailer to the vehicle, the driver must position the vehicle so that a coupling located in the vehicle bed or at the rear of the vehicle aligns with the coupling on the trailer. Often, the process of aligning the vehicle's coupling with the trailer hitch requires repeatedly getting in and out of the vehicle or guiding another person standing outside the vehicle. To address this issue, a vehicle driver can view an image provided by a reversing camera or a rear-facing camera. This image can be used by the operator to guide the vehicle. However, reversing cameras that only provide a visual view may lack other information such as the distance to the trailer, the trailer's angle, and its dimensions. Therefore, it can be helpful to provide additional information to the operator of a vehicle or vehicle system used to align a vehicle with a trailer. Fig. 1 shows a block diagram of a device for determining the articulation angle of a trailer and for providing trailer information 100 (i.e., a device for determining the articulation angle of a trailer) according to an exemplary embodiment. As shown in Fig. 1, the device for determining the articulation angle of a trailer 100 according to an exemplary embodiment includes a controller 101, a power supply 102, a memory 103, an output 104, a user input 106, a trailer detection sensor 107, and a communication device 108. However, the device for determining the articulation angle of a trailer 100 is not limited to the configuration mentioned above and can be configured to include additional elements and / or omit one or more of the aforementioned elements.The device for determining the articulation angle of a trailer 100 can be implemented as part of a vehicle, as a standalone component, as a hybrid between a vehicle and a non-vehicle device or another computer device. The controller 101 controls the overall operation and function of the device for determining the articulation angle of a trailer 100. The controller 101 can control one or more memories 103, an output 104, a user input 106, a trailer detection sensor 107, and a communication device 108 of the device for determining the articulation angle of a trailer 100. The controller 101 can include one or more components consisting of a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, circuits, and a combination of hardware, software, and firmware components. The controller 101 is configured to send and / or receive information from one or more memories 103, output 104, user input 106, trailer detection sensor 107, and communication device 108 of the trailer articulation angle determination device 100. The information can be sent and received via a bus or network, or it can be read or written directly from one or more of the memories 103, output 104, user input 106, trailer detection sensor 107, and communication device 108 of the trailer articulation angle determination device 100. Examples of suitable network connections include a Controller Area Network (CAN), Media Oriented System Transfer (MOST), Local Area Linkage Network (LIN), Local Area Network (LAN), wireless networks such as Bluetooth and 802.11, and other suitable connections such as Ethernet. For example, the controller 101 can receive information from one or more accelerometers, speedometers, ultrasonic sensors, odometers, radar sensors, and wheel speed sensors. The information from one or more of these sensors can be used to determine the distance traveled by a vehicle or the distance between a vehicle and a trailer. The power supply 102 provides power to one or more of the controllers 101, the memory 103, the output 104, the user input 106, the trailer detection sensor 107, and the communication device 108 of the device for determining the articulation angle of a trailer 100. The power supply 102 can include one or more components such as a battery, an outlet, a capacitor, a solar cell, a generator, a wind turbine, an AC generator, etc. Memory 103 is configured to store and retrieve information used by the device for determining the articulation angle of a trailer 100. Memory 103 can be controlled by the controller 101 to store and retrieve information received from the trailer detection sensor 107. This information may include details of a trailer detected by the trailer detection sensor 107, image information from images taken by the trailer detection sensor 107, information from an accelerometer, speedometer, ultrasonic sensor, odometer, radar sensor, or wheel speed sensor, and / or information about whether a vehicle is to be coupled to a trailer.Memory 103 can also contain the computer-executable instructions configured to be executed by a processor to perform the functions of the device for determining the joint angle of a trailer 100. Memory 103 can include one or more floppy disks, optical disks, CD-ROMs (Compact Disc-Read Only Memories), magneto-optical disks, ROMs (Read Only Memories), RAMs (Random Access Memories), EPROMs (erasable programmable read-only memories), EEPROMs (electrically erasable programmable read-only memories), magnetic or optical cards, flash memory, cache memory, and other types of media / machine-readable media suitable for storing machine-executable instructions. Output 104 provides information in one or more forms, including visual, audible, and / or haptic. Output 104 can be controlled via control 101 to provide 100 outputs to the user of the device for determining the articulation angle of a trailer. Output 104 can include one or more outputs from a loudspeaker, audio device, display, centrally located display, head-up display, windshield display, haptic feedback device, vibration device, tactile feedback device, tap feedback device, holographic display, instrument light, indicator light, etc. Output 104 can output notifications that include one or more of the following: an audible alert, a visual alert, and an indicator alert, among others. The notification can indicate that the vehicle is in trailer mode or is attempting to couple to a trailer. Additionally, output 104 can output an image of the rear of a vehicle, depicting the vehicle's cargo bed and / or an area behind the vehicle, including any trailer to which the vehicle is attempting to couple. The image of the rear of the vehicle can be edited or modified to display a graphic corresponding to the articulation angle of a trailer with respect to a line through the center of the vehicle, or a graphic showing the distances between the edges of a trailer and a part of the vehicle (e.g., the side of the trailer).a vehicle cabin) or include a graphic that corresponds to the dimensions of the trailer. User input 106 is configured to provide information and commands to the device for determining the articulation angle of a trailer 100. User input 106 can be used to provide user input, etc., to the controller 101. User input 106 can include one or more inputs from a touchscreen, keyboard, soft keyboard, button, motion detector, voice input detector, microphone, camera, trackpad, mouse, touchpad, etc. User input 106 can be configured to receive user input and thereby acknowledge or reject the notification via output 104. User input 106 can also be configured to receive user input to activate a trailer detection algorithm or to activate a trailer-determining mode of the device for determining the articulation angle of a trailer 100. The trailer detection sensor 107 can incorporate one or more sensors from a variety of options, including an imaging sensor, camera, laser sensor, ultrasonic sensor, infrared camera, LiDAR, radar sensor, ultra-short-range radar sensor, ultra-wideband radar sensor, and microwave sensor. The trailer detection sensor 107 can provide one or more images from one or more rear-facing cameras, which can be analyzed to determine whether a vehicle driver intends to couple a trailer to a hitch on the vehicle's loading platform. This analysis is used to identify a trailer and / or to identify a trailer hitch. Furthermore, other types of information, such as distance, infrared images, speed, acceleration, direction of travel, distance traveled, distance between a trailer and a vehicle or a vehicle camera, camera focal length, etc., can be provided by other types of sensors. For example, information from an accelerometer, a speedometer, an ultrasonic sensor, an odometer, a radar sensor, or a wheel speed sensor, etc. Memory 103 can also contain the computer instructions configured by a processor to execute the functions of the device for determining the articulation angle of the trailer 100.The information can be processed to determine whether an operator of a vehicle intends to couple a trailer to a trailer hitch on a loading platform of the vehicle, to determine a trailer type, to determine a trailer's position, to determine information about a vehicle's movement, such as speed and / or trajectory, or to determine whether a loading platform door of a vehicle is open. The communication device 108 can be used by the trailer articulation angle determination device 100 to communicate with various types of external devices using different communication methods. The communication device 108 can be used to send / receive information, including information about a trailer detected by the trailer detection sensor 107, information from the trailer detection sensor 107 such as image information, vehicle dynamics information such as vehicle speed and trajectory, information from an accelerometer, speedometer, ultrasonic sensor, odometer, radar sensor, or wheel speed sensor, and / or information about whether a vehicle is to be coupled to a trailer.Memory 103 may also contain the computer instructions configured by a processor to execute the functions of the device for determining the articulation angle of the trailer 100, and / or information indicating whether a vehicle is coupled to a trailer, to / from the control 101 of the device for determining the articulation angle of the trailer 100. The communication device 108 can include various communication modules, such as one or more broadcast receiver modules, a short-range communication (NFC) module, a GPS module, and a wireless communication module. The broadcast receiver module can include a terrestrial broadcast receiver module, which contains an antenna for receiving a terrestrial broadcast signal, a demodulator, an equalizer, etc. The NFC module is a module that communicates with an external device located at close range according to an NFC protocol. The GPS receiver is a module that receives a GPS signal from a GPS satellite and determines a current location. The wired communication module can be a module that receives information via a wired network, such as a local area network, a controller area network (CAN), or an external network.A wireless communication module is a module that connects to and communicates with an external network via a wireless communication protocol, such as IEEE 802.11, WiMAX, WLAN, or IEEE communication protocols. The wireless communication module may also include a cellular communication module that accesses a cellular network and enables communication according to various cellular communication standards, such as 3G, 3GPP, LTE, Bluetooth, EVDO, CDMA, GPRS, EDGE, or Zigbee. According to an exemplary embodiment, the control unit 101 of the device for determining the articulation angle of the trailer 100 can be configured to detect an area of ​​a trailer based on a first image of the trailer at a first distance and to generate a first boundary box around the detected area of ​​the trailer in the first image, to detect the area of ​​the trailer based on a second image of the trailer at a second distance and to generate a second boundary box around the detected area of ​​the trailer in the second image, to determine a first set of feature points in the first boundary box and a second set of feature points in the second boundary box that correspond to the first set of feature points.Determine a correspondence between the first set of feature points and the second set of feature points, and a correspondence between the corners of the first bounding frame and the corners of the second bounding frame, and estimate at least one dimension of the pendant by performing a least squares analysis to determine the three-dimensional real-world coordinates of the first and second sets of feature points and the corners of the first and second bounding frames. The estimated minimum dimension can be the height and width of the trailer. The feature points can correspond to the edges of a trailer, objects on the trailer, features of the trailer, or other visually perceptible features arranged on a surface of the trailer. The control 101 of the device for determining the articulation angle of the trailer 100 can be configured to determine an articulation angle of the trailer with respect to a line through a vehicle centerline based on the estimated at least one dimension and / or the three-dimensional coordinates of the real world. The control unit 101 of the device for determining the articulation angle of the trailer 100 can be configured to determine the articulation angle from the distance between the trailer and the vehicle and the width of the trailer. The control unit 101 of the device for determining the articulation angle of the trailer 100 can be configured to determine the distance of information provided by at least one accelerometer, speedometer, ultrasonic sensor, odometer, radar sensor and wheel speed sensor. The 101 control unit can also be configured to receive the first image of the trailer, taken at the first distance, and the second image of the trailer, taken at the second distance, from a camera facing the rear of a vehicle. This camera can be located in a centrally mounted top-mounted brake light (CHMSL). The control unit 101 can also be configured to determine the correspondence between the first set of feature points and the second set of feature points by defining a rotation matrix corresponding to an articulation angle of the trailer with respect to a line through a vehicle centerline. The correspondence between the first set of feature points and the second set of feature points, as well as the correspondence between the corners of the first boundary frame and the corners of the second boundary frame, can be determined based on the rotation matrix. Fig. 2 shows a flowchart for a method for determining the articulation angle of a trailer according to an exemplary embodiment. The method of Fig. 2 can be carried out by the device for determining the articulation angle of a trailer 100 or encoded in a computer-readable medium as instructions that can be executed by a computer to carry out the method. Referring to Fig. 2, in operation S210, the area of ​​a trailer is captured based on a first image of the trailer taken at a first distance, and a first bounding box is created around the detected area of ​​the trailer in the first image. The area of ​​the trailer is then captured based on a second image of the trailer taken at a second distance, and a second bounding box around the detected area of ​​the trailer in the second image is captured in operation S220. In operation S230, a first set of feature points is determined within the first bounding box, and a second set of feature points is determined within the second bounding box, corresponding to the first set of feature points. Operations S210–S230 can be repeated until a sufficient number of sample images have been acquired to perform a least squares analysis. In operation S240, a match is found between the first set of feature points and the second set of feature points, as well as a match between the vertices of the first bounding box and the vertices of the second bounding box. Subsequently, at least one dimension of the pendant is estimated by performing a least squares analysis to determine the three-dimensional real-world coordinates of the first and second sets of feature points and the vertices of the first and second bounding boxes in operation S250. Fig. 3 shows a representation of the recognition of feature points in an image of a pendant according to one aspect of an exemplary embodiment. With reference to Fig. 3, the images 306-308, the perspectives 311-313, and the boundary boxes 302-304 are shown according to their correspondence with the images and perspectives. The projection matrix for each of the perspectives can be used to determine a realistic perspective 305 based on three-dimensional coordinates 309 of the real world of the feature points 301 and 310. In an example, if feature points (N) are detected in all snapshots (K), we obtain 2NK measurements. Each feature point contributes to two measurements. The number of unknowns is determined by the equation 3K + 2N, where there are three variables per snapshot (K) and two variables for each feature point (N) in the yz-plane. Therefore, if 2NK ≥ 3K + 2N, meaning the number of measurements is greater than the number of unknowns, the position of the feature points in the yz-plane can be determined, and the width and height of the trailer can be derived from the position of the feature points. Accordingly, if the number of measurements is greater than the number of unknowns, a least squares algorithm can be applied to solve for the three-dimensional real-world coordinates of a feature point by minimizing min||zij- PiXj||2, where zij is a feature point in the bounding frame of an image, Pij is a projection matrix, and Xj is a three-dimensional real-world coordinate of a feature point. Assuming a truck is moving on level ground, the projection matrix Pi can be determined by the following equation: Pi = P[Ri|ti], where ti is the camera position (xi, yi, h) with h being the camera height, where the rotation matrix Ri is determined by the trailer's articulation angle φ, e.g., Ri = I for the zero target angle, and P is the camera projection matrix (constant), derived from the camera's internal and external parameters. In one example, the constant P can be adjusted based on the focal length, image resolution, camera mounting position, and / or camera angle (e.g., three Euler angles and the 3D position of the real world). According to one example, the projection matrix is ​​determined by the following equation: Fig. 4 shows representations for determining a boundary frame at various distances from a trailer view according to one aspect of an exemplary embodiment. Referring to Fig. 4, the dimensions of the trailer 420 can be determined during a straight reversing maneuver of the vehicle 410, e.g., when a vehicle 410 is moving backwards and straight towards the trailer 420. As shown in Fig. 4, three images 441-443 with different perspectives 401-403 of the trailer are taken at three different distances from the trailer. A boundary frame from the plurality of boundary frames 431-433 can correspond to each perspective. The correspondence between a point 406 (e.g., a corner of a bounding box) in the plurality of bounding boxes 431-433 can be determined. This correspondence can be used to determine a three-dimensional coordinate 407 of the point in the real world. This can be applied to all corners of the bounding box to determine the three-dimensional real-world coordinates of all corners of the bounding box. For example, if we have three snapshots or images, the three-dimensional real-world coordinates of the corners of the bounding frame can be determined. Considering the camera height, the displacement between the first and second images (404), and the displacement between the second and third images (405), we would apply least squares analysis to the nine unknown variables: the distance in the first image (441) and the (x, y) coordinates of the corners of the bounding frames (432 and 433). Least squares analysis would allow us to determine a three-dimensional real-world coordinate (434) of a corner or feature point (407). Fig. 5 shows images for determining a boundary frame and feature points at various distances from a trailer view according to one aspect of an exemplary embodiment. Referring to Fig. 5, a plurality of images 501-504 were taken at various distances between a vehicle and the trailer 510. As shown in each of the plurality of images 501-504, a plurality of boundary frames 511-514 are generated according to the area of ​​the trailer, and a plurality of feature points 521-524 are detected. As the distance between a vehicle and the trailer 510 in images 501-504 decreases, the bounding boxes increase in size (e.g., size 514 > size 513 > size 512 > size 511). Furthermore, the distance between feature points 521-524 and the position of feature points 521-524 relative to the image frame also increase. This distance between feature points 521-524 in each image, the position of feature points 521-524 in each image, the dimensions of the bounding boxes 511-514, and the positions of the corners of the bounding boxes 511-514 can be used to determine the distance between a part of a vehicle and the trailer 510. Figures 6A and 6B illustrate how to calculate the articulation angle of a trailer with respect to a vehicle's centerline according to one aspect of an exemplary embodiment. Referring to Figure 6A, a trailer 601 is shown passing perpendicularly to a centerline through a vehicle. This figure shows a first distance 602 between the left side of a part of the vehicle and the left side of the trailer 601, and a second distance 603 between the right side of a part of the vehicle and the right side of the trailer 601. The distances 602 and 603 are equal or have predetermined values ​​when the trailer 601 is perpendicular to a centerline through the vehicle. Furthermore, the height 605 and width 604 of the trailer may be known from Figure 610 or calculated. Referring to Fig. 6B, the trailer 601 is no longer perpendicular to a centerline through the vehicle due to the movement of the trailer and the vehicle in Fig. 620. Therefore, the first and second distances 602 and 603 have changed. Based on the change in distances 602 and 603 and the width of the trailer 604, the articulation angle of the trailer 601 with respect to a centerline through the vehicle can be calculated. Furthermore, based on the width 604, the height 605, and the calculated articulation angle, it can be determined whether there is a risk of collision between the trailer 601 and a part of the vehicle. The vehicle operator can then be alerted to the risk of collision. The processes, methods, or algorithms disclosed herein may be supplied / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control device or a dedicated electronic control device. Likewise, the processes, methods, or algorithms may be stored by a controller or computer as data or executable instructions in a variety of ways, including, without limitation, permanent storage on non-writable storage media such as ROM, and as modifiable information on writable storage media such as floppy disks, magnetic tapes, CDs, RAM, and other magnetic and optical media. The processes, methods, or algorithms may also be implemented in a software-executable object.Alternatively, the processes, procedures or algorithms can be embodied wholly or partially with suitable hardware components, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components.

Claims

Device (100) for determining a joint angle of a trailer (420; 510, 601) in relation to a vehicle, the device comprising: at least a memory, (103) comprising computer-executable instructions; and at least one processor (101) configured to read and execute the computer-executable instructions, wherein the computer-executable instructions cause the at least one processor (101) to: detect an area of ​​a trailer (420; 510; 601) based on a first image (306; 441; 501) of the trailer (420; 510; 601) captured at a first distance, and generate a first bounding frame (302, 431, 511) around the detected area of ​​the trailer (420; 510; 601) in the first image (306; 441; 501); detect the area of ​​the trailer (420; 510; 601) based on a second image (307; 442; 502) of the trailer (420; 510;601), which was recorded at a second distance, and generating a second bounding frame (303; 432, 512) around the detected area of ​​the trailer (420; 510; 601) in the second image (307; 442; 502); Determining a first set of feature points in the first bounding frame (302, 431, 511) and a second set of feature points in the second bounding frame (303; 432, 512) corresponding to the first set of feature points; Determining a correspondence between the first set of feature points and the second set of feature points and a correspondence between the corners of the first bounding frame (302, 431, 511) and the corners of the second bounding frame (303; 432, 512); Estimating at least one dimension of the trailer (420; 510; 601) by performing a least squares analysis to obtain three-dimensional coordinates (309; 407) of the real world of the first and second sets of feature points and the vertices of the first and second bounding frames (303;432, 512); wherein the computer-executable instructions cause the at least one processor (101) to determine an articulation angle of the trailer (420; 510; 601) with respect to a line through a center of the vehicle (410) based on the estimated at least one dimension and the three-dimensional coordinates (309; 407) of the real world.; Device according to claim 1, wherein the computer-executable instructions cause the at least one processor (101) to determine the articulation angle by determining the articulation angle based on the estimated at least one dimension and distance between the trailer (420; 510; 601) and the vehicle (410), and wherein the estimated at least one dimension comprises a width of the trailer (420; 510; 601). Device according to claim 2, further comprising: at least one accelerometer, tachometer, ultrasonic sensor, odometer, radar sensor and wheel speed sensor, wherein the computer-executable instructions cause the at least one processor (101) to determine the distance from the information provided by the at least one accelerometer, tachometer, ultrasonic sensor, odometer, radar sensor and wheel speed sensor. Device according to claim 1, wherein the estimated at least one dimension comprises a height and a width of the trailer (420; 510; 601). Device according to claim 1, further comprising: a camera configured as a rear-facing camera or reversing camera, wherein the computer-executable instructions cause the at least one processor (101) to receive from the camera the first image (306; 441; 501) of the trailer (420; 510; 601) taken at the first distance and the second image (307; 442; 502) of the trailer (420; 510; 601) taken at the second distance. Device according to claim 5, wherein the camera comprises a centrally mounted top brake light (CHMSL camera). Device according to claim 1, wherein the computer-executable instructions cause the at least one processor (101) to determine the correspondence between the first set of feature points and the second set of feature points by determining a rotation matrix that is related to an articulation angle of the trailer (420; 510; 601) with respect to a line through a center of the vehicle (410), and wherein the correspondence between the first set of feature points and the second set of feature points and the correspondence between the corners of the first bounding frame (302, 431, 511) and the corners of the second bounding frame (303; 432, 512) is determined based on the rotation matrix. Device according to claim 1, wherein the computer-executable instructions cause the at least one processor (101) to estimate at least one dimension of the pendant (420; 510; 601) based on a focal length of a camera that has taken the first image (306; 441; 501) and the second image (307; 442; 502). Method for determining a joint angle of trailers (420; 510; 601) with respect to a vehicle, the method comprising: detecting an area of ​​a trailer (420; 510; 601) based on a first image (306; 441; 501) of the trailer (420; 510; 601) taken at a first distance, and generating a first bounding box around the detected area of ​​the trailer (420; 510; 601) in the first image (306; 441; 501); detecting the area of ​​the trailer (420; 510; 601) based on a second image (307; 442; 502) of the trailer (420; 510; 601) taken at a second distance, and generating a second bounding box around the detected area of ​​the trailer. (420; 510; 601) in the second image (307; 442;502), determining a first set of feature points in the first bounding box and a second set of feature points in the second bounding box corresponding to the first set of feature points, determining a correspondence between the first set of feature points and the second set of feature points and a correspondence between vertices of the first bounding frame (302, 431, 511) and vertices of the second bounding frame (303; 432, 512), and estimating at least one dimension of the pendant (420; 510; 601) by performing a least squares analysis to determine three-dimensional real coordinates of the first and second sets of feature points and the vertices of the first and second bounding frames (303; 432, 512), wherein determining a joint angle of the pendant (420; 510;601) with respect to a line through a center of the vehicle (410) based on the estimated at least one dimension and the three-dimensional coordinates of the real world.;

Images