Camera-based system and method for determining trailer position

Abstract

To provide a method and a camera-based system for detecting the position of a trailer end with sufficient accuracy during backward maneuvering operations and at low speeds for tracking in a monitor image, irrespective of a trailer type and information acquired by sensors.SOLUTION: A method includes: capturing camera images successively in time by an image sensor; determining a plurality of feature points in each of the camera images on the basis of at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; selecting relevant feature points from the plurality of feature points along a preferred direction defined by the density and position of the feature points as path elements on the basis of a positional displacement (dx, dy) between the plurality of feature points in one camera image and a corresponding plurality of feature points in a camera image successive in time; generating at least one path describing a trailer along a predetermined direction on the basis of the path elements; and determining a position on the path at a trailer end.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a camera-based system (viewing system or mirror replacement system) and method for determining the position of the edge (edge) of a trailer that extends rearward from the vehicle and is rotatable relative to the vehicle, the camera-based system and method being suitable for commercial vehicles having a semi-trailer, trailer, or other unit that extends rearward and is rotatable relative to the cab or tractor unit. [Background technology]

[0002] Drivers of vehicles equipped with a tractor unit or towing unit having a trailer extending behind it and pivotable relative to the unit typically rely on side mirrors mounted on the sides of the towing unit to view traffic behind them. These side mirrors are increasingly being replaced with camera systems, such as camera monitoring systems or mirror replacement systems mounted on the sides or rear of the vehicle, respectively. These side mirrors replace or supplement traditional mirror systems provided on motor vehicles, such as exterior and rearview mirrors on automobiles and wide-angle and front mirrors on commercial vehicles.

[0003] In the above-described system, for example, the corresponding field of view that would conventionally be visible through a mirror is always displayed in real time to the driver of the vehicle, for example on an in-vehicle monitor or other display unit, so that the driver of the vehicle can always see the corresponding field of view through the above-described system rather than directly or through an attached mirror.

[0004] A trailer in the sense of the present teaching includes a trailer that is connected to a vehicle (tractor or towing unit) by a trailer coupling, for example a semi-trailer (so-called trailer), that is located in the lowered area at the rear of the tractor unit and that is connected so as to be rotatable about at least one vertical axis. Alternatively, such a trailer may be a trailer attached to a passenger car, in which case the passenger car corresponds to the tractor unit. In general, a trailer is a rear extension of a vehicle that is located rearward relative to the cab, that is movable (rotatable) laterally relative to the cab, and that rotates about an axis perpendicular to the cab when cornering. Articulated trucks and articulated trains are also trailers in the sense of the present invention.

[0005] However, when using such a mirror replacement system, the greater the buckling angle (kink angle or bending angle) between the towing unit and the trailer when the vehicle equipped with the towing unit and trailer is cornering, the more rear traffic will be hidden in the image of the mirror replacement system for field of view class II of the ECE R46 regulation. Furthermore, the rear of the trailer may be lost (disappear) near the rear axle, which may cause the end of the trailer to disappear from the monitor image displayed to the driver.

[0006] To solve this problem, several methods already exist for tracking the trailer edge and shifting the image portion displayed on the monitor so that the trailer edge is displayed as centered as possible in the monitor image. According to EP 16 198 485, this tracking of the trailer edge is currently performed, for example, by vehicle sensors that acquire information about the rotational movement of the wheels. This information is analyzed by a control unit to determine the area behind the rear extension and track it appropriately based on the acquired information about the rotational movement of the wheels. However, since these sensors can only provide reliable signals above a certain speed, for example, about 2 km / h, such tracking of the trailer edge can only be performed when driving forward at high speeds. These sensors cannot be used when driving backward or at low speeds, because they cannot detect the trailer edge with sufficient accuracy to track (update) the trailer edge in the monitor image. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention aims to overcome the above-mentioned problems and to detect the position of the trailer end with sufficient accuracy during driving operations, such as reversing or at low speeds, in order to track (update) it in the monitor image independently of the type of trailer and the information acquired by the installed sensors. [Means for solving the problem]

[0008] The above mentioned object is solved by a method having the features of claim 1 and by a system having the features of claim 15. Advantageous further developments of the invention are set out in the dependent claims.

[0009] In the method of the present invention for determining the position of the trailer end or trailer edge of a trailer extending rearward from a towing unit of a vehicle, camera images captured by an image sensor of a camera-based system in succession over time are processed. The camera images used for processing do not need to be captured in succession; for example, only the second or third camera image can be used for further processing, thereby reducing the amount of image data to be processed. The captured camera images can be appropriately processed, for example, for resolution, contrast, and / or color information, before being processed. The method of the present invention can use raw data from the captured camera images or partially processed camera images.

[0010] When further processing successive camera images, the method according to the invention determines in each camera image so-called feature points based on one or more image parameters, at least calculated amounts of the one or more image parameters, and / or their gradients, where one feature point corresponds to at least one pixel in the camera image. A pixel in a camera image can be determined as a feature point if, for example, the value of the image parameter of this pixel differs from the value of the corresponding image parameter of one or more neighboring pixels by a predetermined value.

[0011] For example, adjacent pixels forming a pixel cluster may be compared with adjacent or nearby pixel clusters with respect to at least one image parameter to determine feature points. The pixels or pixel clusters do not need to be directly adjacent, but only need to be spatially close.

[0012] The method according to the present invention then selects a plurality of path elements from the determined feature points based on the positional displacement (misalignment) between a plurality of feature points in a camera image and corresponding feature points in a temporally subsequent camera image. The path elements are determined, for example, along a predetermined direction extending along the trailer in the direction of the trailer end, preferably determined by the density and / or location of the feature points. The predetermined direction can be fixed or, preferably, dynamically adjusted during processing. The dynamic adjustment can be performed based on the already defined path elements. The method then generates at least one so-called path indicating the course of the trailer based on these path elements, and determines a position on this at least one path as the position of the trailer end edge. The determined path elements or feature points have associated coordinates x, y, dx, and dy, where dx and dy indicate the misalignment or displacement in the x and y directions.

[0013] According to an advantageous embodiment of the method according to the invention, before the path elements are selected, the feature points are weighted. Such weighting can be based, for example, on the contrast difference between a pixel or pixel cluster and an adjacent pixel or pixel cluster or a nearby pixel or pixel cluster. One or more known image parameters are used as a weighting criterion. Furthermore, the positional deviation (dx, dy) between a feature point in a camera image and a corresponding feature point in a subsequent camera image can also be used as an alternative or additional weighting criterion.

[0014] According to a further advantageous embodiment of the method according to the invention, the above-mentioned selection of the path elements, the weighting of the characteristic points and / or the weighting of the path elements can be based on the kink angle and / or information about the relative position between the towing unit and the trailer and / or the vehicle speed. Such kink angle and / or the relative position can be determined independently of the acquired camera images, for example, using a kink angle sensor installed on the vehicle, or can be estimated based on data received from a steering angle sensor, or can be estimated by image analysis. Other signals of the driving situation acquired by sensors installed on the vehicle can also be taken into account additionally or alternatively.

[0015] According to a further advantageous embodiment of the method according to the invention, at least one path is a function defined by path elements and / or weighted path elements, preferably a polynomial of degree n, e.g. a straight line with path elements that are on average approximately equally spaced.

[0016] According to an advantageous embodiment of the method, the image parameters are for example brightness values, color values, gray tones or at least one of these gradients, and before determining the feature points in the camera image, the camera image is left unprocessed or is suitably processed to make contrast differences, brightness differences, etc. more visible. Other image parameters not explicitly mentioned here are also conceivable.

[0017] According to a further advantageous embodiment of the method according to the invention, the last found path element can be used as an initialization point, and further feature points can be selected along the path that are within a predetermined distance from the path. Projecting the last found feature point thus determined onto the path can indicate the position of the trailer edge. In this way, the actual position of the trailer edge can be further approximated. Information about the actually determined position of the trailer edge can be output and / or exchanged between software modules, for example, via a CAN and / or Ethernet system.

[0018] According to a further advantageous embodiment of the method according to the invention, the position of the trailer edge thus found can be tracked (updated) in successive camera images in time so that the trailer edge always appears approximately in the center of each successive camera image, which means that the driver of the vehicle can be more reliably aware of the traffic behind, even if the buckling angle assumed by the trailer relative to the tractor unit remains constant.

[0019] According to a further advantageous embodiment of the method according to the invention, further vehicle characteristics can be taken into account and used to correct the determined position of the trailer edge. Such vehicle characteristics include, for example, geometrical characteristics such as length, width, height, etc., which can also be permanently stored in advance for a particular type of trailer. Other vehicle characteristics, such as the position of the tail lights, can also be used for the correction. This information can also be obtained from other processes, for example via a CAN or Ethernet system, or provided by other software modules.

[0020] According to a further advantageous embodiment of the method according to the invention, the determined position of the trailer edge is verified by generating a block grid having a plurality of rows and columns, which define cells in at least a portion of the camera image. The block grid is preferably generated on the image in which the position of the trailer edge is determined. By generating the block grid, the determined feature points are distributed among a plurality of cells of the block grid, meaning that each cell contains a plurality of feature points. According to the invention, the distribution of feature points in the generated block grid is compared with the distribution of feature points in a corresponding plurality of stored block grids. The stored block grids define different distributions of feature points together with the actual trailer edge. From the stored plurality of block grids, the block grid that is most correlated with the generated block grid is selected for verifying the determined position of the trailer edge. A plausibility check can be performed by comparing the determined position of the trailer edge with the actual position of the trailer edge defined in the selected block grid. If necessary, if the determined position of the trailer edge deviates significantly from the actual position, it can be corrected based on this plausibility check, and the position of the trailer edge can be determined again.

[0021] According to a further advantageous embodiment of the method according to the invention, the distribution of feature points together with the corresponding positions of the trailer ends can be stored in advance in a block grid or generated and stored in real time at fixed or dynamically adjustable time intervals, and the determined distribution is then compared with such a predetermined distribution and the position of the trailer ends is verified as described above.

[0022] According to a further advantageous embodiment of the method according to the invention, the different distribution of the feature points in the block grids stored together with the corresponding trailer end depends on the buckling angle between the trailer and the towing unit. Information about the buckling angle can be stored, for example, together with each block grid. That is, if, for example, the current buckling angle or the relative position of the trailer and the towing unit is known, the stored block grid for this buckling angle can be found directly, and the determined trailer end position can be verified. Other information indicating the position of the trailer relative to the towing unit can also be used.

[0023] The camera-based system according to the invention comprises at least one camera with an image sensor for capturing time-sequential camera images of the side area of ​​the trailer, a monitor for displaying the images captured by the camera, and at least one processor configured to perform the above-mentioned method according to the invention. According to an advantageous further development, the camera-based system according to the invention is a mirror replacement system approved according to the UN ECE R46 regulation, preferably a mirror replacement system for commercial vehicles.

[0024] It may be provided in the following manner. [Aspect 1] 1. A method for determining the position of an end of a trailer (5) in a camera image (6) of a vehicle, the trailer (5) extending rearward from a towing unit (4) and rotatable relative to said towing unit (4), by processing camera images (6) taken in succession over time, comprising: capturing said camera images (6) successive in time by at least one image sensor of at least one image capturing means (2); determining a plurality of feature points (8) in each of the camera images (6) used in the method based on at least one image parameter, each feature point (8) corresponding to at least one pixel in the camera image (6); determining at least one path element (10) by using said plurality of feature points (8) along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy based on the determined positional displacements in corresponding coordinates x and / or y and / or dx and / or dy between said plurality of feature points (8) in at least one said camera image (6) and corresponding plurality of feature points (8) in at least one temporally consecutive camera image (6); generating at least one route (11) along the predetermined direction based on the route elements (10), the route (11) indicating the position of the trailer (5); determining a location on said path (11) as said trailer end. [Aspect 2] In the method according to aspect 1, the predetermined direction is determined by the density and position of the feature points (8). [Aspect 3] In the method of embodiment 1, weighting the feature points (8); determining the path elements (10) from the weighted feature points (8). [Aspect 4] In the method according to any one of aspects 1 to 3, weighting the path elements (10); generating the at least one path (11) based on the weighted path elements (10). [Aspect 5] In the method according to any one of aspects 1 to 4, obtaining information about the relative position between the trailer (5) and the towing unit (4) of the vehicle, The method further comprises a step in which the determination of the path elements, the weighting of the feature points (8) and / or the weighting of the path elements (10) is based on the obtained information regarding the relative positional relationship between the trailer (5) and the towing unit (4) of the vehicle. [Aspect 6] 2. The method according to any one of the preceding aspects, The method, wherein the at least one path (11) is generated as a function determined by the path elements (10) or weighted path elements (10). [Aspect 7] 2. The method according to any one of the preceding aspects, The method, wherein the at least one image parameter is at least one of a luminance value, a color value, a gray tone, a contrast value, and / or a calculation from one of these values ​​and / or a gradient thereof for pixels and / or pixel clusters in the camera image (6). [Aspect 8] 2. The method according to any one of the preceding aspects, setting an initialization point (IP) as a starting point, which is located on the generated path (11) or which is at least one path element (10) of the at least one path (11) or which is at least the further feature point (8) not exceeding a predetermined distance from the at least one path (11) or not exceeding a calculated amount of the path element (10) or the further feature point (8); starting from said starting point along said path (11) and searching for at least a further feature point (8) not exceeding a predetermined distance from said at least one path (11); and setting the coordinates of said at least one further identified feature point (8) and / or a calculated quantity of said at least one feature point (8) as said position (EP) of said trailer end. [Aspect 9] In the method according to any one of aspects 1 to 8, obtaining at least one vehicle feature; and revising the determined position of the trailer end based on the acquired at least one vehicle characteristic. [Aspect 10] 2. The method according to any one of the preceding aspects, further comprising the step of verifying the determined position of the trailer end, wherein the verifying step comprises: generating a block grid (13) defining a plurality of cells in at least a portion of the camera image (6), and determining a distribution of the plurality of feature points (8) over the plurality of cells (14) of the block grid (13) as part of the generated block grid (13); comparing the distribution of the feature points (8) within the generated block grid (13) with a plurality of corresponding stored block grid distributions of feature points, each of the stored block grids defining a different distribution of feature points along with an actual trailer end; selecting one of the stored block grids based on a correlation between a distribution of feature points (8) in the generated block grid (13) and the distribution of the feature points in the stored block grids; and verifying the determined position of the trailer end (12) by comparing the determined position of the trailer end (12) with the actual position of the trailer end defined by the selected block grid. [Aspect 11] 11. The method of claim 10, further comprising the step of modifying the determined position of the trailer end (12) based on the verification. [Aspect 12] In a method according to aspect 10 or 11, the distribution of the feature points and the corresponding trailer ends within the stored block grid are stored in advance or are generated and stored in real time at fixed or dynamically adjustable time intervals. [Aspect 13] In the method according to any one of aspects 10 to 12, The method, wherein the distribution of feature points in one of the stored block grids differs from the distribution of feature points in another of the stored block grids according to information about a relative positional relationship between a trailer (5) and a towing unit (4). [Aspect 14] 2. The method according to any one of the preceding aspects, tracking the determined position of the trailer end in successive camera images (6) in time as a function of the position of the trailer (5) relative to the image sensor acquiring the camera images (6) so that the trailer end appears in a preferred position in a monitor image of at least one display means (1) of a camera-based system. [Aspect 15] 1. A camera-based system for a vehicle having a towing unit (5) and a trailer (4) extending rearward and pivotable relative to the towing unit (5), comprising: at least one image acquisition means (2) provided on the towing unit (5) and comprising at least one image sensor for acquiring time-sequential camera images (6) of the area of ​​the trailer (5); At least one processing means configured to perform the method according to any one of aspects 1 to 14. [Aspect 16] 16. The camera-based system of claim 15, further comprising at least one display means; a camera-based system, wherein the at least one display means is for displaying camera images (6) taken by the at least one image capturing means (2) so that the trailer end appears in a preferred position in the monitor image of the display means (1), and for tracking the determined position of the trailer end in successive camera images (6) over time depending on the position of the trailer (5) relative to the image sensor acquiring the camera images (6). [Aspect 17] 17. The camera-based system of claim 15 or 16, Camera-based system approved according to UN ECE R46. [Brief explanation of the drawings]

[0025] The present invention will now be described, by way of example only, with reference to the accompanying drawings in which like reference numerals indicate the same or similar elements, in which:

[0026] [Figure 1] FIG. 1 is a schematic diagram of a camera-based system according to a preferred embodiment of the present invention.

[0027] [Figure 2a] FIG. 2a is a top view of a vehicle using the camera-based system of FIG. 1 in a first driving situation corresponding to straight forward driving.

[0028] [Figure 2b] FIG. 2b is a top view of a vehicle using the camera-based system of FIG. 1 in a second driving situation corresponding to cornering.

[0029] [Figure 3] FIG. 3 shows a camera image taken by the camera-based system of FIG. 1 as a display image on a monitor to illustrate a method for determining the position of the end of a trailer according to a preferred embodiment of the present invention.

[0030] [Figure 4]FIG. 4 shows the camera image according to FIG. 3 to illustrate the determination of feature points according to a preferred embodiment of the present invention.

[0031] [Figure 5] FIG. 5 is a schematic three-dimensional view of a portion of the camera images of FIG. 3 to illustrate the misalignment of feature points between two successive camera images according to a preferred embodiment of the present invention.

[0032] [Figure 6] FIG. 6 shows a schematic three-dimensional view of FIG. 5 to illustrate the determination of path elements according to a preferred embodiment of the present invention.

[0033] [Figure 7] FIG. 7 shows the camera image of FIG. 3 with path elements superimposed along the bottom edge of the trailer in accordance with a preferred embodiment of the present invention.

[0034] [Figure 8] FIG. 8 shows the camera image of FIG. 3 overlaid with alternative path elements along the trailer configuration according to the preferred embodiment.

[0035] [Figure 9] Figure 9 shows the camera image of Figure 7 with the movements of the feature points and route elements symbolized.

[0036] [Figure 10] FIG. 10 is a diagram illustrating the determination of trailer end position in accordance with a preferred embodiment of the present invention.

[0037] [Figure 11] FIG. 11 shows the trailer ends in the camera image of FIG. 9 as determined by a preferred embodiment of the present invention.

[0038] [Figure 12] FIG. 12 shows the tracked (updated) trailer end in the camera image of FIG. 3 according to a preferred embodiment of the present invention.

[0039] [Figure 13] FIG. 13 shows a block grid in accordance with a preferred embodiment of the present invention in the camera image of FIG.

[0040] [Figure 14] FIG. 14 shows feature points in the cells of the block grid of FIG. 13 according to a preferred embodiment of the present invention.

[0041] [Figure 15] FIG. 15 shows the tracked (updated) trailer end in the camera image of FIG. 3 according to a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0042] FIG. 1 shows a camera-based system according to a preferred embodiment of the present invention, comprising display means 1a, 1b, image capture means 2a, 2b and processing means 3a, 3b.

[0043] The image acquisition means 2a, 2b are preferably arranged on opposite sides of the vehicle and connected to the display means 1a, 1b via processing means 3a, 3b. The image acquisition means 2a, 2b are arranged on opposite sides of the vehicle, for example as shown in Figure 2. The processing means 3a, 3b according to a preferred embodiment comprise buckling angle determination means and image data processing means (not described in further detail) and are in particular able to carry out the inventive method for determining the position of the trailer end and for tracking (updating) and correcting the trailer end on the display means 1a, 1b as will be explained below.

[0044] For simplicity, only the display means 1a, the image capture means 2a and the processing means 3a will be described. However, the following description equally applies to the display means 1b, the image capture means 2b and the processing means 3b on the opposite side of the vehicle. Furthermore, according to a preferred embodiment, only one processing unit may be provided which performs the functions of the processing means 3a and 3b.

[0045] As shown in Figures 2a and 2b, the vehicle may include, for example, a towing vehicle 4 and a trailer 5 rotatably attached to the towing vehicle 4. The image capture means 2a has a recording area that covers a viewing area or field of view corresponding to "Main Mirror (Large) Group II" according to the ECE-R46 regulation. The image capture means 2a may also cover other appropriately defined fields of view based on national regulations.

[0046] FIG. 2a shows a state in which there is no buckling angle between the towing vehicle 4 and the trailer 5 and the vehicle is traveling straight ahead. When cornering, as shown in FIG. 2b, a buckling angle occurs between the towing vehicle 4 and the trailer 5, and as the buckling angle increases, the rear end of the trailer 5 shifts (moves) into the field of view of the image capturing means 2a. FIG. 3 shows a camera image 6 (monitor image) captured by the image capturing means 2a and displayed on a monitor; as the buckling angle increases, the end of the trailer 5 becomes more likely to obscure rearward traffic. The image capturing means 2a is configured to capture a series of images that are consecutive in time.

[0047] Figure 3 shows a camera image 6 taken by the camera-based system of Figure 1 as an image of a trailer 5 displayed on a monitor with traffic 7 behind it. Figure 4 serves to illustrate the determination of feature points 8 within this camera image 6.

[0048] According to a preferred embodiment of the method of the present invention for determining a trailer end or trailer edge in a camera image 6, a plurality of feature points 8 are determined in a two-dimensional camera image 6 having coordinate axes x and y, as shown in FIG. 4. Each feature point 8 corresponds to one or more pixels in a pixel cluster. FIG. 4 shows a pixel field 9 in a close-up view of a portion of the camera image 6. According to a preferred embodiment, pixels adjacent to, for example, pixels 1-5, are determined to be feature points if they have a predetermined contrast difference with at least some of their neighboring pixels. Other image parameters for determining whether a pixel is a feature point can be used alternatively or cumulatively.

[0049] According to a preferred embodiment of the method of the present invention, the feature points 8 determined in Fig. 4 are weighted to emphasize or better determine their contrast differences with neighboring or spatially nearby pixels. The weighting parameter is, for example, the speed at which the feature points change relative to the towing vehicle; i.e., points that do not change or only change slightly with respect to their position on the image sensor are weighted higher, for example, as they are assumed to be moving with the vehicle and therefore located on the trailer. Since only feature points with a predetermined weight or higher are used for further processing by the method of the present invention, it is not necessary to use all feature points, which saves computational capacity.

[0050] According to a preferred embodiment of the method according to the invention, the path elements 10 are selected from the feature points 8 shown in FIG. 4, as will be explained below with reference to FIGS.

[0051] According to a preferred embodiment of the method according to the invention, first, a start point 0 is defined in the camera image 6, as shown in FIG. 4. The start point 0 may be, but does not necessarily have to be, a feature point 8. The start point 0 can be set, for example, based on the buckling angle and the resulting position of the trailer relative to the image capture means. Starting from the start point 0, as shown in FIG. 5, an image section is defined, which in FIGS. 5 and 6 is shown as a three-dimensional block of a fixed or dynamically adjustable block size (bs) for better visualization. Within this three-dimensional block, there are a corresponding number of feature points of FIG. 4 distributed in three-dimensional space around the start point 0 as the center point of a block of this size (bs). Each of these feature points has coordinates x(bs x ), coordinate y(bs y ), coordinate dx (not shown in FIG. 5), and coordinate dy (bs dy The coordinates dx and dy represent the positional deviation (displacement) of the feature points between camera images captured successively in time.

[0052] From the "cloud of feature points" shown in Figure 5, at least one path element 10.1 is determined. According to a preferred embodiment of the method of the present invention, this path element corresponds to the center (centre of gravity) of the "cloud of feature points". The path element 10.1 can therefore be one of the feature points of the "cloud of feature points" or it can simply be the position within the block where the most feature points are spatially located. The path element 10.1 therefore does not necessarily have to be a feature point 8.

[0053] As shown in Figure 6, to select the next path element 10.2, starting from path element 10.1 as a new starting point, a new image section or 3D block is defined, for example with the same or a different block size as the initial block, depending on the feature points present in this section. The selection of path element 10.2 is performed similarly to the selection of path element 10.1. In this way, the path element 10 is determined from the feature points along a preferred direction R. According to a preferred embodiment of the method according to the invention, the preferred direction R is determined by the density and position of the feature points and can be a constant direction or can be dynamically adjusted during the method.

[0054] 7 shows a diagram in which a path element 10 determined by a preferred embodiment of the method of the present invention, together with feature points 8, is superimposed on the camera image 6. According to FIG. 7, the path element 10 is the lower edge of the trailer 5 and extends along the preferred direction R. Depending on the structure of the trailer 5 and a corresponding alternative distribution of feature points and / or path elements, the path element 10 may also extend, for example, as shown in FIG. 8, i.e. not along the lower edge of the trailer 5. In any case, the position of the trailer end or trailer edge can be determined by a preferred embodiment of the method of the present invention, as will be explained further below.

[0055] To better illustrate the determination of the path or trailer end, Figure 9 symbolically shows the movement of the determined path elements 10.1-10.11 or feature points 8 in the camera image 6. For better understanding, the path elements 10.1-10.11 and feature points 8 in Figure 9 are projected onto the coordinate system of Figure 10. Figure 10 shows the coordinate axes y, x (out of plane) and dy.

[0056] According to a preferred embodiment of the method according to the invention, a path 11 is generated based on the path elements, which in this preferred embodiment are straight lines. The path 11 can also be generated as a polynomial of degree n, and multiple different paths with different polynomial degrees can also be generated. The path elements 10 are weighted at least in part based on their position or order (order) before the path 11 is generated. A metric comprising the weight of the path elements can be used, but also characteristics of the coordinates of the path elements and / or the set of feature points they represent, such as the number of feature points, the variance of the coordinate values ​​of the feature points, the variance of the "score values", i.e. the characteristics of the transformation of feature points into feature points.

[0057] As shown in Figure 10, all path elements 10.1 to 10.9 are within a predetermined distance from path 11. The determined path elements 10.10 and 10.11 do not belong to trailer 5 but to the curb, as shown in Figure 9, and are therefore outside the predetermined distance and are therefore not on path 11. As shown in Figures 9 and 10, path elements 10.10 and 10.11 move faster than path elements 10.1 to 10.9 when cornering, and therefore can be identified as not belonging to path 11.

[0058] According to a preferred embodiment of the method of the present invention, to determine the location of the trailer end, the last path element 10.9 considered for determining the path 11 and identified with sufficient certainty as belonging to the trailer is used as the initial point for analyzing further feature points 8 with respect to their spatial proximity to the path 11. For example, in the previous steps, feature points may not have been identified as belonging to the path 11 due to a coarse threshold, or the orientation of the image section or block shown in FIG. 6 for determining the path elements may have been different. In this way, the method of the present invention is further improved, and the trailer end is depicted at the coordinates of the last feature point found within a predetermined distance from the path 11. Alternatively, the trailer end can be calculated from several last feature points found. In FIGS. 9 and 10, the trailer end is indicated by EP or a. This means that the location of the trailer end is the furthest from the towing unit when viewed away from the towing vehicle, and is determined as the position on the path where the image parameters determined for selecting feature points always change in this direction for the first time (i.e., not immediately after a short break along the path, for example). The initial point IP can be on the path and / or at least one path element and / or at least one feature point at a predetermined distance from the path. Calculated variables (calculations) of multiple path elements and / or feature points can be used, such as mean values, centroids, least square deviations, etc.

[0059] FIG. 11 shows the projection of the perpendicular line 12 at coordinate a of the last feature point found in the camera image 6 to indicate the trailer end determined by the method according to the invention.

[0060] According to a preferred embodiment of the method of the present invention, the trailer end position determined by the above-described method is further used to track (update) the trailer end in a monitor image displayed to the vehicle driver while the vehicle is cornering, so that the trailer end appears in a preferred position in the monitor image, for example, at the center or 0% to 25% offset to the left or right of the center, as shown in Figure 12, in order to ensure reliable monitoring of traffic, particularly behind.

[0061] According to a preferred embodiment of the method of the present invention, the trailer edge determined by the above-described method is further verified. For this purpose, a block grid 13 is generated in a portion of the camera image 6, as shown in Figure 13. According to a preferred embodiment of the method of the present invention, the block grid 13 has a fixed shape and a fixed number of rows and columns forming a number of cells 14. In the block grid 13 of Figure 13, the number of columns differs from the number of rows. In principle, grid layouts other than rectangular cells can also be used.

[0062] According to a preferred embodiment of the method of the present invention, the distribution of the plurality of feature points 8 determined in the above-described manner is determined over a plurality of cells 14. For example, the cell 14 shown in FIG. 14 contains two feature points 8. The distribution of feature points over the cells of the generated block grid 13 is then compared with the distribution of feature points in a plurality of corresponding stored block grids (not shown). Each stored block grid defines a different distribution of feature points along with the position and buckling angle of the actual trailer end. According to a preferred embodiment of the method of the present invention, the corresponding stored block grid in which the actual trailer end is defined is selected based on information about the buckling angle and / or position of the trailer end, or the relative positional relationship between the towing unit and the trailer, and the correlation between the distribution of feature points in the generated block grid and the stored block grid. This actual trailer end can be verified by comparing it with a previously determined (input) trailer end and, if necessary, rejected or corrected.

[0063] According to a preferred embodiment of the method of the present invention, the distribution of feature points and corresponding trailer ends within the stored block grid can be pre-stored or generated and stored in real time during execution of the method at fixed or dynamically selected time intervals.

[0064] By using a block grid that stores information about the trailer edge in addition to the distribution of feature points on the cells, the trailer edge can be determined even when there is no trailer edge to verify by comparing the generated block grid with the stored block grid, and the trailer edge can also be used to track the center within the camera image as described above.

[0065] 15 shows the tracked (updated) position of the trailer edge in camera image 6 at a preferred position within the displayed monitor image. This preferred position may be, for example, the center of the monitor image or a position offset 0%-25% to the left or right of the center. The preferred position may also be offset 0%-25% from the left or right edge of the monitor image.

[0066] It is emphasized that all features disclosed in the specification and / or claims are understood to be separate and independent from one another, not only for the purposes of the original disclosure, but also for the purposes of limiting the claimed invention independently of any combination of features in the embodiments and / or claims. The designation of ranges or groups of units expressly includes any intermediate values ​​or subgroups of units, not only for the purposes of the original disclosure, but also for the purposes of limiting the claimed invention, and in particular for the purposes of limiting the range. [Explanation of symbols]

[0067] 0: starting point 1a, 1b: Display means 2a, 2b: image capturing means 3a, 3b: Processing means 4: Towing vehicle or towing unit 5: Trailer 6: Camera image 7: Rear traffic 8: Feature points 9: Pixel field 10: Route element 11: Route 12: Line indicating the end of the trailer 13: Block grid 14: Cell R: Prescribed direction IP: Initialization point VP: Preferred Position EP: End Point

Claims

1. 1. A method for determining the position of an end of a trailer in a camera image of a trailer extending rearward from a towing unit of a vehicle and rotatable relative to said towing unit, by processing camera images taken in succession in time by at least one processing means, comprising: The at least one processing means performs the following steps: In the imaging step, at least one image sensor of at least one image capturing means captures the camera images successive in time; the feature point determining step determines a plurality of feature points in each of the camera images used in the method based on at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; In the path element determining step, based on the determined positional displacements in corresponding coordinates x and / or y and / or dx and / or dy between the plurality of feature points in at least one of the camera images and corresponding plurality of feature points in at least one temporally consecutive camera image, at least one path element is determined by using the plurality of feature points along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy; In the generating step, at least one route is generated along the predetermined direction based on the route elements, where the route indicates the position of the trailer, and where the predetermined direction is determined by the density and position of the feature points; The method of claim 1, wherein the step of determining a position determines a position on the route as the trailer end.

2. 10. The method of claim 1, The at least one processing means further performs the following steps: In the weighting step, the feature points are weighted; In the determining step, the route elements are determined from the weighted feature points.

3. 10. The method of claim 1, The at least one processing means further performs the following steps: In the weighting step, weights are assigned to the route elements; In the generating step, the at least one route is generated based on the weighted route elements.

4. 10. The method of claim 1, The at least one processing means further performs the following steps: In the obtaining step, information regarding a relative positional relationship between the trailer and the towing unit of the vehicle is obtained; wherein the determination of the route elements, the weighting of the feature points, and / or the weighting of the route elements is performed based on the acquired information regarding the relative positional relationship between the trailer and the towing unit of the vehicle.

5. 10. The method of claim 1, The method, wherein the at least one path is generated as a function specified by the path elements or weighted path elements.

6. 10. The method of claim 1, 10. The method of claim 9, wherein the at least one image parameter is at least one of a luminance value, a color value, a gray tone, a contrast value, and / or a calculated quantity from one of these values ​​and / or a gradient thereof for a pixel and / or a pixel cluster in a camera image.

7. A method for determining the position of a trailer end in a camera image of a trailer extending rearward from a towing unit of a vehicle and rotatable relative to said towing unit by processing camera images taken successively in time by at least one processing means, comprising: The at least one processing means performs the following steps: In the imaging step, at least one image sensor of at least one image capturing means captures the camera images successive in time; the feature point determining step determines a plurality of feature points in each of the camera images used in the method based on at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; In the path element determining step, based on the determined positional displacements in corresponding coordinates x and / or y and / or dx and / or dy between the plurality of feature points in at least one of the camera images and corresponding plurality of feature points in at least one temporally consecutive camera image, at least one path element is determined by using the plurality of feature points along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy; In the generating step, at least one route is generated along the predetermined direction based on the route elements, wherein the route indicates the position of the trailer; In the setting step, an initialization point (IP) is set as a starting point; the initialization point is located on the generated path, or is at least one path element of the at least one path, or is at least the further feature point not exceeding a predetermined distance from the at least one path or not exceeding a calculated amount of the path element or further feature point; a searching step starting from the starting point along the path and searching for at least a further feature point not more than a predetermined distance from the at least one path; In the position setting step, the coordinates of at least one further searched feature point and / or the calculated quantity of the at least one feature point are set as the position (EP) of the trailer end.

8. 10. The method of claim 1, The at least one processing means further performs the following steps: In the vehicle feature acquisition step, at least one vehicle feature is acquired; A method in which a modifying step modifies the determined position of the trailer end based on the acquired at least one vehicle characteristic.

9. A method for determining the position of a trailer end in a camera image of a trailer extending rearward from a towing unit of a vehicle and rotatable relative to said towing unit by processing camera images taken successively in time by at least one processing means, comprising: The at least one processing means performs the following steps: In the imaging step, at least one image sensor of at least one image capturing means captures the camera images successive in time; the feature point determining step determines a plurality of feature points in each of the camera images used in the method based on at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; In the path element determining step, based on the determined positional displacements in corresponding coordinates x and / or y and / or dx and / or dy between the plurality of feature points in at least one of the camera images and corresponding plurality of feature points in at least one temporally consecutive camera image, at least one path element is determined by using the plurality of feature points along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy; In the generating step, at least one route is generated along the predetermined direction based on the route elements, wherein the route indicates the position of the trailer; In the position determination step, a position on the route as the trailer end is determined; A verification step verifies the determined position of the trailer end, wherein the verification step includes the steps of: In the generating and determining step, a block grid defining a plurality of cells is generated in at least a portion of the camera image, and a distribution of the plurality of feature points on the plurality of cells of the block grid is determined as part of the generated block grid; a comparing step of comparing the distribution of feature points within the generated block grid with distributions of feature points of a plurality of corresponding stored block grids, each of the stored block grids defining a different distribution of feature points along with an actual trailer end; a selection step of selecting one of the stored block grids based on a correlation between a distribution of feature points in the generated block grid and a distribution of feature points in the stored block grid; The verification step includes: (a) verifying the determined position of the trailer end by comparing the actual position of the trailer end defined by the selected block grid with the determined position of the trailer end; and / or (b) further comprising a modifying step, wherein the determined position of the trailer end is modified based on the verification; and / or (c) the distribution of feature points and the corresponding trailer ends within the stored block grid are pre-stored or are generated and stored in real time at fixed or dynamically adjustable time intervals.

10. 10. The method of claim 9, The method of claim 1, wherein the distribution of feature points in one of the stored block grids differs from the distribution of feature points in another of the stored block grids according to information about a relative positional relationship between a trailer and a towing unit.

11. 10. The method of claim 1, The at least one processing means further performs a tracking step; In the tracking step, the determined position of the trailer end is tracked in successive camera images in time depending on the position of the trailer relative to the image sensor that acquires the camera images so that the trailer end appears in a desired position in a monitor image of at least one display means of a camera-based system.

12. 1. A camera-based system for a vehicle having a towing unit and a trailer extending rearward and pivotable relative to the towing unit, comprising: at least one image capture means provided on the towing unit and comprising at least one image sensor for capturing time-sequential camera images of the area of ​​the trailer; and at least one processing means configured to perform the method according to any one of claims 1 to 11.

13. 13. The camera-based system of claim 12, further comprising at least one display means; A camera-based system, wherein the at least one display means displays camera images captured by the at least one image capturing means so that the trailer end appears in a preferred position within the monitor image of the display means, and tracks the determined position of the trailer end within successive camera images over time depending on the position of the trailer relative to the image sensor that captures the camera images.

14. 13. The camera-based system of claim 12, Camera-based system approved according to UN ECE R46.

Images