Camera-based trailer position determination system and method

Abstract

A method and camera-based system for determining an end position of a trailer that extends rearward from a traction unit of a vehicle and is rotatable by processing camera images captured in succession times, the method performed by the camera-based system comprises the steps of: capturing camera images continuously in time by an image sensor of at least one image capturing device of the camera-based system; determining a plurality of feature points in each camera image according to the at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; selecting a relevant feature point from the plurality of feature points as a path element in a preferred direction defined by the density and position of the feature points according to a temporally successive position displacement (dx, dy) between the plurality of feature points in the one camera image and the corresponding plurality of feature points in the one camera image; generating at least one path describing the trailer along the preferred direction according to the path elements; and determining the position on the path as the tail end of the trailer.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the priority of German Patent Application No. 102023131347.9 filed on November 10, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The invention relates to a camera-based system (observation system or rearview mirror replacement system) and a method for determining the position of the end (edge) of a trailer extending rearwards, which is pivotally connected to the vehicle, suitable for use with a semitrailer, trailer or other commercial vehicle having a pivotally connected cab or tractor unit and extending rearwards. Background Art

[0004] For vehicles with a tractor unit or with a trailer extended to the rear and swivelled to the tractor unit, the driver of the vehicle usually observes the traffic behind via side mirrors attached to the sides of the tractor unit. Such side mirrors are increasingly being replaced by camera systems, such as camera monitoring systems, mirror replacement systems mounted on the sides or rear of the vehicle, respectively. They replace or supplement the conventional mirror systems prescribed for motor vehicles, such as exterior mirrors (main mirrors) on cars, interior mirrors on cars or wide-angle mirrors and front and rear mirrors on commercial vehicles.

[0005] In the above-mentioned systems, for example, the corresponding field of view which is usually visible via a rearview mirror is permanently displayed in real time to the vehicle driver on a monitor or other display unit (e.g. inside the vehicle), so that the vehicle driver always sees the corresponding field of view even though he neither directly sees the corresponding field of view nor has an existing rearview mirror.

[0006] According to current technology, trailers include trailers connected to a vehicle (tractor or trailer unit) via a trailer coupler, such as a semitrailer (so-called trailer), and placed in the rear lowered area of ​​the tractor unit and pivotably connected around at least one vertical axis. Alternatively, such trailers can also be trailers connected to a passenger car, in which case the passenger car is equivalent to the tractor unit. Generally speaking, the trailer is the part that is located behind and extends backward relative to the vehicle cab, and is also the part that is movable (pivotable) to one side relative to the cab, and pivots relative to the cab around a vertical axis when turning. Articulated trucks or articulated trains are also trailers referred to in the present invention.

[0007] However, when using a rearview mirror replacement system as described above, when a vehicle with a towing unit and a trailer turns, as the flexion angle (kink angle or bend angle) between the towing unit and the trailer increases, rear traffic becomes increasingly hidden in the image of the rearview mirror replacement system of field of view category II according to ECE R46. In addition, the rear of the trailer is lost (disappears) near the rear axle, which may cause the end of the trailer to disappear from the monitor image displayed to the driver.

[0008] In order to solve this problem, there are methods for tracking the end edge of the trailer and moving the image portion displayed on the monitor so that the end edge of the trailer is displayed as close to the center of the monitor image as possible. According to EP 16 198 485, the tracking of the end edge of the trailer is currently carried out by, for example, vehicle sensors, which obtain information about the rotational movement of the wheels. This information is analyzed by a control unit to determine the rear area of ​​the rearward extension and to perform appropriate tracking of this rear area based on the information obtained about the rotational movement of the wheels. However, since these sensors can only provide reliable signals above a certain speed, for example from about 2 km / h, tracking of the end edge of the trailer can only be provided when driving forward at higher speeds. These sensors cannot be used to detect the end edge of the trailer with sufficient accuracy during backward maneuvers and at low speeds and to track (update) the end edge of the trailer in the monitor image. Summary of the invention

[0009] The object of the present invention is to solve the above-mentioned problems and to detect the position of the end of the trailer with sufficient accuracy for tracking (updating) in the monitor image during rearward maneuvers and at low speeds, independently of information obtained by, for example, the type of trailer and the installed sensors.

[0010] This problem is solved by a method having the features of claim 1 and a system having the features of claim 15. Advantageous further developments of the invention are given in the dependent claims.

[0011] The method according to the invention for determining the position of the trailer end or the trailer end edge of a trailer extending rearwardly from a towing unit of a vehicle processes camera images captured in time and continuously by an image sensor of a camera-based system. The camera images for processing do not have to be captured directly and continuously, for example, only the second or third camera image may be used for further processing, which may reduce the amount of image data to be processed. Before processing the captured camera images, their resolution, contrast and / or color information may be appropriately processed. In the method according to the invention, the raw data of the captured camera images may be used, or partially processed camera images may be used.

[0012] When further processing the successive camera images, the method according to the invention determines in each camera image a so-called feature point based on one or more image parameters, at least one or more image parameters and / or calculated quantities of their gradients, wherein the feature point corresponds to at least one pixel within the camera image. A pixel in a camera image may be determined as a feature point if, for example, the value of the image parameter of the pixel differs from the value of the corresponding image parameter of one or more neighboring pixels by a preset value.

[0013] For example, several adjacent pixels forming a pixel cluster may also be compared with adjacent or proximate pixel clusters with respect to at least one image parameter used to determine a feature point. Pixels or pixel clusters do not have to be directly adjacent, and spatial proximity between them may be sufficient.

[0014] The method according to the invention then selects a plurality of path elements from the determined feature points based on the position displacement (position offset) between a plurality of feature points in the camera image and a corresponding plurality of feature points in a subsequent camera image. The determination of the path elements is carried out, for example, along a predetermined direction of the trailer in the direction of the trailer tail and is preferably determined by the density and / or position of the feature points. The predetermined direction can be fixed or preferably dynamically adjusted. The dynamic adjustment can be based on already defined path elements. Based on these path elements, the method according to the invention then generates at least one so-called path (path), which describes the trailer path and determines a position (location) on the at least one path as the position (location) of the trailer end edge. The determined path elements or feature points have associated x, y, dx, dy coordinates, wherein the dx and dy coordinates represent a position offset or position shift in the x and y directions.

[0015] According to an advantageous embodiment of the method according to the invention, the feature points are weighted before the path elements are selected. For example, such weighting can be based on the contrast difference between a pixel or a pixel cluster and adjacent pixels or pixel clusters, or close pixels or pixel clusters. One or more known image parameters can serve as a basis for the weighting. Furthermore, the position offset (dx, dy) between a feature point in a camera image and a corresponding feature point in a subsequent camera image can also be used as an alternative or additional basis for the weighting.

[0016] According to another advantageous embodiment of the method according to the invention, the above-mentioned selection of path elements, weighting of characteristic points and / or weighting of path elements can be based on the flexion angle (kink angle) and / or information about the relative position relationship between tractor unit and trailer and / or vehicle speed. This flexion angle or relative position relationship can be determined independently of the acquired camera image, for example by a flexion angle sensor installed on the vehicle, or can be estimated by data received from a steering angle sensor. Alternatively, it can also be estimated by image analysis. Other signals of the driving state obtained by sensors installed in the vehicle can also be taken into account in addition.

[0017] According to another advantageous embodiment of the method according to the invention, the at least one path is a mathematical function, preferably an nth order polynomial, defined by path elements and / or weighted path elements, such as a straight line with path elements being substantially equally spaced.

[0018] According to an advantageous embodiment of the method, the image parameter is, for example, at least one of a brightness value, a color value, a gray tone or a gradient of these values, wherein before determining the feature points in the camera image, the camera image remains unprocessed or is suitably processed in order to make contrast differences, brightness differences, etc., even more clearly visible. Other image parameters not explicitly listed here are also possible.

[0019] According to another advantageous embodiment of the method according to the invention, the last found path element can be used as an initialization point and further feature points can be selected along the path within a predetermined distance from the path. The projection of the last found feature point determined in this way onto the path can indicate the position of the trailer end edge. In this way, the actual position of the trailer end can be further approximated. Information about the actually determined position of the trailer end can be output, for example, via a CAN and / or Ethernet system and / or can be exchanged between software modules.

[0020] According to a further advantageous embodiment of the method according to the invention, the position of the trailer end edge found in this way can be continuously tracked (updated) in the camera images so that the trailer end always appears substantially in the center of each camera image that follows one another. This means that even at a given flexion angle of the trailer relative to the towing unit, the driver of the vehicle can perceive the traffic behind more reliably.

[0021] According to another advantageous embodiment of the method according to the invention, further vehicle characteristics can be taken into account and used to correct the determined position of the trailer end edge. These vehicle characteristics can include geometric characteristics such as length, width, height, etc. and can be permanently stored in advance for a specific type of trailer. Other vehicle characteristics, such as the position of the rear position lights can be used for correction. This information can be obtained from another process, for example via a CAN- or Ethernet system, or can be provided by other software modules.

[0022] According to another advantageous embodiment of the method according to the invention, the determined position of the trailer end edge is verified by generating a block grid with a plurality of row and column defining cells on at least a portion of the camera image. The block grid is preferably created on the image in which the position of the trailer end edge has been determined. By creating the block grid, the determined feature points are distributed over a plurality of cells of the block grid. This means that each cell consists of several feature points. According to the invention, the distribution of the feature points within the created block grid is compared with the distribution of the feature points in a plurality of corresponding stored block grids. The stored block grids define different distributions of feature points as well as the actual trailer end. From the stored block grids, the block grid that is most relevant to the created block grid is selected to verify the determined position of the trailer end edge and a plausibility check can be performed by comparing the determined trailer end position with the actual position of the trailer end defined in the selected block grid. If necessary, the determined position of the trailer end can be corrected according to this plausibility check if the deviation from the actual position is too large or the position of the trailer end can be determined again.

[0023] According to another advantageous embodiment of the method according to the invention, the distribution of the characteristic points and the corresponding position of the trailer end can be pre-stored in a block grid or can be generated and stored in real time at fixed or dynamically adjustable time intervals. The determined distribution is then compared with the predetermined distribution, thereby verifying the position of the trailer end.

[0024] According to another advantageous embodiment of the method according to the invention, the different distribution of the characteristic points of the respective trailer ends in the storage block grids depends on the buckling angle between the trailer and the tractor unit. For example, information about the buckling angle can be stored with each block grid. This means that, for example, if the current buckling angle or the relative position relationship between the trailer and the tractor unit is known, the storage block grid for this buckling angle can be directly found and the determined position of the trailer end can be verified. Other information can also be used to indicate the position of the trailer relative to the tractor unit.

[0025] The camera-based system according to the invention comprises at least one camera having an image sensor for continuously capturing (acquiring) camera images of the side area of ​​the trailer, a monitor for displaying the images captured by the camera, and at least one processor configured to perform the above-described method according to the invention. According to an advantageous further development, the camera-based system according to the invention is a rearview mirror replacement system approved according to UNECE R46, preferably for commercial vehicles.

[0026] Aspects of the Invention 1. A method for determining the position of the end of a trailer in a camera image (6) of a trailer (5) by processing the camera image (6) captured continuously over time, wherein the trailer (5) is rotationally connected to a tractor unit (4) of a vehicle and extends rearwardly from the tractor unit (4), comprising the following steps: capturing the camera image (6) continuously in time by at least one image sensor of at least one image capture device (2); Determining a plurality of feature points (8) in each camera image (6) used in the method based on at least one image parameter, wherein the feature point (8) corresponds to at least one pixel in the camera image (6); Determining at least one path element (10) along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy using a plurality of feature points (8), the determination being based on positional offsets in the respective coordinates x and / or y and / or dx and / or dy between the plurality of feature points (8) in the at least one camera image (6) and a corresponding plurality of feature points (8) that are temporally consecutive in the at least one camera image (6); generating at least one path (11) along the predetermined direction based on the path elements (10), the path (11) describing the position of the trailer (5); and A position on the path (11) is determined as the end of the trailer.

[0027] 2. The method according to claim 1, wherein the predetermined direction is determined by the density and position of the feature points (8).

[0028] 3. The method according to claim 1, further comprising the steps of: Weighting the feature points (8); and Path elements (10) are determined from the weighted feature points (8).

[0029] 4. The method according to any one of claims 1 to 3, further comprising the steps of: weighting the path elements (10); and The at least one path (11) is generated based on the weighted path elements (10).

[0030] 5. The method according to any one of claims 1 to 4, further comprising the steps of: Information about the relative positional relationship between the trailer (5) and the vehicle traction unit (4) is obtained, wherein the determination of the path elements, the weighting of the characteristic points (8) and / or the weighting of the path elements (10) are based on the information about the relative positional relationship between the trailer (5) and the vehicle traction unit (4).

[0031] 6. The method according to any of the preceding claims, wherein the at least one path (11) is generated as a mathematical function specified by a path element (10) or a weighted path element (10).

[0032] 7. Method according to any of the preceding claims, wherein the at least one image parameter is at least one of a brightness value, a color value, a grayscale, a contrast value of a pixel and / or a pixel cluster in the camera image (6) and / or a quantity calculated from these values ​​and / or their gradients.

[0033] 8. The method according to any one of the preceding claims, further comprising Setting an initialization point (IP) as a starting point, the initialization point being located on the generated path (11), or at least one path element (10) in at least one path (11), or a further feature point (8) not exceeding a predetermined distance from at least one path (11), or a calculated amount of the path element (10) or the further feature point (8); and Searching for at least one further feature point (8) starting from the starting point and along the path (11), which is no more than a predetermined distance from the at least one path (11); and The coordinates of the at least one further determined characteristic point (8) and / or the calculated number of the at least one characteristic point (8) are provided as the position (EP) of the trailer end.

[0034] 9. The method according to any one of claims 1 to 8, further comprising the steps of: obtaining at least one vehicle characteristic; and The determined position of the end of the trailer is revised based on the acquired at least one vehicle characteristic.

[0035] 10. A method according to any preceding claim, further comprising the step of verifying that the end of the trailer has been located, wherein the verifying step comprises the steps of: Generating a block grid (13) on at least a portion of the camera image (6), wherein the block grid (13) defines a plurality of cells (14), and determining a distribution of a plurality of feature points (8) on the plurality of cells (14) of the block grid (13) as a portion of the generated block grid (13); comparing the distribution of feature points (8) in the generated block grid (13) with the distribution of feature points of a plurality of stored corresponding block grids, wherein the stored block grids each define a different distribution of feature points and an actual trailer end; Selecting one of the storage block grids based on a correlation between the distribution of the feature points (8) in the generated block grid (13) and the distribution of the feature points in the storage block grids; The determined position of the trailer end (12) is verified by comparing the actual position of the trailer end defined by the selected block grid with the determined position of the trailer end (12).

[0036] 11. The method according to claim 10, further comprising a correction of the determined position of the trailer end (12) based on the verification.

[0037] 12. The method according to claim 10 or 11, wherein the distribution of characteristic points and corresponding trailer ends in the stored block grid is pre-stored or generated and stored in real time at fixed defined or dynamically adjustable time intervals.

[0038] 13. A method according to any one of claims 10 to 12, wherein the distribution of feature points in one stored block grid differs from the distribution of feature points in another stored block grid based on information about the relative positional relationship between the trailer (5) and the tractor unit (4).

[0039] 14. A method according to any preceding claim, comprising Based on the position of the trailer (5) relative to the image sensor that acquires the camera image (6), the determined position of the trailer end in the camera image (6) is tracked over a continuous period of time so that the trailer end appears at a preferred position in the monitor image of at least one display device (1) of the camera-based system.

[0040] 15. A camera-based vehicle system having a tractor unit (4) and a trailer (5) extending rearwardly and rotatable to the tractor unit (4), comprising: at least one image capture device (2) arranged on the tractor unit (4) and having at least one image sensor for capturing a time-continuous camera image (6) of the area of ​​the trailer (5); and At least one processing device (3) is configured to perform a method according to any one of claims 1 to 14.

[0041] 16. A camera-based system according to claim 15, further comprising at least one display device (1) for displaying at least camera images (6) captured by the image capture device (2) and tracking the determined position of the trailer end in the camera images (6), the images being consecutive and depending on the position of the trailer (5) relative to the image sensor acquiring the camera images (6), so that the trailer end appears at a preferred position in the monitor image of the display device (1).

[0042] 17. A camera based system according to claim 15 or 16, being compliant with UN Economic Commission for Europe Regulation No. 46 (UN RCE R46). BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In the following, the present invention is described more fully by way of examples with reference to the accompanying drawings, in which the same reference characters designate the same or similar parts.

[0044] Figure 1 A schematic diagram showing a preferred embodiment of a camera-based system according to the present invention; Figure 2a Displays the first driving situation corresponding to straight ahead driving, using Figure 1 A top view of a vehicle in a camera-based system; Figure 2b Display the second driving situation corresponding to the turn, use Figure 1 A top view of a vehicle in a camera-based system; Figure 3 Show use Figure 1 A camera image captured by a camera-based system as an image displayed on a monitor for explaining a method for determining a trailer end position according to a preferred embodiment of the present invention; Figure 4 Show according to Figure 3 A camera image to illustrate the determination of feature points according to a preferred embodiment of the present invention; Figure 5 Show according to Figure 3 A schematic diagram of a three-dimensional representation of a portion of a camera image, used to illustrate the position offset between feature points of two consecutive camera images according to a preferred embodiment of the present invention; Figure 6 Show according to Figure 5 A three-dimensional representation schematic diagram is used to illustrate the determination of path elements according to a preferred embodiment of the present invention; Figure 7 The path elements superimposed along the lower edge of the trailer are shown according to a preferred embodiment of the present invention. Figure 3 camera image; Figure 8 The preferred embodiment of the present invention shows that alternative path elements are superimposed along the structure of the trailer according to Figure 3 camera image; Fig. 9 Displays the basis for symbolic representation of motion with feature points and path elements Figure 7 camera image; Fig.10 Shown for illustrating the determination of the position of the trailer end according to a preferred embodiment of the present invention; Fig.11 Display having the trailer end position determined according to the preferred embodiment of the present invention Fig. 9 The camera image shown; Fig.12 Display having tracked (updated) trailer end according to a preferred embodiment of the present invention Figure 3 The camera image shown; Fig.13 A block grid showing a preferred embodiment of the present invention Figure 3 The camera image shown; Fig.14 The preferred embodiment of the present invention has characteristic points. Fig.13 A cell map of the block grid; Fig.15 A diagram showing the end of a trailer being tracked (updated) according to a preferred embodiment of the present invention. Figure 3 The camera image shown. DETAILED DESCRIPTION

[0045] Figure 1 A camera based system according to a preferred embodiment of the invention is shown having display means 1a, 1b, image capturing means 2a, 2b and processing means 3a, 3b.

[0046] The image capture devices 2a, 2b are preferably arranged on opposite sides of the vehicle and are connected to the display devices 1a, 1b respectively via processing devices 3a, 3b. The image capture devices 2a, 2b are arranged on opposite sides of the vehicle, as shown in FIG2. The processing devices 3a, 3b according to a preferred embodiment comprise flexion angle determination means and image data processing means, not described in more detail, and are in particular capable of executing the method according to the invention for determining the position of the trailer end, for tracking (updating) and correcting the trailer end on the display devices 1a, 1b, as described below.

[0047] For simplicity, only the display device 1a, the image capture device 2a and the processing device 3a are described. However, the explanation is similarly applicable to the display device 1b, the image capture device 2b and the processing device 3b located on the opposite side of the vehicle. In addition, according to a preferred embodiment, only a single processing unit having the same functions as the processing devices 3a and 3b may be provided.

[0048] like Figure 2a and 2b As shown, the vehicle comprises a towing vehicle 4 and a pivotally connected trailer 5. The image capture device 2a covers with its recording area a viewing area or field of view which corresponds to the "primary mirror (large) group II" according to ECE-R46. Depending on national regulations, the image capture device 2a may cover other suitable defined fields of view.

[0049] Figure 2a The vehicle is shown traveling straight ahead with no camber between the towing vehicle 4 and the trailer 5. When turning, as Figure 2b As shown, a flexion angle is generated between the towing vehicle 4 and the trailer 5, and as the flexion angle increases, the rear end of the trailer 5 is displaced (moved) in the field of view of the image capturing device 2a. Figure 3 A camera image 6 captured by the image capture device 2a on a monitor (monitor image) is shown, wherein the end of the trailer 5 hides the following traffic at an increasing flexion angle. The image capture device 2a is configured to capture a series of images successively in time.

[0050] Figure 3 Shown by Figure 1 A camera image 6 is captured by a camera-based system as an image of the trailer 5 and the traffic 7 behind it displayed on a monitor. Figure 4 It is used to explain the determination of the feature point 8 in the camera image 6.

[0051] According to a preferred embodiment of the method for determining a trailer end or a trailer end edge in a camera image 6 according to the invention, a plurality of feature points 8 are determined on a two-dimensional camera image 6 having coordinate axes x and y, such as Figure 4 Each of the feature points 8 corresponds to a single pixel or several pixels in a pixel cluster. Figure 4 A pixel field 9 is shown in a partially enlarged view of a camera image 6. According to a preferred embodiment, a pixel having neighboring pixels 1 to 5 is determined to be a feature point, for example, if there is a predetermined contrast difference for at least a portion of the neighboring pixels. Other image parameters for determining whether a pixel is a feature point may be used alternatively or cumulatively.

[0052] According to a preferred embodiment of the method according to the invention, Figure 4The feature points 8 determined in the image sensor are weighted in order to emphasize or better determine the contrast differences between adjacent pixels or spatially neighboring pixels. The weighting parameter can be, for example, the speed with which the feature points change with respect to the traction, i.e. with respect to their position on the image sensor: points with no or only slight changes can be weighted higher, since it can be assumed that they are points that move with the vehicle and are therefore on the trailer. For further processing according to the method of the invention, for example, only feature points with a predetermined weight or a higher weight are applied, so that not all feature points have to be applied, so that not all feature points are further processed, which saves computing power.

[0053] According to a preferred embodiment of the method according to the present invention, Figure 4 Select the path element 10 from the feature point 8 shown, as described below Figure 5 and Figure 6 described.

[0054] According to a preferred embodiment of the method according to the present invention, a starting point 0 is first defined in the camera image 6, such as Figure 4 As shown. The starting point 0 may be, but need not be, a feature point 8. For example, the starting point 0 may be set based on the flexion angle and the resulting position of the trailer relative to the image capture device. Figure 5 As shown, starting from the starting point 0, an image part is defined. For better visualization, Figure 5 and Figure 6 It is shown as a three-dimensional block with a fixed or dynamically adjustable block size (bs). In this three-dimensional block, there are Figure 4 The corresponding multiple feature points in the block are distributed in the three-dimensional space near the starting point 0 as the center point of the block size (bs). The features of each feature point are the x coordinate (bsx), y coordinate (bsy), dx coordinate ( Figure 5 The dx coordinate and dy coordinate represent the position offset (displacement) of a feature point between camera images captured consecutively in time.

[0055] from Figure 5 In the "feature point cloud" shown, at least one path element 10.1 is determined. In a preferred embodiment of the method according to the invention, the path element corresponds to the center (center of gravity) of the "feature point cloud". Therefore, the path element 10.1 can be one of the feature points of the "feature point cloud" or only exists at the position where the most feature points are spatially distributed in the block. Therefore, it is not necessarily a feature point 8.

[0056] like Figure 6As shown, in order to select the next path element 10.2, starting from the path element 10.1, a new image portion or 3D block is defined as the new starting point, with the same or different block size as the initial block, for example, depending on the feature points occurring in this section. The selection of the path element 10.2 is performed in the same way as the path element 10.1. In this way, the path element 10 is determined along the preferred direction R from the feature points. According to a preferred embodiment of the method according to the invention, the preferred direction R is determined by the density and position of the feature points and can be fixed or dynamically adjusted in the method.

[0057] Figure 7 FIG. 2 shows a situation where the determined path elements 10 are superimposed on the camera image 6 together with the feature points 8 according to a preferred embodiment of the method of the present invention. Figure 7 , the path element 10 extends along the preferred direction R at the lower edge of the trailer 5. According to the structure of the trailer 5 and the corresponding characteristic points or / and the different distribution of the path elements, the path element 10 can be as follows Figure 8 It extends as shown, for example, not along the lower edge of the trailer 5. In both cases, the position of the trailer end or the trailer end edge can be determined according to a preferred embodiment of the method according to the invention, as will be explained further below.

[0058] Fig. 9 A symbolic representation of the movement of the path elements 10.1 to 10.11 or characteristic points 8 determined in the camera image 6 is shown to better illustrate the determination of the path or trailer end. Fig. 9 The path elements 10.1 to 10.11 and the feature point 8 are projected onto Fig.10 's coordinate system. Fig.10 The coordinate axes y, x (out of plane), and dy are shown.

[0059] According to a preferred embodiment of the method of the present invention, the path 11 is generated based on the straight lines in the path elements. The path 11 can also be generated as a polynomial of order n, wherein a plurality of different paths with polynomials of different orders can also be generated. Before the path 11 is generated, the path elements 10 can be at least partially weighted according to the position or sequence (order). Metrics with path element weights can be used, wherein the coordinates of the path elements and / or feature point sets they represent can be used, such as how many feature points, a scattering of coordinate values, a scattering of "score values", i.e., converting feature points into attributes of feature points.

[0060] like Fig.10 As shown, all path elements 10.1 to 10.9 are located within a predetermined distance from the path 11. Fig. 9 As shown, the determined path elements 10.10 and 10.11 do not belong to the trailer 5, but to the roadside, which are located outside the predetermined distance and are therefore not on the path 11. Fig. 9 and Fig.10 As shown, during the turn, path elements 10 . 10 and 10 . 11 move faster than path elements 10 . 1 to 10 . 9 and can therefore be identified as not belonging to path 11 .

[0061] According to a preferred embodiment of the method of the invention, in order to determine the position of the trailer end, the last path element 10.9 which has been considered in the path 11 and is therefore sufficiently determined to belong to the trailer is used as the starting point, from which the spatial proximity of further feature points 8 to the path 11 is analyzed. For example, in the previous step, a feature point may not be identified as belonging to the path 11 due to too coarse a threshold setting, or Figure 6 The image parts or blocks used to determine the path elements are already in different directions. Thus, the method according to the invention is further refined, wherein the coordinates of the last found feature point within a predetermined distance from the path 11 describe the end of the trailer. Alternatively, the end of the trailer can also be calculated from several last found feature points. Fig. 9 and Fig.10 In the example, this is denoted by EP or a. In other words, the trailer end, seen in the direction away from the tractor, is determined as the position on the path at which the image parameters used for selecting the feature point change permanently in this direction for the first time (e.g. not after a short stop along the path). The initial point IP can be located on the path and / or at least one path element and / or at least one feature point at a given distance from the path. Calculated variables (calculated quantities) of several path elements and / or feature points can be used, such as mean values, centroids, least squares deviations, etc.

[0062] Fig.11 The projection of a vertical line 12 at the coordinates a of the last found feature point in the camera image 6 is shown to indicate the end of the trailer determined by the method according to the invention.

[0063] According to a preferred embodiment of the method of the invention, the position of the trailer end determined in the above-described manner is further used to track (update) the trailer end in a monitor image displayed to the vehicle driver during a vehicle turn so that it appears in a preferred position in the monitor image, for example in the center or offset to the left or right by plus or minus 0% to 25% from the center, such as Fig.12 As shown, in order to reliably monitor the traffic situation behind.

[0064] According to a preferred embodiment of the method according to the invention, there is also a verification of the trailer end determined in the above-described manner. To this end, a block grid 13 is generated on a portion of the camera image 6, such as Fig.13 According to a preferred embodiment of the method of the present invention, the block grid 13 has a fixed shape and a fixed number of rows and columns, forming a plurality of cells 14. Fig.13In the block grid 13 in FIG. 1 , the number of columns is different from the number of rows. In principle, grid layouts other than rectangular cells can also be used.

[0065] According to a preferred embodiment of the method of the present invention, the distribution of the plurality of feature points 8 determined in the above manner on the plurality of cells 14 is determined. For example, Fig.14 The cell 14 shown comprises two characteristic points 8. The distribution of the characteristic points on the cells of the generated block grid 13 is then compared with the distribution of the characteristic points of a plurality of corresponding stored block grids (not shown). The stored block grids each define a different distribution of characteristic points, as well as the position and the flexion angle of the actual trailer end. According to a preferred embodiment of the method according to the invention, the corresponding stored block grid defining the actual trailer end is selected based on the flexion angle and / or the position of the trailer end or information about the relative positional relationship between the tractor unit and the trailer, and the correlation of the characteristic point distributions in the generated block grid and the stored block grids. By comparing this actual trailer end with a previously determined (input) trailer end, a verification can be performed and, if necessary, also a rejection or correction can be performed.

[0066] According to a preferred embodiment of the method of the present invention, the distribution of feature points and corresponding trailer ends in the storage block grid can be pre-stored or generated and stored in real time at fixed defined or dynamically selected time intervals during the execution of the method.

[0067] By using a stored block grid in which, in addition to the distribution of feature points on the cells, information about the trailer end is stored, the trailer end can be determined by comparing the generated block grid with the stored block grid without the need for verification of the trailer end and can be used for center tracking in the camera image as described above.

[0068] Fig.15 The tracked (updated) position of the end of the trailer in the monitor image displayed in the camera image 6 is shown. The preferred position is, for example, the center of the monitor image, or shifted 0% to 25% to the left or right from the center. The preferred position can also be offset 0% to 25% from the left edge or right edge of the monitor image.

[0069] It is expressly emphasized that all features disclosed in the description and / or claims are to be understood as separate and independent from each other, both for the purpose of original disclosure and for the purpose of limiting the claimed invention, regardless of the combination of features in the embodiments and / or claims. It is expressly pointed out that any range specification or specification of a group of units includes any intermediate values ​​or subgroups of units, both for the purpose of original disclosure and for the purpose of limiting the claimed invention, in particular also including the limits of the range indication.

[0070] Reference designation list 0 Starting point 1a, 1b Display device 2a, 2b Image capture device 3a, 3b Processing device 4 Tractor or tractor unit 5. Trailer 6 Camera images 7 Rear traffic 8 Feature points 9 pixel field 10 Path Elements 11 Path 12 Line indicating the end of the trailer 13 grids 14 cells R Predetermined direction IP Initialization Point VP Preferred Position EP Endpoint

Claims

1. A method for determining the position of the end of a trailer in a camera image (6) of a trailer (5) extending rearward from a tractor unit (4) of a vehicle and rotatably connected to the tractor unit (4) by processing camera images (6) captured continuously over time, comprising the following steps: capturing the camera image (6) continuously in time by at least one image sensor of at least one image capture device (2); Determining a plurality of feature points (8) in each camera image (6) used in the method based on at least one image parameter, wherein the feature point (8) corresponds to at least one pixel in the camera image (6); Determining at least one path element (10) along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy using a plurality of feature points (8), the determination being based on positional offsets in the respective coordinates x and / or y and / or dx and / or dy between the plurality of feature points (8) in the at least one camera image (6) and a corresponding plurality of feature points (8) that are temporally consecutive in the at least one camera image (6); generating at least one path (11) along the predetermined direction based on the path elements (10), the path (11) describing the position of the trailer (5); and A position on the path (11) is determined as the end of the trailer.

2. The method according to claim 1, wherein: The predetermined direction is determined by the density and position of the feature points (8).

3. The method according to claim 1, further comprising the steps of: Weighting the feature points (8); and Path elements (10) are determined from the weighted feature points (8).

4. The method according to claim 1, further comprising the steps of: weighting the path elements (10); and The at least one path (11) is generated based on the weighted path elements (10).

5. The method according to claim 1, further comprising the steps of: Information about the relative positional relationship between a trailer (5) and a traction unit (4) of a vehicle is obtained, wherein the determination of path elements, the weighting of feature points (8) and / or the weighting of path elements (10) are based on the obtained information about the relative positional relationship between the trailer (5) and the traction unit (4) of the vehicle.

6. The method according to claim 1, wherein: The at least one path (11) is generated as a mathematical function specified by the path element (10) or the weighted path element (10).

7. The method according to claim 1, wherein: The at least one image parameter is at least one of a brightness value, a color value, a grayscale, a contrast value of a pixel and / or a pixel cluster in the camera image (6) and / or a quantity calculated based on these values ​​and / or their gradients.

8. The method according to claim 1, further comprising Setting an initialization point (IP) as a starting point, the initialization point being located on the generated path (11), or being at least one path element (10) in at least one path (11), or being at least one further feature point (8) not exceeding a predetermined distance from at least one path (11), or being a calculated amount of the path element (10) or the further feature point (8); and Searching for at least one more feature point (8) along the path (11) from the starting point, wherein the distance between the feature point and at least one path (11) does not exceed a predetermined distance; and The coordinates of at least one further identified characteristic point (8) and / or a calculated value of at least one characteristic point (8) are set as the position (EP) of the trailer end.

9. The method according to claim 1, further comprising the steps of: obtaining at least one vehicle characteristic; and The determined position of the end of the trailer is revised based on the acquired at least one vehicle characteristic.

10. The method of claim 1 further comprising the step of verifying that the trailer end has been located, wherein the verifying step comprises the steps of: Generating a block grid (13) on at least a portion of the camera image (6), wherein the block grid (13) defines a plurality of cells (14), and determining a distribution of a plurality of feature points (8) on the plurality of cells (14) of the block grid (13) as a portion of the generated block grid (13); comparing the distribution of feature points (8) in the generated block grid (13) with the distribution of feature points of a plurality of stored corresponding block grids, wherein the stored block grids each define a different distribution of feature points and an actual trailer end; Selecting one of the storage block grids based on a correlation between the distribution of the feature points (8) in the generated block grid (13) and the distribution of the feature points in the storage block grids; The determined position of the trailer end (12) is verified by comparing the actual position of the trailer end defined by the selected block grid with the determined position of the trailer end (12), and / or further includes a correction of the determined position of the trailer end (12) based on the verified, and / or the distribution of characteristic points and corresponding trailer ends in the stored block grid is pre-stored or generated and stored in real time at fixed defined or dynamically adjustable time intervals.

11. The method according to claim 10, wherein: Based on information about the relative positional relationship between the trailer (5) and the traction unit (4), the distribution of feature points in one of the stored block grids is different from the distribution of feature points in the other stored block grid.

12. The method according to claim 1, comprising: Based on the position of the trailer (5) relative to an image sensor that acquires the camera images (6), a determined position of the trailer end is tracked in temporally consecutive camera images (6) so that the trailer end appears at a preferred position in a monitor image of at least one display device (1) based on the camera system.

13. A camera-based vehicle system having a tractor unit (4) and a trailer (5) extending rearward and rotatably connected to the tractor unit (4), comprising: at least one image capture device (2) arranged on the tractor unit (4) and having at least one image sensor for capturing a time-continuous camera image (6) of the area of ​​the trailer (5); and At least one processing device (3) is configured to perform a method according to any one of claims 1 to 12.

14. The camera-based system according to claim 13, further comprising at least one display device (1) for displaying a camera image (6) captured by at least the image capture device (2) and tracking the determined position of the trailer end in the camera image (6), the camera image (6) being continuous in time and depending on the position of the trailer (5) relative to the image sensor acquiring the camera image (6), so that the trailer end appears at a preferred position in the monitor image of the display device (1).

15. The camera based system of claim 13, approved under UNECE R46.

Images