Camera-based system and method for determining position of trailer

Abstract

To provide a method and a camera-based system for detecting the position of a trailer end with sufficient accuracy during backward maneuvering operations and at low speeds for tracking in a monitor image, irrespective of a trailer type and information acquired by sensors.SOLUTION: A method includes: capturing camera images successively in time by an image sensor; determining a plurality of feature points in each of the camera images on the basis of at least one image parameter, wherein one feature point corresponds to at least one pixel in the camera image; selecting relevant feature points from the plurality of feature points along a preferred direction defined by the density and position of the feature points as path elements on the basis of a positional displacement (dx, dy) between the plurality of feature points in one camera image and a corresponding plurality of feature points in a camera image successive in time; generating at least one path describing a trailer along a predetermined direction on the basis of the path elements; and determining a position on the path at a trailer end.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to a camera-based system (viewing system or mirror replacement system) and method for determining the position of an end (edge) of a trailer that extends rearwardly from the vehicle and is rotatable relative to the vehicle, the camera-based system and method being suitable for commercial vehicles having a semi-trailer, trailer or other unit that extends rearwardly and is rotatable relative to the cab or tractor unit. [Background technology]

[0002] The driver of a vehicle equipped with a tractor unit or towing unit having a trailer extending behind it and rotatable relative to the unit typically views the traffic behind them through side mirrors mounted on the sides of the towing unit. Such side mirrors are increasingly being replaced by camera systems, such as camera monitoring systems or mirror replacement systems mounted on the sides or rear of the vehicle, respectively. Such side mirrors replace or supplement the traditional mirror systems provided for motor vehicles, such as the exterior (main) and rear-view mirrors of automobiles and the wide-angle and front mirrors of commercial vehicles.

[0003] In the above-described system, for example, the corresponding field of view that would conventionally be visible through a mirror is always displayed in real time to the driver of the vehicle, for example on an in-vehicle monitor or other display unit, so that the driver of the vehicle can always view the corresponding field of view through the above-described system and not directly or through an attached mirror.

[0004] A trailer in the sense of the present teaching includes a trailer that is connected to a vehicle (tractor or towing unit) by a trailer coupling, for example a semi-trailer (so-called trailer), that is arranged in the rear lowered area of ​​the tractor unit and that is connected so as to be pivotable about at least one vertical axis. Alternatively, such a trailer may be a trailer attached to a passenger car, which in this case corresponds to the tractor unit. In general, a trailer is a rear extension of a vehicle that is located rearward with respect to the cab of the vehicle, that is laterally movable (pivotable) with respect to the cab and that pivots when cornering about an axis perpendicular to the cab. Articulated trucks and articulated trains are also trailers in the sense of the present invention.

[0005] However, when using a mirror replacement system as described above, the greater the buckling angle (kink angle or bending angle) between the towing unit and the trailer when the vehicle equipped with the towing unit and trailer is cornering, the more the rear traffic is hidden in the image of the mirror replacement system for field of view class II of the ECE R46 regulation. Furthermore, the rear of the trailer may be lost (disappeared) near the rear axle, causing the trailer end to disappear from the monitor image displayed to the driver.

[0006] To solve this problem, there are already some methods for tracking the trailer edge and for shifting the image part displayed on the monitor so that the trailer edge is displayed as centrally as possible in the monitor image. According to EP 16 198 485, this tracking of the trailer edge is currently performed, for example, by vehicle sensors which acquire information about the rotational movement of the wheels. This information is analyzed by a control unit in order to determine the rear area of ​​the rear extension and to track it appropriately based on the acquired information about the rotational movement of the wheels. However, such tracking of the trailer edge can only be performed when driving forward at high speeds, since these sensors can only provide reliable signals from a certain speed, for example from about 2 km / h. These sensors cannot be used when driving backwards or at low speeds, since they cannot detect the trailer edge with sufficient accuracy for tracking (updating) it in the monitor image. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention aims to overcome the above-mentioned problems and to detect the position of the trailer ends with sufficient accuracy, for example during reversing or low speed driving operations, in order to track (update) them in the monitor image independently of the type of trailer and the information acquired by the installed sensors. [Means for solving the problem]

[0008] The above mentioned object is achieved by a method having the features of claim 1 and by a system having the features of claim 15. Advantageous further developments of the invention are set out in the dependent claims.

[0009] In the method of the invention for determining the position of the trailer end or trailer edge of a trailer extending backward from the towing unit of a vehicle, camera images taken successively in time by an image sensor of a camera-based system are processed. The camera images used for processing do not have to be taken successively, for example only the second or third camera image can be used for further processing, reducing the amount of image data to be processed. Before being processed, the captured camera images can be suitably processed, for example with respect to resolution, contrast and / or color information. The method of the invention can use raw data of the captured camera images or partially processed camera images.

[0010] When further processing successive camera images, the method according to the invention determines in each camera image so-called feature points based on one or more image parameters, at least calculated quantities of the one or more image parameters and / or their gradients, where one feature point corresponds to at least one pixel in the camera image. A pixel in a camera image can be determined as a feature point, for example, if the value of an image parameter of this pixel differs from the value of a corresponding image parameter of one or more neighboring pixels by a preset value.

[0011] For example, adjacent pixels forming a pixel cluster may be compared to adjacent or nearby pixel clusters with respect to at least one image parameter to determine feature points. The pixels or pixel clusters do not need to be directly adjacent, but only need to be spatially close.

[0012] The method according to the invention then selects a number of path elements from the determined feature points based on a positional displacement (misregistration) between a number of feature points in a camera image and a number of corresponding feature points in a time-subsequent camera image. The path elements are determined, for example, along a predefined direction running along the trailer in the direction of the trailer end, preferably by the density and / or the position of the feature points. The predefined direction can be fixed or, preferably, dynamically adjusted during the process. The dynamic adjustment can be made on the basis of already defined path elements. The method according to the invention then generates, on the basis of these path elements, at least one so-called path showing the course of the trailer and determines a position on this at least one path as the position of the trailer edge. The determined path elements or feature points have associated coordinates x, y, dx, dy, where dx and dy indicate a misregistration or a displacement in the x and y directions.

[0013] According to an advantageous embodiment of the method according to the invention, before the path element is selected, a weighting of the feature points is performed. Such weighting can be based, for example, on the contrast difference between a pixel or pixel cluster and an adjacent pixel or pixel cluster or a neighboring pixel or pixel cluster. One or more known image parameters are the basis for the weighting. Furthermore, the positional deviation (dx, dy) between a feature point in a camera image and a corresponding feature point in a subsequent camera image can also be used as an alternative or additional weighting criterion.

[0014] According to a further advantageous embodiment of the method according to the invention, the above-mentioned selection of the path elements, the weighting of the characteristic points and / or the weighting of the path elements can be based on the kink angle and / or on information about the relative position between the towing unit and the trailer and / or on the speed of the vehicle. Such kink angle and / or the relative position can be determined independently of the acquired camera images, for example by means of a kink angle sensor installed on the vehicle, or can be estimated by means of data received from a steering angle sensor, or by image analysis. Other signals of the driving situation acquired by sensors installed on the vehicle can additionally or alternatively be taken into account.

[0015] According to a further advantageous embodiment of the method according to the invention, at least one path is a function defined by the path elements and / or weighted path elements, preferably a polynomial of degree n, e.g. a straight line whose path elements are on average approximately equally spaced.

[0016] According to an advantageous embodiment of the method, the image parameters are for example luminance values, color values, grey tones or at least one of these gradients, and before determining the feature points in the camera image, this camera image is left unprocessed or suitably processed in such a way that contrast differences, luminance differences etc. become more clearly visible. Other image parameters not explicitly mentioned here are also conceivable.

[0017] According to a further advantageous embodiment of the method according to the invention, the last found path element can be used as an initialization point and further feature points can be selected along the path that are within a predefined distance from the path. The last found feature points thus determined can be projected onto the path to indicate the position of the trailer edge. In this way, the actual position of the trailer end can be further approximated. Information regarding the actually determined position of the trailer end can be output, for example, by a CAN and / or Ethernet system and / or exchanged between the software modules.

[0018] According to a further advantageous embodiment of the method according to the invention, the position of the trailer edge thus discovered can be tracked (updated) in temporally consecutive camera images such that the trailer end always appears approximately in the center of each successive camera image. This means that even if the buckling angle of the trailer relative to the tractor unit assumed is constant, the driver of the vehicle can more reliably recognize the traffic behind.

[0019] According to a further advantageous embodiment of the method according to the invention, further vehicle features can be considered and used to correct the determined position of the trailer edge. Such vehicle features include, for example, geometric features such as length, width, height, etc., and can also be stored in advance and always for a specific type of trailer. Other vehicle characteristics such as the position of the tail lamp can also be used for correction. This information can be obtained from other processes, for example, via a CAN or an Ethernet system, or provided by other software modules.

[0020] According to a further advantageous embodiment of the method according to the invention, the determined position of the trailer edge is verified by generating a block grid with a number of rows and columns, which define cells in at least a part of the camera image. The block grid is preferably generated on the image in which the position of the trailer edge is determined. The generation of the block grid distributes the determined feature points in a number of cells of the block grid, which means that each cell contains a number of feature points. According to the invention, the distribution of the feature points in the generated block grid is compared with the distribution of the feature points of a corresponding number of stored block grids. The stored block grids define different distributions of the feature points together with the actual trailer edge. From the stored number of block grids, the block grid which is most correlated with the generated block grid is selected for verifying the determined position of the trailer edge, and a plausibility check can be performed by comparing the determined position of the trailer edge with the actual position of the trailer edge defined in the selected block grid. If necessary, the determined position of the trailer edge can be corrected on the basis of this plausibility check if it deviates too much from the actual position, and the determination of the position of the trailer edge can also be performed again.

[0021] According to a further advantageous embodiment of the method according to the invention, the distribution of feature points together with the corresponding position of the trailer end can be stored beforehand in a block grid or generated and stored in real time at fixed or dynamically adjustable time intervals, and the determined distribution is then compared with such predefined distribution and the position of the trailer end is verified as described above.

[0022] According to a further advantageous embodiment of the method according to the invention, the different distribution of the feature points in the block grids stored together with the corresponding trailer end depends on the buckling angle between the trailer and the towing unit. Information on the buckling angle can for example be stored together with each block grid. That is, if for example the current buckling angle or the relative position of the trailer and the towing unit is known, the stored block grid for this buckling angle can be found directly and a verification of the determined trailer end position can be carried out. Other information indicating the position of the trailer relative to the towing unit can also be used.

[0023] The camera-based system according to the invention comprises at least one camera with an image sensor for taking (acquiring) time-sequential camera images of the side area of ​​the trailer, a monitor for displaying the images taken by the camera, and at least one processor configured to execute the above-mentioned method according to the invention. According to an advantageous further development, the camera-based system according to the invention is a mirror replacement system approved according to the UN ECE R46 regulation, preferably a mirror replacement system for commercial vehicles.

[0024] It may be provided in any of the following ways: [Aspect 1] 1. A method for determining the position of a trailer end in a camera image (6) of a trailer (5) extending rearward from a towing unit (4) of a vehicle and rotatable relative to said towing unit (4) by processing camera images (6) taken in succession in time, comprising: - capturing said camera images (6) successive in time by at least one image sensor of at least one image capture means (2); - determining a number of feature points (8) in each of the camera images (6) used in the method based on at least one image parameter, one feature point (8) corresponding to at least one pixel in the camera image (6); Based on the determined positional displacements in the corresponding coordinates x and / or y and / or dx and / or dy between the plurality of feature points (8) in at least one of the camera images (6) and the corresponding plurality of feature points (8) in at least one temporally consecutive camera image (6), along a predetermined direction in the relevant coordinates x and / or y and / or dx and / or dy, by using the plurality of feature points (8), determining at least one path element (10); Generating at least one path (11) along the predetermined direction based on the path element (10), the path (11) indicating the position of the trailer (5); Determining a position on the path (11) as the trailer end, a method characterized by comprising. [Aspect 2] In the method according to Aspect 1, the predetermined direction is determined by the density and position of the feature points (8). [Aspect 3] In the method according to Aspect 1, weighting the feature points (8); determining the path element (10) from the weighted feature points (8), a method comprising. [Aspect 4] In the method according to any one of Aspects 1 to 3, weighting the path element (10); generating the at least one path (11) based on the weighted path element (10), a method further comprising. [Aspect 5] In the method according to any one of Aspects 1 to 4, acquiring information regarding the relative positional relationship between the trailer (5) and the towing unit (4) of the vehicle, The method further comprises a step in which the determination of the path elements, the weighting of the feature points (8) and / or the weighting (10) of the path elements is performed based on the obtained information regarding the relative positional relationship between the trailer (5) and the towing unit (4) of the vehicle. [Aspect 6] 20. The method according to any one of the preceding aspects, The method, wherein the at least one path (11) is generated as a function determined by the path elements (10) or weighted path elements (10). [Aspect 7] 20. The method according to any one of the preceding aspects, wherein the at least one image parameter is at least one of a luminance value, a color value, a graytone, a contrast value, and / or a calculation from one of these values ​​and / or a gradient thereof for pixels and / or pixel clusters in a camera image (6). [Aspect 8] 20. The method according to any one of the preceding aspects, setting as a starting point an initialization point (IP) which is located on said generated path (11) or which is at least one path element (10) of said at least one path (11) or which is at least said further feature point (8) not exceeding a predetermined distance from said at least one path (11) or not exceeding a calculated amount of said path element (10) or further feature point (8); - searching for at least a further feature point (8) starting from said starting point along said path (11) and not exceeding a predetermined distance from said at least one path (11); and setting the coordinates of said at least one further identified feature point (8) and / or a calculated quantity of said at least one feature point (8) as said position (EP) of said trailer end. [Aspect 9] In the method according to any one of aspects 1 to 8, obtaining at least one vehicle characteristic; revising the determined position of the trailer end based on the acquired at least one vehicle characteristic. [Aspect 10] 20. The method according to any one of the preceding aspects, and verifying the determined position of the trailer end, wherein the verifying step comprises: generating a block grid (13) defining a plurality of cells in at least a portion of the camera image (6), determining a distribution of the plurality of feature points (8) over the plurality of cells (14) of the block grid (13) as part of the generated block grid (13); comparing the distribution of the feature points (8) within the generated block grid (13) with a number of corresponding stored block grid distributions of feature points, each of the stored block grids defining a different distribution of feature points together with an actual trailer end; selecting one of the stored block grids based on a correlation between a distribution of feature points (8) in the generated block grid (13) and the distribution of the feature points in the stored block grids; and verifying the determined position of the trailer end (12) by comparing the actual position of the trailer end defined by the selected block grid with the determined position of the trailer end (12). [Aspect 11] 11. The method of claim 10, further comprising the step of modifying the determined position of the trailer end (12) based on the verification. [Aspect 12] In a method according to aspect 10 or 11, the distribution of the feature points and the corresponding trailer ends within the stored block grid are pre-stored or are generated and stored in real time at fixed or dynamically adjustable time intervals. [Aspect 13] In the method according to any one of aspects 10 to 12, wherein the distribution of feature points in one of the stored block grids differs from the distribution of feature points in another of the stored block grids according to information about a relative positional relationship between a trailer (5) and a towing unit (4). [Aspect 14] 20. The method according to any one of the preceding aspects, tracking the determined position of the trailer end in successive camera images (6) in time depending on the position of the trailer (5) relative to the image sensor acquiring the camera images (6) so that the trailer end appears in a preferred position in a monitor image of at least one display means (1) of a camera-based system. [Aspect 15] 1. A camera-based system for a vehicle having a towing unit (5) and a trailer (4) extending rearward and rotatable relative to the towing unit (5), comprising: at least one image acquisition means (2) provided on the towing unit (5) and comprising at least one image sensor for acquiring camera images (6) of the area of ​​the trailer (5) successive in time; At least one processing means configured to execute a method according to any one of aspects 1 to 14. [Aspect 16] 16. The camera-based system of claim 15, further comprising: Further comprising at least one display means; The at least one display means displays a camera image (6) captured by the at least one image capturing means (2) such that the trailer end appears at a preferred position within the monitor image of the display means (1), and tracks the determined position of the trailer end in the temporally consecutive camera images (6) according to the position of the trailer (5) relative to the image sensor that acquires the camera image (6). It is a camera-based system for this purpose. [Aspect 17] In the camera-based system according to Aspect 15 or 16, A camera-based system approved in accordance with UN ECE R46.

Brief Description of the Drawings

[0025] Hereinafter, the present invention will be described merely as an example with reference to the attached drawings in which the same reference numerals indicate the same or similar components.

[0026] [Figure 1] FIG. 1 is a schematic diagram of a camera-based system according to a preferred embodiment of the present invention.

[0027] [Figure 2a] FIG. 2a is a top view of a vehicle using the camera-based system of FIG. 1 in a first driving situation corresponding to straight-ahead forward driving.

[0028] [Figure 2b] FIG. 2b is a top view of a vehicle using the camera-based system of FIG. 1 in a second driving situation corresponding to cornering.

[0029] [Diagram 3] FIG. 3 shows a camera image captured by the camera-based system of FIG. 1 as a display image of a monitor for explaining a method of determining the position of the trailer end according to a preferred embodiment of the present invention.

[0030] [Figure 4]FIG. 4 shows the camera image according to FIG. 3 to illustrate the determination of feature points according to a preferred embodiment of the present invention.

[0031] [Diagram 5] FIG. 5 is a schematic three-dimensional view of a portion of the camera images of FIG. 3 to illustrate the misalignment of feature points between two successive camera images according to a preferred embodiment of the present invention.

[0032] [Figure 6] FIG. 6 shows a schematic three-dimensional view of FIG. 5 to illustrate the determination of path elements according to a preferred embodiment of the present invention.

[0033] [Figure 7] FIG. 7 shows the camera image of FIG. 3 with path elements superimposed along the bottom edge of the trailer in accordance with a preferred embodiment of the present invention.

[0034] [Figure 8] FIG. 8 shows the camera image of FIG. 3 with alternative path elements superimposed along the structure of the trailer in accordance with the preferred embodiment.

[0035] [Figure 9] FIG. 9 is a diagram in which the movements of feature points and route elements are symbolically represented in the camera image of FIG.

[0036] [Figure 10] FIG. 10 is a diagram illustrating the determination of trailer end position in accordance with a preferred embodiment of the present invention.

[0037] [Figure 11] FIG. 11 shows the trailer ends in the camera image of FIG. 9 as determined by a preferred embodiment of the present invention.

[0038] [Figure 12] FIG. 12 shows the tracked (updated) trailer end in the camera image of FIG. 3 according to a preferred embodiment of the present invention.

[0039] [Figure 13] FIG. 13 shows a block grid in accordance with a preferred embodiment of the present invention in the camera image of FIG.

[0040] [Figure 14] FIG. 14 illustrates feature points in the cells of the block grid of FIG. 13 according to a preferred embodiment of the present invention.

[0041] [Figure 15] FIG. 15 shows the tracked (updated) trailer end in the camera image of FIG. 3 according to a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] FIG. 1 shows a camera-based system according to a preferred embodiment of the invention, comprising display means 1a, 1b, image capture means 2a, 2b and processing means 3a, 3b.

[0043] The image acquisition means 2a, 2b are preferably arranged on opposite sides of the vehicle and connected to the display means 1a, 1b via the processing means 3a, 3b. The image acquisition means 2a, 2b are arranged on opposite sides of the vehicle, for example as shown in Fig. 2. The processing means 3a, 3b according to a preferred embodiment comprises buckling angle determination means and image data processing means (not described in more detail) and is in particular capable of executing the inventive method for determining the position of the trailer end and for tracking (updating) and correcting the trailer end on the display means 1a, 1b as described below.

[0044] For simplicity, only the display means 1a, the image capture means 2a and the processing means 3a will be described. However, the following description applies equally to the display means 1b, the image capture means 2b and the processing means 3b on the opposite side of the vehicle. Furthermore, according to a preferred embodiment, only one processing unit may be provided which assumes the functions of the processing means 3a and 3b.

[0045] As shown in Figures 2a and 2b, the vehicle comprises, for example, a towing vehicle 4 and a trailer 5 rotatably attached to the towing vehicle 4. The image capture means 2a has a recording area covering a viewing area or field of view corresponding to "Main Mirror (Large) Group II" according to the ECE-R46 regulation. The image capture means 2a may also cover other appropriately defined fields of view based on national regulations.

[0046] FIG. 2a shows a state in which there is no buckling angle between the towing vehicle 4 and the trailer 5 and the vehicle is moving straight ahead. When cornering, as shown in FIG. 2b, a buckling angle occurs between the towing vehicle 4 and the trailer 5, and as the buckling angle increases, the rear end of the trailer 5 shifts (moves) into the field of view of the image capturing means 2a. FIG. 3 shows a camera image 6 (monitor image) captured by the image capturing means 2a and displayed on a monitor, in which the larger the buckling angle, the more the end of the trailer 5 hides the rear traffic. The image capturing means 2a is configured to capture a series of images that are consecutive in time.

[0047] Figure 3 shows a camera image 6 taken by the camera-based system of Figure 1 as an image of a trailer 5 displayed on a monitor together with traffic 7 behind. Figure 4 serves to illustrate the determination of feature points 8 in this camera image 6.

[0048] According to a preferred embodiment of the method of the invention for determining a trailer end or trailer edge in a camera image 6, a number of feature points 8 are determined in a two-dimensional camera image 6 having coordinate axes x and y, as shown in FIG. 4. Each feature point 8 corresponds to one or more pixels in a pixel cluster. FIG. 4 shows a pixel field 9 in an enlarged view of a portion of the camera image 6. According to a preferred embodiment, pixels adjacent to, for example, pixels 1-5 are determined as feature points if they have a predefined contrast difference with at least some of the adjacent pixels. Other image parameters for determining whether a pixel is a feature point or not can be used alternatively or cumulatively.

[0049] According to a preferred embodiment of the method according to the invention, the feature points 8 determined in FIG. 4 are weighted to emphasize or better determine the contrast difference with adjacent or spatially proximate pixels. The weighting parameter is, for example, the speed at which the feature point changes relative to the tractor, i.e., if there is no change or only a slight change with respect to the position on the image sensor, it is assumed to be a point that moves with the vehicle and is therefore on the trailer, and thus is given a higher weight. For further processing according to the method of the invention, for example, only feature points above a predetermined weight are used, so it is not necessary to use all feature points, and the computing capacity can be saved.

[0050] According to a preferred embodiment of the method according to the invention, as will be described later with respect to FIGS. 5 and 6, the path element 10 is selected from the feature points 8 shown in FIG. 4.

[0051] According to a preferred embodiment of the method according to the invention, first, as shown in FIG. 4, a starting point 0 is defined in the camera image 6. The starting point 0 may be a feature point 8, but not necessarily so. The starting point 0 can be set, for example, based on the buckling angle and the relative position of the resulting trailer with respect to the image capturing means. As shown in FIG. 5, starting from the starting point 0, an image section is defined. In FIGS. 5 and 6, this image section is shown as a three-dimensional block of a constant or dynamically adjustable block size (bs) for better visibility. Within this three-dimensional block, there are a plurality of corresponding feature points of FIG. 4 distributed in the three-dimensional space around the starting point 0 as the center point of the block of this size (bs). Each of these feature points is represented by coordinates x(bs x ), coordinates y(bs y ), coordinates dx (not shown in FIG. 5), and coordinates dy(bs dy ). The coordinates dx and coordinates dy represent the displacement (shift) of the feature point between the camera images captured continuously in time.

[0052] From the "cloud of feature points" shown in Fig. 5, at least one path element 10.1 is determined. According to a preferred embodiment of the method of the invention, this path element corresponds to the center (centre of gravity) of the "cloud of feature points". The path element 10.1 can therefore be one of the feature points of the "cloud of feature points" or it can simply be the position in the block where the most feature points are spatially located. The path element 10.1 therefore does not necessarily have to be a feature point 8.

[0053] As shown in Fig. 6, to select the next path element 10.2, starting from the path element 10.1 as a new starting point, a new image section or 3D block is defined, for example with the same or another block size as the initial block, depending on the feature points present in this section. The selection of the path element 10.2 is performed similarly to the selection of the path element 10.1. In this way, the path element 10 is determined from the feature points along a preferred direction R. According to a preferred embodiment of the method according to the invention, the preferred direction R is determined by the density and the position of the feature points and may be a constant direction or may be dynamically adjusted while the method is performed.

[0054] Fig. 7 shows a view of a path element 10 determined by a preferred embodiment of the method according to the invention, together with the feature points 8, superimposed on the camera image 6. According to Fig. 7, the path element 10 is the lower edge of the trailer 5 and runs along the preferred direction R. Depending on the structure of the trailer 5 and on a corresponding alternative distribution of the feature points or / and the path elements, the path element 10 may also run, for example as shown in Fig. 8, i.e. not along the lower edge of the trailer 5. In any case, the position of the trailer end or the trailer edge can be determined by a preferred embodiment of the method according to the invention, as will be explained further below.

[0055] To better illustrate the determination of the path or trailer end, Fig. 9 symbolically shows the movement of the determined path elements 10.1-10.11 or feature points 8 in the camera image 6. For better understanding, the path elements 10.1-10.11 and feature points 8 of Fig. 9 are projected into the coordinate system of Fig. 10. Fig. 10 shows the coordinate axes y, x (out of plane) and dy.

[0056] According to a preferred embodiment of the method according to the invention, a path 11 is generated based on the path elements, which in this preferred embodiment are straight lines. The path 11 can also be generated as a polynomial of degree n, and also several different paths with different polynomial degrees can be generated. The path elements 10 are weighted at least in part based on their position or order before the path 11 is generated. A metric with the weight of the path elements can be used, but also characteristics of the coordinates of the path elements and / or of the set of feature points they represent, e.g. the number of feature points, the variance of the coordinate values ​​of the feature points, the variance of the "score values", i.e. the properties that transform feature points into feature points.

[0057] As shown in Figure 10, all path elements 10.1 to 10.9 are within a predefined distance from the path 11. The determined path elements 10.10 and 10.11 are outside the predefined distance and therefore not on the path 11, as they do not belong to the trailer 5 but to the curb, as shown in Figure 9. As shown in Figures 9 and 10, the path elements 10.10 and 10.11 can be identified as not belonging to the path 11, since they move faster than the path elements 10.1 to 10.9 when cornering.

[0058] According to a preferred embodiment of the method according to the invention, in order to determine the position of the trailer end, the last path element 10.9 taken into account for the determination of the path 11 and identified with sufficient certainty as belonging to the trailer is used as the initial point for the analysis of further feature points 8 with respect to their spatial proximity to the path 11. For example, in the previous steps, the threshold may have been set too roughly, so that the feature point was not identified as belonging to the path 11, or the orientation of the image section or block of FIG. 6 for determining the path element may have been different. In this way, the method according to the invention is further improved, and the trailer end is drawn at the coordinates of the last found feature point that is within a predefined distance from the path 11. Alternatively, the trailer end can be calculated from several last found feature points. In FIGS. 9 and 10, the trailer end is indicated by EP or a. That is to say, the position of the trailer end is the furthest from the towing unit when viewed in the direction away from the towing vehicle, and is determined as the position on the path at which the image parameters determined for selecting the feature point always change in this direction for the first time (i.e., not immediately after a small break along the path, for example). The initial point IP can be on the path and / or at least one path element and / or at least one feature point having a predefined distance from the path. Calculated variables (calculations) of multiple path elements and / or feature points can be used, for example, mean values, centroids, least square deviations, etc.

[0059] FIG. 11 shows the projection of a perpendicular line 12 in coordinate a of the last feature point found in the camera image 6 to indicate the trailer end determined by the method according to the invention.

[0060] According to a preferred embodiment of the method of the present invention, the position of the trailer end determined by the above-described method is further used to track (update) the trailer end in the monitor image displayed to the vehicle driver while the vehicle is cornering, so that the trailer end appears in a preferred position in the monitor image, for example at the center or 0% to 25% offset to the left or right of the center, as shown in Figure 12, in order to ensure monitoring of traffic, particularly to the rear.

[0061] According to a preferred embodiment of the method of the present invention, the trailer end determined by the above method is further verified. For this purpose, as shown in FIG. 13, a block grid 13 is generated in a part of the camera image 6. According to a preferred embodiment of the method of the present invention, the block grid 13 has a certain shape and a certain number of rows and columns forming a plurality of cells 14. In the block grid 13 of FIG. 13, the number of columns is different from the number of rows. In principle, grid layouts other than rectangular cells can also be used.

[0062] According to a preferred embodiment of the method of the present invention, the distribution of a plurality of feature points 8 determined by the above method is determined over a plurality of cells 14. For example, the cell 14 shown in FIG. 14 includes two feature points 8. Next, the distribution of the feature points on the cells of the generated block grid 13 is compared with the distribution of the feature points of a plurality of corresponding stored block grids (not shown). Each of the stored block grids defines a different distribution of feature points together with the position and buckling angle of the actual trailer end. According to a preferred embodiment of the method of the present invention, the corresponding stored block grid that defines the actual trailer end is selected based on the buckling angle and / or position of the trailer end, or information regarding the relative positional relationship between the towing unit and the trailer, and the correlation of the distribution of the feature points in the generated block grid and the stored block grid. This can be verified by comparing this actual trailer end with the previously determined (input) trailer end and rejected or corrected if necessary.

[0063] According to a preferred embodiment of the method of the present invention, the distribution of the feature points and the corresponding trailer end in the stored block grid can be stored in advance or generated and stored in real time while executing this method at regular time intervals or dynamically selected time intervals.

[0064] In addition to the distribution of feature points on the cell, by using a block grid in which information about the trailer end is stored, even when there is no trailer end to be verified in the comparison between the generated block grid and the stored block grid, the trailer end can be determined, and the trailer end can also be used for tracking the center in the camera image as described above.

[0065] FIG. 15 shows the tracked (updated) position of the trailer end in the camera image 6 at a preferred position in the displayed monitor image. This preferred position is, for example, the center of the monitor image, or a position shifted 0% to 25% to the left or right from the center. The preferred position may be offset 0% to 25% from the left end or the right end of the monitor image.

[0066] It is emphasized that all features disclosed in the specification and / or the claims are to be understood as being separate or independent from each other not only for the purpose of the original disclosure, but also for the purpose of limiting the claimed invention independent of the combinations of features in the embodiments and / or the claims. It is expressly stated that the specification of a range or a group of units includes any intermediate value or sub-group of units not only for the purpose of the original disclosure, but also for the purpose of limiting the claimed invention, particularly for the purpose of limiting the range.

Explanation of Reference Numerals

[0067] 0: Starting point 1a, 1b: Display means 2a, 2b: Image capturing means 3a, 3b: Processing means 4: Tractor or towing unit 5: Trailer 6: Camera image 7: Rear traffic 8: Feature point 9: Pixel field 10: Route element 11: Route 12: Line indicating the trailer end 13: Block grid 14: Cell R: Prescribed direction IP: Initialization point VP : Preferred Position EP: The End

Claims

1. 1. A method for determining the position of a trailer end in a camera image of a trailer extending rearwardly from a towing unit of a vehicle and rotatable relative to said towing unit by processing camera images taken in succession in time, comprising: It includes the following steps: In the imaging step, the camera images are captured successively in time by at least one image sensor of at least one image capturing means; determining a plurality of feature points in each of the camera images used in the method based on at least one image parameter, where one feature point corresponds to at least one pixel in the camera image; In the path element determination step, based on the determined positional displacements in corresponding coordinates x and / or y and / or dx and / or dy between the feature points in at least one of the camera images and corresponding feature points in at least one temporally consecutive camera image, at least one path element is determined by using the feature points along a predetermined direction in associated coordinates x and / or y and / or dx and / or dy; In the generating step, at least one route is generated along the predetermined direction based on the route elements, where the route indicates the position of the trailer; A method according to claim 1, wherein the step of determining a position determines a position on the route as the trailer end.

2. 10. The method of claim 1 , The method, wherein the predetermined direction is determined by a density and a position of the feature points.

3. 10. The method of claim 1 , It further includes the steps of: In the weighting step, the feature points are weighted; In the determining step, the route elements are determined from the weighted feature points.

4. 10. The method of claim 1 , It further includes the steps of: In the weighting step, weights are assigned to the route elements; A method comprising: generating the at least one route based on the weighted route elements in a generating step.

5. 10. The method of claim 1 , It further includes the steps of: In the obtaining step, information regarding a relative positional relationship between the trailer and the towing unit of the vehicle is obtained; A method according to claim 1, wherein the determination of the route elements, the weighting of the feature points and / or the weighting of the route elements are performed based on the obtained information regarding the relative positional relationship between the trailer and the towing unit of the vehicle.

6. 10. The method of claim 1 , The method, wherein the at least one path is generated as a function determined by the path elements or weighted path elements.

7. 10. The method of claim 1 , The method of claim 1, wherein the at least one image parameter is at least one of a brightness value, a color value, a graytone, a contrast value, and / or a calculated quantity from one of these values ​​and / or a gradient thereof at a pixel and / or a pixel cluster in a camera image.

8. 10. The method of claim 1 , It further includes the steps of: In the setting step, an initialization point (IP) is set as a starting point; said initialization point being located on said generated path or being at least one path element of said at least one path or being at least said further feature point not exceeding a predetermined distance from said at least one path or not exceeding a calculated amount of said path element or further feature point; a searching step starting from the starting point along the path and searching for at least a further feature point not more than a predetermined distance from the at least one path; Setting the coordinates of said at least one further identified feature point and / or a calculated quantity of said at least one feature point as said position (EP) of said trailer end.

9. 10. The method of claim 1 , It further includes the steps of: In the vehicle feature acquisition step, at least one vehicle feature is acquired; A method comprising: modifying the determined position of the trailer end based on the acquired at least one vehicle characteristic in a modifying step.

10. 10. The method of claim 1 , The method further includes a step of verifying the determined position of the trailer end, wherein the step of verifying includes the steps of: In the generating and determining step, a block grid defining a plurality of cells is generated in at least a portion of the camera image, and a distribution of the plurality of feature points on the plurality of cells of the block grid is determined as part of the generated block grid; a comparing step of comparing the distribution of feature points within the generated block grid with a plurality of corresponding stored distributions of feature points of block grids, each of the stored block grids defining a different distribution of feature points along with an actual trailer end; In the selection step, one of the stored block grids is selected based on a correlation between a distribution of feature points in the generated block grid and the distribution of feature points in the stored block grid; The verification step is a step that satisfies at least condition (a) among the following conditions (a) to (c): Condition (a) is that in the verification step, the determined position of the trailer end is verified by comparing the actual position of the trailer end defined by the selected block grid with the determined position of the trailer end; Condition (b) is that the verification step further includes a correction step, in which the determined position of the trailer end is corrected based on the verification; Condition (c) is that the distribution of feature points and the corresponding trailer ends within the stored block grid are pre-stored or are generated and stored in real time at fixed or dynamically adjustable time intervals.

11. 11. The method of claim 10, The method of claim 1, wherein the distribution of feature points in one of the stored block grids differs from the distribution of feature points in another of the stored block grids according to information regarding a relative positional relationship between a trailer and a towing unit.

12. 10. The method of claim 1 , A tracking step is included. A method in which the tracking step tracks the determined position of the trailer end in successive camera images in time depending on the position of the trailer relative to the image sensor that acquires the camera images so that the trailer end appears in a desired position in a monitor image of at least one display means of a camera-based system.

13. 1. A camera-based system for a vehicle having a towing unit and a trailer extending rearwardly and rotatable relative to the towing unit, comprising: at least one image acquisition means provided on the towing unit and comprising at least one image sensor for acquiring camera images of the area of ​​the trailer successive in time; At least one processing means arranged to carry out the method according to any one of claims 1 to 12.

14. 14. The camera-based system of claim 13, Further comprising at least one display means; A camera-based system, wherein the at least one display means is for displaying camera images captured by the at least one image capturing means so that the trailer end appears in a preferred position within the monitor image of the display means, and for tracking the determined position of the trailer end within successive camera images in time depending on the position of the trailer relative to the image sensor that acquires the camera images.

15. 14. The camera-based system of claim 13, Camera-based system approved according to UN ECE R46.

Images