Method and mobile device for determining a vehicle's position relative to a reference position

Abstract

A method for determining the position (2) of a vehicle (1) relative to a reference position (3), comprising: - capturing (S1) a reference image (10) using a camera (4) mounted on the vehicle (1), which depicts at least part of the vehicle's surroundings, wherein the vehicle (1) is located in the reference position (3), - capturing (S2) a comparison image (11) using the camera (4) mounted on the vehicle (1), - comparing (S3) the reference image (10) with the comparison image (11), whereby a change in position and / or size of at least one feature in the comparison image (11) relative to the reference image (10) is determined, and - determining (S4) a vehicle position (2) of the vehicle (1) relative to the reference position (3) based on the change in position and / or size, wherein - in the first step (S1) the reference image (10) is additionally filtered with a Sobel filter.and- in the second process step (S2) the comparison image (11) is additionally filtered with the Sobel filter, wherein- the filtered reference image (10) and the filtered comparison image (11) are stored, and- in the third process step (S3) the filtered images are compared, wherein- the reference image (10) and the comparison image (11) are captured by a mobile device (40), in particular a smartphone, which is arranged in the vehicle (1) such that the camera (4) images at least a part of the vehicle's (1) surroundings.

Description

Prior ArtThe present invention relates to a method and a device for determining a vehicle position of a vehicle with respect to a reference position.Semi-automatic parking systems have already been in series production for a few years. In such systems, the driver drives along parking spaces with a vehicle. The vehicle recognizes the parking spaces by means of laterally installed ultrasonic sensors. If the driver applies the reverse gear, the parking system takes over control of the steering wheel and controls the vehicle into the parking space. However, the driver must himself put gas and actuate the brake. Parking systems are known as a further expansion stage in which the driver does not have to put gas or brake. This is also taken over by the parking system, for example by direct access to the longitudinal guidance actuator system of the vehicle, which includes, for example, brake and engine control.In development, there are also already parking systems which are only taught where a specific parking space is located. The specific parking space is, for example, a home garage. Following the teaching process, the driver only needs to place the vehicle in the vicinity of the previously taught parking position or target position. The vehicle is intended to independently identify the current position relative to the target position and to move automatically from the current position to the learned parking position or target position. The use of the environment sensor system of the vehicle is important here, since the particular advantage of the system is that the driver does not have to take care of an imprecision of his stopping position before the beginning of the automatic parking. Such automatic parking is also referred to as home zone parking or garage parking.Home zone parking comprises two phases:In a first phase, the system is trained. The desired path from a starting position to a target position is defined by the driver in that the driver travels along the path with the vehicle and in the process the system stores the path and the objects in the vehicle environment on the basis of a suitable environment sensor system.In a second phase, the system is used. The driver drives the vehicle near the "trained" starting position with a similar orientation. The driver activates the system. The system locates itself and thereby determines an offset of the vehicle in position and orientation relative to the stored starting position. The system then assumes the automatic guidance of the vehicle from the current position to the predefined target position. During guidance, a calibration must also take place in the system in order to determine a current position with respect to the stored path.Qualitatively suitable localization of the vehicle position by means of a surroundings sensor system (cameras, ultrasonic sensors, laser scanners, etc.) is the decisive challenge here. Previously, sensor systems have been provided for this purpose, which have a range of more than 20 m and a very good angular and distance resolution in order to enable a detection of surrounding objects. Such sensor systems are, however, technically complicated and thus expensive.The document DE 10 2008 002 598 A1 discloses a device for semi-autonomous assistance of the steering movement of a vehicle.Document JP 2013-124957 A discloses an on-board device having navigation function mounted on a vehicle.Disclosure of the InventionThe method according to the invention for determining a vehicle position of a vehicle with respect to a reference position comprises a detection of a reference image by means of a camera arranged on the vehicle, which images at least part of a surrounding area of the vehicle, wherein the vehicle is located in the reference position, a detection of a comparison image by means of the camera arranged on the vehicle, a comparison of the reference image with the comparison image, wherein a change in position and / or a change in size of at least one feature in the comparison image with respect to the reference image is determined, and a determination of a vehicle position of the vehicle with respect to the reference position on the basis of the change in position and / or the change in size. According to the invention, a filter function of the reference image is performed with a Sobel filter and a filter function of the comparison image is performed with the Sobel filter, wherein the qefiltered reference image and the filtered comparison image are stored and the qefiltered images are compared.According to the invention, the acquisition of the reference image and the comparison image is carried out by a mobile device, in particular a smartphone, which is arranged in the vehicle in such a way that the camera images at least part of a surrounding area of the vehicle. This also allows the vehicle position of a vehicle to be determined for those vehicles which do not have a fixedly installed camera, as a result of which vehicle functionality is extended via the mobile device.The mobile device according to the invention for determining a vehicle position of a vehicle with respect to a reference position comprises a camera which is configured to capture a reference image which images at least part of a surrounding area of the vehicle when the vehicle is located in the reference position, and to capture a comparison image. Furthermore, the mobile device according to the invention comprises an evaluation unit which is configured to additionally perform filtering of the reference image using a Sobel filter and additionally perform filtering of the comparison image using the Sobel filter.In particular, the feature whose change in position and / or change in size in the comparison image with respect to the reference image is determined is either first extracted from the reference image and then sought in the comparison image or else first extracted from the comparison image and then sought in the reference image. A method according to the invention and a device according to the invention are advantageous since it does not require a highly accurate environment sensor system, but rather favorable monocamera can be used to determine a vehicle position. Real and difficult-to-acquire distance information is not needed. In addition, many current vehicles are already equipped with a camera arranged on the vehicle, so that already existing hardware, such as a rear view camera, can be used. The requirements imposed on the camera are significantly lower than in the case of cameras which detect distances to objects with the aid of the StructureForMo method.With the aid of a monocamera which is positioned at any desired position on or in the vehicle (front, rear, lateral), it is thus possible to carry out a local localization which is independent of the desired direction of travel of the vehicle and also without determining the exact distance from objects in the environment of the vehicle. Thus, for example, a rear view camera can be used to determine the position of the vehicle during a home zone parking. This can also take place during a journey of the vehicle.The dependent claims show preferred developments of the invention.It is advantageous if the change in position is a change in the position of a feature in the horizontal direction. Thus, both a lateral offset of the vehicle with respect to the recording direction of the camera with respect to the reference position and a rotation of the vehicle with respect to the reference position can be determined in a simple manner.It is likewise advantageous if the change in position is a change in the position of a feature in the vertical direction. This makes it possible to easily determine the offset of the vehicle with respect to the reference position in the recording direction of the camera.In particular, the mobile device is arranged in a holder provided for this purpose. Thus, after removal, the mobile device can be arranged in the vehicle again at a position which corresponds approximately to the position before removal. The ascertained vehicle position is not adulterated by a change in position of the camera on the vehicle, in particular if the captured images comprise parts of the vehicle. In this case, use of commercially available smartphone or PDA holders is possible, as a result of which it is possible to resort to cost-effective elements which are often already available.It is advantageous if a calibration is carried out with respect to the vertical and horizontal orientation of the camera. As a result, a change in the position of the camera with respect to its vertical and horizontal orientation can be compensated, which is advantageous in particular when the camera is surrounded by a mobile device, since here the camera is often repositioned, which can bring about a change in the position of the camera. Thus, no fixedly adjusted holder is necessary.It is likewise advantageous if the comparison of the reference image with the comparison image takes place on the basis of selected representative features in the comparison image and the reference image. In this case, in particular, a filtering of the reference image and / or of the comparison image can take place before the comparison in order to determine the representative features. This is advantageous since it is thus not necessary to compare the entire images. The memory requirement and the required computing power are reduced. The use of cost-effective hardware is made possible.In particular, it is advantageous if the representative features are edges and / or surfaces. These can be determined by filtering with a low computing effort and can be stored with a low memory requirement.Furthermore, a vehicle having a device according to the invention for ascertaining a vehicle position of a vehicle in relation to a reference position is advantageous, since this has all the advantages of the device described above. In particular, the device according to the invention is included in this case by a parking system.Brief Description of the DrawingsHereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings. Identical or functionally identical parts are denoted by the same reference numerals. In the drawing, the following is: FIG. 1 shows a vehicle having a device according to the invention for ascertaining a vehicle position of the vehicle with respect to a reference position in a first exemplary embodiment, FIG. 2 shows an exemplary flow diagram of a method according to the invention for ascertaining a vehicle position of the vehicle with respect to a reference position, FIG. 3 shows an exemplary reference image which was acquired by the device according to the invention in the first embodiment and an associated exemplary comparison image which was acquired after an offset of the vehicle with respect to the reference position along its transverse axis, FIG. 4 shows an exemplary reference image which was acquired by the device according to the invention in the first embodiment and an associated exemplary comparison image which was acquired after an offset of the vehicle with respect to the reference position along its longitudinal axis, FIG. 5 shows an exemplary reference image which was acquired by the device according to the invention in the first embodiment and an associated exemplary comparison image which was acquired after a rotation of the vehicle with respect to the reference position along its vertical axis, and FIG. 6 shows a mobile device arranged on the vehicle, which comprises the camera of the device according to the invention in a second embodiment of the invention.Embodiments of the InventionFIG. 1 shows a vehicle 1 having a device according to the invention for ascertaining a vehicle position 2 of the vehicle 1 with respect to a reference position 3 in a first exemplary embodiment. The device comprises a camera 4 arranged on the vehicle 1. the camera 4 can be arranged at any location of the vehicle 1 and be directed with any orientation onto the environment of the vehicle 1. In this embodiment, the camera 4 is a rear view camera at the rear of the vehicle 1 and is directed rearward along a longitudinal axis of the vehicle 1 with respect to the vehicle 1. The apparatus further comprises an evaluation unit 5. the evaluation unit 5 is a digital computing unit which executes software which controls the sequence of the method according to the invention described below. For this purpose, the evaluation unit 5 in this first embodiment is connected to the camera 4 via a video line 6.FIG. 2 shows an exemplary flow chart for the method according to the invention in this first exemplary embodiment. The method is started by a user in this embodiment. This occurs in particular when the user intends to store a reference position 3 in the surroundings of which the vehicle 1 is typically parked. This reference position is, for example, a position in front of a garage which is particularly favorable for the driver to leave the vehicle 1. The vehicle 1 is configured to automatically move to a predefined target position, the position of which relative to the reference position 3 is known, based on the vehicle position 2 determined according to the invention. The target position is, for example, a parking position of the vehicle 1 in the garage.After the method has been started, a first method step S 1 is carried out. In the first method step S 1, a reference image 10 is captured by means of the camera 4 arranged on the vehicle 1, which images at least part of a surrounding area of the vehicle 1, wherein the vehicle 1 is located in the reference position 3. The reference image 10 is transmitted to the evaluation unit 5 and stored for a later access. In this first method step S 1, the reference position 3 is thus defined.After execution of the first method step S 1, the method in this embodiment is first paused in a waiting loop P and is only continued if a request for ascertaining a vehicle position 2 is present. Such a request could be made, for example, if the user requests an automatic parking process after he has parked the vehicle 1 in the region of the reference position 3. Optionally, further method steps are carried out during this pause. For example, a route between the reference position 3 and the target position is traversed by the vehicle 1 and the position thereof relative to the reference position 3 is thus stored for a later execution of an automatic movement of the vehicle 1 from the reference position 3 to this target position.If such a request for carrying out an automatic parking process is present, a second method step S 2 is carried out. In the second method step S 2, a comparison image 11 is captured by means of the camera 4 arranged on the vehicle 1. Since the method has been paused after the first method step S 1, it is possible and also to be expected that the vehicle 1 has been moved out of the reference position 3 and back into the region thereof and the comparison image 11 captured by the camera 4 does not exactly correspond to the reference image 10, but deviates from the reference position in accordance with an offset of the vehicle 1 from the reference position and thus in accordance with an offset of the camera 4. The comparison image 11 is also stored.After carrying out the second method step S 2, a third method step S 3 is carried out. In the third method step S 3, the reference image 10 is compared with the comparison image 11, wherein a change in position and a change in size of at least one feature in the comparison image 11 with respect to the reference image 10 are determined. A feature is, for example, an object imaged in the reference image 10. This object is sought in the comparison image 11 by means of software. If a feature was found from the reference image 10 in the comparison image 11, a change in position and a change in size of the feature are determined. In order to determine the change in position of the feature, the reference image 10 and the comparison image 11 are placed one above the other. A distance between the positions of a feature in the reference image 10 and in the comparison image 11 describes the change in position. To determine the change in size, the size of the feature in the reference image 10 is determined and the size of the feature in the comparison image 11 is determined. A difference between these quantities describes the change in size.After carrying out the third method step S 3, a fourth method step S 4 is carried out. In fourth method step S 4, a vehicle position 2 of vehicle 1 is ascertained with respect to reference position 3 on the basis of the change in position and the change in size. In this embodiment, a first offset 30 between the vehicle position 2 and the reference position 3 in the direction of the transverse axis of the vehicle 1 is determined, a second offset 31 between the vehicle position 2 and the reference position 3 in the direction of the longitudinal axis of the vehicle 1 is determined, and a rotation angle α of the vehicle position 2 relative to the reference position 3 is determined. An offset between the vehicle position 2 and the reference position 3 refers to the location of the vehicle at which the camera is arranged.In this embodiment, the first offset 30 is determined when there is a change in the position of the features in the horizontal direction, the size of the features remains the same, that is to say there is no change in size, and features in a near region of the camera 4 experience a greater change in position than features which are further away from the camera 4.FIG. 3 shows an exemplary reference image 10 (top) and an associated exemplary comparison image 11 (bottom) that was acquired after an offset of the vehicle 1 with respect to the reference position 3 along its transverse axis. As an exemplary feature, an entrance 12 is selected that is mapped with a first width 13 ain the reference image 10 and a second width 13 bin the comparison image 11. The entrance 12 is shifted horizontally to the left in the comparison image 11 by a first distance 14 with respect to the reference image 10. Thus, there is a change in position of the feature in the horizontal direction. The size of the entrance 12 is represented here by the first and second widths 13 a, 13 b, which respectively describe the width of the entrance 12 in the reference image 10 and the comparison image 11. The first width 13a is equal to the second width 13b in this example. There is therefore no change in size with respect to the entrance 12. As a further exemplary feature, a parked vehicle 15 is selected which is further away from the camera 4 than the entrance 12 The parked vehicle 15 is displaced horizontally to the left in the comparison image 11 with respect to the reference image 10 by a second distance 16. The first distance 14 is greater than the second distance 16. In the embodiment shown, a shift of the features to the left indicates a first offset 30 of the vehicle 1 relative to the viewing direction of the camera 4 to the right.The second offset 31 is determined in this embodiment when there is a change in size of the features and a change in position of the features in the vertical direction.FIG. 4 shows the exemplary reference image 10 (left) and an associated exemplary comparison image 11 (right) that was acquired after an offset of the vehicle 1 with respect to the reference position 3 along its longitudinal axis. As an exemplary feature, a channel cap 17 is selected that is mapped with a third width 18 ain the reference image 10 and a fourth width 18 bin the comparison image 11. The size of the channel cover 17 is represented here by the third and fourth widths 18 a, 18 b. The third width 18a is greater than the fourth width 18b in this example. There is thus a change in size with respect to the channel cover 17. In the comparison image 11, the channel cover 17 is displaced vertically downwards by a fifth distance 19 with respect to the reference image 10, whereby there is a change in position in the vertical direction. Thus, in the case shown in FIG. 4, a second offset 21 is determined. In the embodiment shown, a displacement of the features downward indicates a second offset 31 of the vehicle 1 toward the rear, i.e. in the direction of the viewing direction of the camera 4 and thus of the rear of the vehicle.In this embodiment, the angle of rotation α is determined when there is a change in the position of the features in the vertical direction, the size of the features remains the same, and features in a near range of the camera 4 experience the same change in position as features that are further away from the camera 4.FIG. 5 shows the exemplary reference image 10 (top) and an associated exemplary comparison image 11 (bottom) that was acquired after a rotation of the vehicle 1 by an angle α with respect to the reference position 3. As an example feature, a second parked vehicle 20 is selected. The second parked vehicle 20 is horizontally displaced to the right in the comparison image 11 with respect to the reference image 10 by a third distance 21. Thus, there is a change in position of the feature in the horizontal direction. As another exemplary feature, the channel cover 17 is selected, which is closer to the camera 4 than the second parked vehicle 20, and the channel cover 17 is horizontally displaced to the right in the comparison image 11 with respect to the reference image 10 by a fourth distance 22. The third path 21 is equal to the fourth path 22, and thus in the case shown in FIG. 3, a rotation of the vehicle 1 by an angle α is determined. In the embodiment shown, a displacement of the features to the right indicates a rotation of the vehicle 1 in the counterclockwise direction and thus a rotation of the camera 4 relative to its original viewing direction to the left.The vehicle position 2 of the vehicle 1 with respect to the reference position 3 is finally determined by a combination of the first offset 30, the second offset 31 and the rotation by the angle α. The magnitude of a deviation, i.e. the magnitude of the first offset 30, the second offset 31 and the rotation by the angle α, is a measure for the distance and the degree of rotation between the vehicle position 2 and the reference position 3.After carrying out the fourth method step S 4, the method branches back to the second method step S 2. In this way, the vehicle position 2 of the vehicle 1 is ascertained again with respect to a reference position 3 and is therefore constantly updated. This vehicle position 2 is provided to a control unit which carries out an autonomous longitudinal and transverse guidance of the vehicle in order to move the vehicle 1 into the target position. This is possible since both the position of the reference position 3 with respect to the target position and the current vehicle position 2 with respect to the target position are known. Therefore, only a deviation of the vehicle position from the target position has to be minimized. Obstacles can be bypassed in this case by individual waypoints being traveled along, the position of which relative to the reference position 3 is likewise known, for example by a preliminary travel along these waypoints. The vehicle position 2 of the vehicle can be specified by way of odometry as soon as a vehicle position 2 has been determined for the first time.A second embodiment of the invention substantially corresponds to the first embodiment, but the camera 4 is surrounded by a mobile device 40. The mobile device 40 is a smartphone in this second embodiment. FIG. 6 shows a mobile device 40 arranged on the vehicle 1, which comprises the camera 4 of the device according to the invention in a second embodiment of the invention. Above the dashboard 43 of the vehicle 1, a bracket 41 for the mobile device 40 is attached to the windshield 42 in the interior of the vehicle 1. The mobile device 40 can be inserted into the holder 41 by the user and can also be removed therefrom if necessary. In this second embodiment, the camera 4 is thus directed in the direction of the vehicle front.A data connection is established between the mobile device 40 and the evaluation unit 5 in order to transmit the reference image 10 and the comparison image 11 via the latter. The data connection can be effected, for example, via a data cable, a Bluetooth connection, a WLAN connection or similar. Optionally, a transmission of the reference image 10 and the comparison image 11 is also possible via an Internet service, with which both the mobile device and the evaluation unit 5 can exchange data.The holder 41 comprises a joint, by means of which the mobile device 40 can be brought into a position that is comfortable for the user. It may thus be that the vertical and / or horizontal orientation of the mobile device 40 with respect to the vehicle 1 and thus also the orientation of the camera 4 with respect to the vehicle 1 changes. In order to reduce errors in the determination of the vehicle position 2 of the vehicle 1 with respect to the reference position 3, a calibration with respect to the vertical and horizontal orientation of the camera 4 therefore takes place.In this second embodiment, such a calibration is carried out in each case before the acquisition of the reference image 10 and before the acquisition of the comparison image 11. From a movement of objects in these two images, it is possible to infer the vertical and horizontal orientation of mobile device 40 with respect to the vehicle. The comparison image 11 is correspondingly distorted in order to achieve a representation of the comparison image 11 as if it had been recorded with the same vertical and horizontal orientation of the mobile device 40 with respect to the vehicle 1, in which the reference image 10 was also recorded.Additionally or alternatively, a horizontal orientation of the mobile device 40 is detected by means of an inclination sensor in the mobile device during the detection of the reference image 10 and the comparison image 11 and a corresponding distortion of the comparison image 11 takes place in order to achieve a representation of the comparison image 11 as if it had been recorded with the same horizontal orientation of the mobile device 40 with respect to the vehicle 1, in which the reference image 10 was also recorded.Additionally or alternatively, a vertical orientation of the mobile device 40 is detected by means of a direction sensor in the mobile device during the detection of the reference image 10 and the comparison image 11, and a corresponding distortion of the comparison image 11 takes place in order to achieve a representation of the comparison image 11 as if it had been recorded with the same vertical orientation of the mobile device 40 with respect to the vehicle 1 in which the reference image 10 was also recorded.In order not always to have to compare the entire image, it is advantageous to extract relevant regions and to compare them. In this case, it is decisive how the relevant regions are selected. A corresponding third embodiment of the invention corresponds to the first or second embodiment, wherein the comparison of the reference image 10 with the comparison image 11 takes place based on selected representative features in the comparison image 11 and the reference image 10. Such representative features may be edges and / or surfaces, for example.In this case, representative features are selected in this third embodiment by additionally filtering the reference image 10 using a Sobel filter in the first method step S 1. Edges in the reference image are extracted by a Sobel filter, which then represent the representative feature. In the second method step S 2, the comparison image 11 is additionally filtered using the Sobel filter.In the third method step S 3, the filtered images and thus the representative features are compared in order to determine a change in position and / or a rotation. For this purpose, a Hough transform is used, for example, to find the edges of the reference image 10 in the comparison image 11. If a representative feature was found from the reference image 10 in the comparison image 11, an associated change in position and change in size is determined.In order to be able to determine the vehicle position 2 on the basis of a mono-image, it is important to use image areas that are very clear. Thus, areas with structure, for example with edges, are more suitable than areas without structural differences, such as areas with the same color. In this third embodiment, therefore, regions with high structural differences are first determined in order to recognize these regions of the reference image 10 in the comparison image 11. In this case, in this third embodiment, it is not necessary to store the complete reference image 10 and the complete comparison image 11, but it is sufficient to store the filtered reference image 10 and the filtered comparison image 11, whereby the amounts of data to be stored are significantly reduced.In addition to the above written disclosure, explicit reference is made to the disclosure of FIGS. 1, 2, 3, 4, 5 to 6.

Claims

Method for determining a vehicle position (2) of a vehicle (1) with respect to a reference position (3), comprising: - acquiring (S1) a reference image (10) by means of a camera (4) arranged on the vehicle (1), which images at least part of a surrounding area of the vehicle, wherein the vehicle (1) is located in the reference position (3), - acquiring (S2) a comparison image (11) by means of the camera (4) arranged on the vehicle (1), - comparing (S3) the reference image (10) with the comparison image (11), wherein a change in position and / or a change in size of at least one feature in the comparison image (11) with respect to the reference image (10) is determined, and - determining (S4) a vehicle position (2) of the vehicle (1) with respect to the reference position (3) on the basis of the change in position and / or the change in size, wherein - in the first method step (S1), the reference image (10) is additionally filtered using a Sobel filter, and - in the second method step (S2), the comparison image (11) is additionally filtered using the Sobel filter, wherein - the filtered reference image (10) and the filtered comparison image (11) are stored, and - in the third method step (S3), the filtered images are compared, wherein - the acquisition of the reference image (10) and of the comparison image (11) is carried out by a mobile device (40), in particular a smartphone, which is arranged in the vehicle (1) in such a way that the camera (4) images at least part of a surrounding field of the vehicle (1).Method according to Claim 1, characterized in that the change in position is a change in the position of a feature in the horizontal direction.Method according to one of the preceding claims, characterized in that the change in position is a change in the position of a feature in the vertical direction.Method according to claim 1, characterised in that the mobile device (40) is arranged in a holder (41) provided for this purpose.Method according to one of the preceding claims, characterized in that a calibration takes place with respect to the vertical and horizontal orientation of the camera (4).Mobile device (40), in particular a smartphone, for determining a vehicle position (2) of a vehicle (1) with respect to a reference position (3), comprising: - a camera (4) which is configured to: ◯ capture a reference image (10) which images at least part of a surrounding area of the vehicle (1) when the vehicle (1) is located in the reference position (3), and o capture a comparison image (11), and - an evaluation unit (5) which is configured to: o additionally perform filtering of the reference image (10) with a Sobel filter, o additionally perform filtering of the comparison image (11) with the Sobel filter, ◯ store the filtered reference image (10) and the filtered comparison image (11), ◯ compare the reference image (10) with the comparison image (11), wherein the filtered images are compared and a change in position and / or a change in size of at least one feature in the filtered comparison image (11) with respect to the filtered reference image (10) is determined, and ◯ a vehicle position (2) of the vehicle (1) with respect to the reference position (3) is determined on the basis of the change in position and / or the change in size.

Images