Method for positioning a motor vehicle
By using a fisheye lens camera to acquire image data, combined with map data and GNSS signals, the problems of complex data processing and satellite signal shielding in motor vehicle positioning are solved, achieving efficient and accurate positioning results.
Patent Information
- Application Number
- CN202510313035.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-03-18
- Filing Date
- 2025-03-17
- Publication Date
- 2025-09-19
AI Technical Summary
In the prior art, when using environmental sensors to locate a motor vehicle, there are problems such as complex data processing and GNSS satellite signals being easily shielded, resulting in low positioning accuracy.
An upward-pointing camera with a fisheye lens is used to acquire image data. This data is combined with map data and GNSS satellite signals to perform positioning by identifying the idle visual range and environmental features, reducing dependence on GNSS satellite signals.
It achieves efficient and accurate vehicle positioning, reduces computational complexity and improves positioning accuracy, and is particularly suitable for urban environments.
Smart Images
Figure CN120668157A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for locating a motor vehicle. Background Art
[0002] Vehicle localization typically uses numerous environmental sensors, such as lidar, radar, or cameras, which monitor the vehicle's surroundings. The data acquired by these sensors is evaluated in the vehicle using a control unit and a computer, which implements filters and can perform localization or position determination based on this data.
[0003] Here, a new scheme for acquiring data for positioning using sensors and performing positioning based on the data acquired using these sensors will be described. Summary of the Invention
[0004] A method for locating a motor vehicle is described herein, comprising the following steps:
[0005] a) receiving image data, which are acquired by means of a camera arranged on a motor vehicle and directed upwards, the camera having a fisheye lens and a viewing axis directed upwards; and
[0006] b) performing a localization operation for determining the position of the motor vehicle by using the image data received in step a).
[0007] The motor vehicle is preferably a passenger vehicle. However, the method is equally applicable to various types of motor vehicles. The camera is, in particular, a video camera that can be used to permanently record current image data of the vehicle's surroundings. In the context of the method, the camera is an environmental sensor that can be used to monitor the vehicle's surroundings.
[0008] Localization here refers in particular to the determination of the position of the vehicle at a specific point in time (at the time of localization).
[0009] The camera used here features a fisheye lens. A fisheye lens is a special lens with a very large viewing angle (extreme wide-angle lens) and, in particular, can image the entire field of view. Unlike conventional non-fisheye lenses, which image proportionally to an object plane perpendicular to the optical axis (sundial projection), fisheye lenses image a larger portion of a hemisphere or even a sphere, with sharp but not excessive distortion. Therefore, the projection method used for fisheye lenses is preferably not sundial projection, but rather hemispherical imaging. A fisheye lens images straight lines that do not pass through the image center as curves. Using a fisheye lens generally allows for more realistic imaging of area ratios or radial distances than using conventional wide-angle lenses using sundial projection. The viewing angle of a fisheye lens is typically 180°, but in extreme cases it can reach 220°. Such extreme image angles are impossible to achieve using conventional (especially sundial) projection methods. Despite the extremely large image angle, the brightness drop at the edges of the image is easier to correct with a fisheye lens than with a wide-angle lens because the image scale does not increase as much at the edges, and the light does not need to illuminate such a large area.
[0010] By using a fisheye lens to acquire camera image data, image data can be generated that offers new and particularly advantageous possibilities for performing localization. Methods and techniques for localization using such image data are characterized by their efficient implementation (with reduced computational effort). Methods and techniques for localization based on such image data also have the advantage that relevant information can be derived from the image data with little or no further data enrichment.
[0011] By using a camera with a fisheye lens, very comprehensive information can be obtained with only one camera (only one sensor).
[0012] Especially when the camera is oriented upward, the image data captured by the camera is initially irrelevant to the current state of the vehicle. In step a), image data that covers the hemisphere surrounding the vehicle as completely as possible is preferably acquired. This image data already contains highly relevant information about the vehicle's surroundings, which can be directly and regularly used to generate information relevant to automated or highly automated driving functions.
[0013] Particularly preferably, in step b), the vehicle is localized by comparing the image data with map data describing the road conditions.
[0014] The image data preferably depicts the situation above the vehicle, referred to herein as the road situation. Depending on the angle of view of the fisheye lens, the image data received in step a) also includes information about the vehicle's surroundings in the plane in which the vehicle is located. At an angle of view of 180 degrees, the image data also includes image information about the situation at the level of the plane in which the vehicle is located. This image data is particularly located in the edge regions of the image contained in the image data.
[0015] Furthermore, it is preferred that in step b) road signs are identified in the image data and are compared with the map data for localization.
[0016] Road signs are typically fixed to elevated signposts in the surrounding area of a roadway. The name of the road on which the road sign is located is typically written on the road sign. Positioning can be performed by comparison with the road name information in the map data. Particularly preferably, the map data also contains the exact coordinates of the road signs installed along the road. Once a road sign is identified using the described method, positioning can be performed using the coordinates of the corresponding road sign stored in the map data.
[0017] Furthermore, it is preferred that houses are identified in the image data and compared with map data for positioning.
[0018] For example, house outlines can be identified in the image data, which are then further identified in the map data for positioning purposes. In another embodiment, house facades can also be identified in the image data, which are then further identified in the map data for positioning purposes. Positioning by identifying houses is particularly useful in highly developed urban areas near the roadway where the vehicle is located.
[0019] Furthermore, it is preferred that in step b) an unobstructed visual range above the vehicle is identified using the image data, wherein GNSS satellite signals are selected for performing the positioning by taking the unobstructed visual range into consideration.
[0020] Furthermore, it is preferred that in step b), the GNSS satellite signals are selected from GNSS satellites whose line of sight toward the motor vehicle extends through an idle visible range.
[0021] A significant challenge in determining a position using GNSS signals from Global Navigation Satellite System (GNSS) satellites (such as GPS, BeiDou, and / or GLONASS) is the obstruction of visible satellites by tall objects in the surrounding environment, as well as distortion of the GNSS satellite signals due to signal reflections and multipath effects. Excluding satellites from the position determination that are not directly visible to the signal receiver is a very reliable method for improving the accuracy of position determination using GNSS satellite signals. However, in practice, it is often difficult to identify which satellites have a direct line of sight.
[0022] Here, it is now proposed to use image data from an upward-facing camera with a fisheye lens to identify the clear visual range above the motor vehicle. The clear visual range identified in step b) is preferably conical and describes a cone extending upward from the vehicle. Using the camera's image data, this cone or clear visual range can be very easily determined based on the width or size of the clear area in the image data. Typically, a hemisphere surrounding the fisheye lens is projected onto a plane using the fisheye lens. Distances from the image center point in the image data can preferably be converted into angles between the viewing axes of the camera or fisheye lens. A circle around the image center point in the image data corresponds to a cone within the hemisphere. The radius of this circle preferably corresponds to the angle of the cone. This cone can be determined very efficiently from the image data. This cone can be used to select GNSS satellites for positioning. Preferably, orbital data indicates where the specific GNSS satellites are visible and the angles at which their GNSS satellite signals reach the vehicle. These angles can be compared with the clear visual range determined from the image data. Satellites are used for positioning only when their boresight passes through a clear viewing range (or cone).
[0023] In a variant embodiment of the method, the clear visual range can be defined not only in the form of a cone. Due to common conditions (for example, in built-up areas), the horizon from the vehicle is often farther away or lower than transversely to the direction of travel. In other words, along the roadway, the horizon is lower. On the roadside, the horizon is often limited by adjacent buildings. Based on the data obtained by a camera with a fisheye lens, the angle of the horizon is preferably determined separately along the entire periphery of the fisheye lens (around the observation axis) to define the clear visual range. In this way, the actual clear visual range from the motor vehicle can be identified very accurately. As a result, the number of GNSS satellites that are unnecessarily (unnecessarily) excluded from positioning can be reduced.
[0024] Furthermore, it is preferable that the fisheye lens have an imaging angle exceeding 160 degrees.
[0025] Particularly preferably, the imaging is 180 degrees or more, so that the camera captures up to the plane in which the camera or the vehicle is located.
[0026] Furthermore, it is preferred that the viewing axis of the fisheye lens deviates from the vertical direction of the motor vehicle by a maximum of 10 degrees.
[0027] The viewing axis of the camera with the fisheye lens is preferably positioned vertically on the motor vehicle, wherein the vertical direction is vertical when the vehicle is parked on level ground. If the vehicle is parked on an inclined surface, the viewing axis is tilted accordingly to the inclination of the surface.
[0028] A control unit for a motor vehicle is also described here, which is configured to carry out the method.
[0029] Here, a motor vehicle is likewise described, which has a camera with a fisheye lens, directed upward, and at least one control unit described above, which is provided to carry out the method. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The present invention and its technical background will be explained in more detail below with reference to the accompanying drawings. The drawings illustrate preferred embodiments of the present invention but do not limit it. It should be noted in particular that the drawings and in particular the proportions shown in the drawings are only schematic. The drawings show:
[0031] Figure 1 : the motor vehicle;
[0032] Figure 2 : Example of image data;
[0033] Figure 3 : Example of map data
[0034] Figure 4 : Description of the idle visual range; and
[0035] Figure 5 : Flowchart of the method. DETAILED DESCRIPTION
[0036] Figure 1 The motor vehicle 1 is shown with a camera 3 with a fisheye lens 4. Camera 3 is mounted on the roof 26 of motor vehicle 1 to provide a very good all-round view, allowing it to monitor the entire surroundings of motor vehicle 1. Camera 1 has a viewing axis 15. This viewing axis 15 is preferably oriented in a vertical direction 16. The center point of the image produced by camera 3 lies on viewing axis 15. Camera 3 or fisheye lens 4 has an imaging angle 14, which defines the area around imaging axis 15 that can be captured by camera 3 and is 180 degrees in this case. Thus, camera 3 can capture the entire hemisphere surrounding motor vehicle 1. If necessary, the camera can be positioned even higher relative to roof 26. It is particularly advantageous if the imaging angle 14 of camera 3 or fisheye lens 4 is even greater than 180 degrees. This allows camera 3 to also monitor a plane 27 (stationary plane) in which motor vehicle 1 is located. Motor vehicle 1 preferably has a controller 17 or computer in which the method described herein is implemented.
[0037] Figure 2An example of image data 2 that can be captured using a camera is shown. Image data 2 shows road conditions 7, which include tall buildings 9 in the surroundings of the roadway on which motor vehicle 1 is located. Road signs 8 are also identified. For example, localization can be performed by identifying buildings 9 in map data and / or by evaluating road signs 8.
[0038] Figure 3 The map data 6 are schematically shown, which for example show Figure 2 . The position 5 of the motor vehicle 1 in the map data 6 can be identified. The road 25 on which the motor vehicle 1 is located and the other roads 25 can also be identified. The position 5 can be determined by comparing the image data 2 with the map data 6.
[0039] Figure 4 The method for determining the free viewing range 10 by means of the method is explained. Figure 4 The upper area of the diagram shows a road situation 7 with motor vehicle 1 at position 5. A three-dimensional schematic representation of this situation is shown on the left. The same situation is shown from a horizontal perspective on the right. Houses 9 can be seen on either side of the road 25 where motor vehicle 1 is located. The viewing axes 15 of the cameras on motor vehicle 1 (not shown separately here) are indicated.
[0040] Figure 4 The lower area of shows how free viewing range 10 is affected by road conditions 7 with buildings 9. Due to buildings 9, free viewing range 10 is formed along road 25 and is particularly narrowed on the right and left sides of motor vehicle 1. In the image data, the angle of the horizon relative to viewing axis 15 of camera 3 can be identified in any direction in the plane surrounding the vehicle. This allows for efficient determination of free viewing range 10.
[0041] In the lower left area, an unobstructed visual range 10 is shown projected from above. GNSS satellites are each indicated by a cross. For each GNSS satellite 12 within the unobstructed visual range 10, there is an unobstructed line of sight 13 that is not disturbed by the building 9, so that the GNSS signal 11 from the GNSS satellite 12 can directly reach the vehicle 1. The line of sight 13 of the GNSS satellite 12 outside the unobstructed visual range 10 is disturbed. Figure 4 In the lower right area, this can be seen even better in a horizontal perspective.
[0042] Figure 5A flow chart of the method is shown, which can be implemented in a control unit 17 or a computer of a motor vehicle 1. Schematically shown is a camera 3 with a fisheye lens 5, which generates image data 2. In the control unit 17, the image data 2 is fed to an object detection unit 18 to execute the method, using which objects are identified in the image data 2. The image data 2 processed in this manner (preferably also including data about objects in the surroundings of the motor vehicle 1) is then fed to a visual range detection unit 21. In a variant embodiment, data from a GNSS receiver 20, in particular data including GNSS signals 11, can also be used for visual range detection 2. In a sensor fusion 22, the output data of the visual range detection unit 21, in particular describing the clear visual range 10, are fused with data from the GNSS receiver 20 and, if necessary, data from an IMU (inertial sensor) 19. This results in a positioning 23 and, if necessary, an additional velocity determination 24 to determine the velocity of the motor vehicle 1.
Claims
1. A method for locating a motor vehicle (1), comprising the following steps: a) receiving image data (2), which are acquired by means of a camera (3) arranged on the motor vehicle (1) and directed upwards, the camera having a fisheye lens (4) and a viewing axis (15) directed upwards; and b) performing a localization operation to determine the position (5) of the motor vehicle (1) by using the image data (2) received in step a).
2. The method according to claim 1, wherein in step b), the motor vehicle (1) is located by comparing the image data (2) with map data (6) depicting road conditions (7).
3. The method according to claim 2, wherein in step b) road signs (8) are identified in the image data (2), the road signs being compared with map data (6) to perform the localization.
4. The method according to claim 2 or 3, wherein in step b) a house (9) is identified in the image data and the house is compared with map data (6) to perform positioning.
5. The method according to claim 1 , wherein in step b), a clear visual range (10) above the motor vehicle (1) is identified using the image data (2), wherein GNSS satellite signals (11) are selected for performing the positioning by taking into account the clear visual range (10).
6. The method according to claim 5, wherein in step b), the GNSS satellite signals (11) are selected from the following GNSS satellites (12): the line of sight (13) of the GNSS satellite towards the motor vehicle (1) extends through the clear visual range (10).
7. Method according to any of the preceding claims, wherein the fisheye lens (4) has an imaging angle (14) exceeding 160 degrees.
8. The method according to claim 1, wherein the viewing axis (15) of the fisheye lens (4) deviates from the vertical (16) of the motor vehicle (1) by a maximum of 10 degrees.
9. A control unit (17) for a motor vehicle (1), the control unit being configured to carry out the method according to any one of the preceding claims.
10. A motor vehicle (1) having an upwardly directed camera (3) with a fisheye lens (4) and at least one control unit (17) according to claim 9, the control unit (17) being configured to carry out the method according to any one of claims 1 to 8.