Method for positioning a map representation of a vehicle's surroundings within a semantic road map - Patents.com

Abstract

The invention relates to a method (100) for positioning a map representation (200) of a surrounding environment (501) of a vehicle (500) in a semantic road map (300), comprising: a step (101) of receiving surrounding environment sensor data (505) of at least one surrounding environment sensor (503) of the vehicle (500); a step (103) of creating a map representation (200) of the surrounding environment (501) of the vehicle (500) based on the surrounding environment sensor data (505) of the vehicle (500); a step (105) of determining characteristic elements (201) in the map representation (200) and characteristic elements (301) in the semantic road map (300) which describe the surrounding environment (501); a step (107) of comparing the characteristic elements (201) of the map representation (200) with the characteristic elements (301) of the semantic road map (300); The present invention relates to a method (100) comprising: a step (109) of determining (303) an area (303) of a semantic road map (300), in which, in respect of the area (303), at least one characteristic element (301) of the area (303) of the semantic road map (300) matches at least one characteristic element (201) of the map representation (200) and the map representation (200) and the semantic road map (300) depict the same area of ​​the surrounding environment (501) of the vehicle (500); and a step (111) of identifying a position (P) of the area (303) in the semantic road map (300) as the position of the map representation (200) in the semantic road map (300).

Description

[Technical field]

[0001] The present invention relates to a method for positioning a map representation of a vehicle's surroundings within a semantic road map. [Background technology]

[0002] To accurately control a vehicle, especially an autonomous vehicle, a detailed map representation of the surroundings of the vehicle to be controlled is required. In this regard, data of the surroundings can be recorded while the vehicle is traveling or during a test drive of the vehicle designed for this purpose, and a semantic road map with semantic information about the surroundings can be created based on this data. However, such a road map must be kept consistently up to date and must be provided with additional details. Summary of the Invention [Problem to be solved by the invention]

[0003] It is therefore an object of the present invention to provide an improved method for positioning a map representation of a vehicle's surroundings in a semantic road map. [Means for solving the problem]

[0004] This problem is solved by a method for positioning a map representation of the vehicle's surroundings in a semantic road map according to the independent claim 1. Advantageous configurations are the subject of the dependent claims.

[0005] According to one aspect of the invention, there is provided a method for positioning a map representation of a vehicle's surroundings within a semantic road map, comprising: receiving ambient environment sensor data from at least one ambient environment sensor of the vehicle; generating a map representation of the vehicle's surroundings based on the vehicle's surroundings sensor data; determining feature elements in a semantic road map depicting feature elements in a map display and surroundings; comparing characteristic elements of the map display with characteristic elements of the semantic road map; determining an area of ​​the semantic road map for which a number of characteristic elements of the area of ​​the semantic road map match a number of characteristic elements of the map representation and the map representation and the semantic road map depict the same region of the vehicle's surroundings; and identifying the location of the area in the semantic road map as a location of a map representation in the semantic road map.

[0006] Thereby, a technical advantage can be achieved that an improved method can be provided for positioning a map representation of the vehicle's surroundings in a semantic road map. In this regard, based on surroundings sensor data of at least one surroundings sensor of the vehicle, a map representation of the vehicle's surroundings depicted by the surroundings sensor data is created. The map representation can in this case be limited to the vehicle's immediate surroundings and shows objects arranged in the surroundings. Following this, characteristic elements in the map representation and characteristic elements in the semantic road map are determined in order to position the map representation. The characteristic elements can be, for example, objects arranged in the surroundings that allow for characterization of the surroundings. The characteristic elements can be, for example, road markings or road signs in the surroundings. In the semantic road map, such objects are stored as characteristic elements with a semantic description.

[0007] To determine the location, the characteristic elements of the map display are compared with the characteristic elements of the road map, and an area in the semantic road map is determined, for which at least one characteristic element of this area of ​​the semantic road map matches at least one characteristic element of the map display. Furthermore, the location of this area in the semantic road map is identified as the positioning of the map display in the semantic road map. This area may in this case have an extent corresponding to the extent of the map display.

[0008] A semantic road map, in the sense of this application, is a digitized road map that carries semantic information about the objects represented by the road map. According to one embodiment, the step of comparing the characteristic elements comprises: determining a descriptor for each feature element of the semantic road map and a descriptor for each feature element of the map display; Comparing the keypoint descriptors of the feature elements of the semantic road map with the keypoint descriptors of the feature elements of the map display, in which the feature elements match if the keypoint descriptors of the respective elements match.

[0009] Thereby, a technical advantage may be achieved in that accurate comparison between various feature elements of a semantic road map and a map display is enabled by the feature element descriptors and comparison of the various feature element descriptors.

[0010] In the sense of this application, a characteristic element in a map display coincides with a characteristic element in a semantic road map if the characteristic elements in each map are from the same semantic class and are positioned at equivalent locations in each map.

[0011] According to one embodiment, the step of determining the descriptors of the characteristic elements comprises: determining a geometric grid structure of descriptors at the locations of the key points of each characteristic element in the map display or semantic road map, the geometric grid structure including a number of adjacent spatial regions; determining a property of each characteristic element for each spatial region of the lattice structure; - comparing the value of the property in one spatial region with the value of the property for the spatial regions immediately adjacent to this spatial region of the geometric structure; assigning the spatial regions of the lattice structure a numerical value of 1 if the value of the property in each spatial region is greater than the value of the property in the immediately adjacent spatial region, and assigning the spatial regions a numerical value of 0 if the value of the property in that spatial region is not greater than the value of the property in the immediately adjacent spatial region.

[0012] Thereby, the technical advantage can be achieved that a detailed and unambiguous descriptor can be provided, in this respect the descriptor is formed as a neighbor binary landmark descriptor, which allows an unambiguous description of different characteristic elements of a cartographic representation as well as of different characteristic elements of a semantic road map.

[0013] Alternatively, a descriptor without neighborhood comparison can be used, in which no comparison of adjacent spatial regions is performed. According to one embodiment, the step of comparing the descriptors comprises: It involves performing a nearest neighbor search for the descriptors in the k-dim tree.

[0014] This may achieve a technical advantage in that an accurate comparison of various descriptors of semantic road maps and map displays is enabled. According to one embodiment, descriptors for characteristic elements of different semantic classes are determined and only the descriptors for characteristic elements of the same semantic class are compared with each other.

[0015] This may achieve the technical advantage that a comparison of the descriptors of different characteristic elements may be more accurate.By adding semantic information about the characteristic elements, a more accurate comparison of the characteristic elements and thus the positioning of the map representation within the semantic road map may be achieved.

[0016] According to one embodiment, semantic classes of feature elements include road markings, road boundary lines, street lights, traffic lights, and traffic signs. This may achieve the technical advantage that various objects commonly found in the vehicle's surroundings may be taken into account for the positioning of the map representation in the semantic road map, which may make the positioning more accurate. The road markings taken into account may include solid or dashed lines in white or yellow.

[0017] According to one embodiment, the property is one from the following list: point density of points in the spatial domain, length of the characteristic element displayed as a line in the spatial domain, area of ​​the characteristic element displayed as a surface in the spatial domain, volume of the characteristic element displayed as a volume in the spatial domain.

[0018] This can achieve the technical advantage that depending on the geometric properties of the characteristic elements, the corresponding properties are taken into account in the descriptor. This allows various geometrically formed characteristic elements to be taken into account by the descriptor, which increases the number of characteristic elements available for comparison, which can lead to a more accurate positioning of the map display relative to the semantic road map. In order to calculate the descriptor, the corresponding part of the map display or the semantic road map containing the characteristic elements can be rotated around the respective key point for which the descriptor is to be calculated, depending on the average orientation of the road markings. This part can be formed, for example, as a circle. The rotation can be performed until a match between the map display and the semantic road map is achieved, i.e. due to this rotation, maps rotated relative to each other can be compared with each other. This can achieve the rotation invariance of the method.

[0019] According to one embodiment, the step of determining the area comprises: generating multiple location hypotheses for possible locations of the map display within the semantic road map, the generating of location hypotheses comprising: performing a position translation in the semantic road map of key points of characteristic elements in the semantic road map whose descriptors match the descriptors of the characteristic elements of the map display by a distance corresponding to the distance of the respective corresponding characteristic element of the map display to the center point of the map display in the map display; identifying the locations of the clusters as location hypotheses for the map display; and verifying the location hypothesis as the location of the area, the verifying step comprising: determining the number of keypoints displaced to the centers of the clusters and identifying the cluster with the largest number of keypoints displaced to the centers of the clusters as the location hypothesis for the area having the greatest match with the map representation; or The method includes the steps of: rendering a map display within a semantic road map based on the location hypothesis; comparing characteristic elements at the location of the location hypothesis with characteristic elements of the map display rendered at the location of the location hypothesis; and identifying the location hypothesis with the greatest number of matching characteristic elements.

[0020] This may achieve the technical advantage that multiple location hypotheses may be made for the positioning of the map display relative to the semantic road map. With multiple location hypotheses, the actual positioning of the map display may become more accurate. That is, in the selection process, the location hypothesis that displays the correct position of the map display relative to the semantic road map with the greatest probability is selected. In this case, the selected location hypothesis corresponds to the area in the semantic road map that has the most characteristic elements that match the characteristic elements of the map display.

[0021] In the sense of the present application, the descriptors of a characteristic element match if these descriptors have similarity. According to one embodiment, the comparison of the characteristic elements at the location of the location hypothesis with the characteristic elements of the map representation depicted at the location of the location hypothesis includes performing a RANSAC algorithm.

[0022] This may achieve the technical advantage that by implementing the RANSAC algorithm, outliers, particularly with regard to the positioning of characteristic elements within the semantic road map, may be removed and left unconsidered in the comparison, thereby achieving an accurate comparison of characteristic elements and an accurate determination of maximum matches.

[0023] According to one embodiment, the method further comprises: performing a pre-positioning of the map display relative to the semantic road map based on the geographic location of the vehicle's surroundings displayed by the map display; and determining a sub-area of ​​the semantic road map based on the determined geographic location, where characteristic elements of the semantic road map within the sub-area are determined.

[0024] This may achieve the technical advantage of allowing for simplified positioning of the map display relative to the semantic road map. Due to the pre-positioning based on the geographical location of the surroundings displayed by the surroundings sensor data of the vehicle's surroundings sensor, the semantic road map for comparing the characteristic elements may be reduced to a sub-area including the geographical location of the surroundings of the vehicle. This may simplify the comparison process between the characteristic elements of the map display and the characteristic elements of the semantic road map, since only the characteristic elements of the semantic road map located within the sub-area located around this geographical location are taken into account.

[0025] According to one embodiment, the method further comprises: Forming feature associations between features of the feature elements of the semantic road map and features of the feature elements of the map display for which there is a respective match of the respective descriptor.

[0026] Thereby, a technical advantage may be achieved in that positioning of a map representation within a semantic road map may be made more accurate by taking into account feature associations of features of characteristic elements of the semantic road map and the map representation.

[0027] According to a second aspect, there is provided a method for supplementing a semantic road map with information of a map representation of a vehicle's surroundings, comprising: - performing a method for positioning a map representation of the vehicle's surroundings in a semantic road map according to one of the above embodiments; and incorporating map display information into the defined area of ​​the semantic road map.

[0028] This can achieve the technical advantage of providing an improved method for supplementing a semantic road map with information about a map representation of a vehicle's surroundings, including an improved method for positioning a map representation of a vehicle's surroundings within a semantic road map, having the technical advantages listed above.

[0029] According to a third aspect of the invention, there is provided a computing unit adapted to perform a method for positioning a map representation of a vehicle's surroundings within a semantic road map and / or a method for supplementing the semantic road map with information of the map representation of the vehicle's surroundings according to one of the above embodiments.

[0030] According to a fourth aspect of the present invention, there is provided a computer program product comprising instructions which, when executed by a data processing unit, cause the data processing unit to perform a method for positioning a map representation of a vehicle's surroundings within a semantic road map and / or a method for supplementing a semantic road map with information of the map representation of a vehicle's surroundings according to one of the above embodiments.

[0031] Exemplary embodiments of the present invention will now be described with reference to the following drawings. [Brief description of the drawings]

[0032] [Figure 1] 1 is a schematic diagram of a vehicle equipped with a computing unit for executing a method for positioning a map representation of the vehicle's surroundings in a semantic road map; [Diagram 2] FIG. 2 is a schematic diagram of a semantic road map and map display of a vehicle's surroundings. [Diagram 3] FIG. 2 is a schematic diagram of a descriptor. [Figure 4] FIG. 2 is a schematic diagram of a descriptor applied to a semantic road map or map display; [Diagram 5]FIG. 2 is a flow diagram of a method for positioning a map representation of a vehicle's surroundings within a semantic road map. [Figure 6] FIG. 2 is a flow diagram of a method for supplementing a semantic road map with information of a map display of the vehicle's surroundings. [Figure 7] FIG. 1 is a schematic diagram of a computer program product. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] FIG. 1 shows a schematic diagram of a vehicle 500 equipped with a computation unit 505 for executing the method 100 for positioning a map representation 200 of the vehicle's surroundings 501 in a semantic road map 300 .

[0034] 1 shows a vehicle 500 on a roadway 507 with roadway boundaries 509 and road markings 511. The vehicle 500 comprises at least one surrounding environment sensor 503 and a computing unit 505. With the surrounding environment sensor 503, the vehicle 500 can record surrounding environment sensor data of the surrounding environment 501 surrounding the vehicle 500. The computing unit 505 is adapted to implement the method according to the invention for positioning a map representation of the surrounding environment of the vehicle in a semantic road map. The computing unit 505 can further be adapted to execute the method according to the invention for supplementing the semantic road map with information of the map representation of the surrounding environment of the vehicle. For this purpose, the semantic road map can be stored in the computing unit 505, for example in a corresponding memory device.

[0035] FIG. 2 shows a schematic diagram of a semantic road map 300 and a map display 200 of a vehicle's 500 surroundings 501 . Figure 2 shows a map view 200 of a vehicle's surroundings in accordance with the vehicle 500 shown in Figure 1. The map view 200 is based on surroundings sensor data from surroundings sensors 503 of the vehicle 500, which are recorded from the surroundings 501 surrounding the vehicle 500 at each moment while the vehicle 500 is travelling. In the embodiment shown, the map view 200 captures a roadway 203 having roadway boundaries 205 and pavement markings 207. The map view 200 further shows street lights 209 located at the edges of the roadway 203.

[0036] 2 further illustrates a semantic road map 300, which also illustrates a roadway 304 with roadway boundaries 305 and pavement markings 307. The semantic road map 300 illustrates a larger portion of the surrounding environment than the map view 200. Thus, the semantic road map illustrates a large number of different objects located along the roadway 304, such as street lamps 309 or traffic lights 311.

[0037] In order to position the map representation 200 in the semantic road map 300 based on the method according to the invention, firstly a characteristic element 201 is determined in the map representation 200. The characteristic element 201 may for example be caused by a road boundary 205, a road marking 207 or a street light 209. Furthermore, a corresponding characteristic element 301 is determined in the semantic road map 300. The characteristic element 301 may also be caused by a road boundary 305, a road marking 307, a street light 309 or a traffic light 311.

[0038] Following this, the determined characteristic elements 201 of the map display 200 are compared with the determined characteristic elements 301 of the semantic road map. By comparing the characteristic elements 201 of the map display 200 with the characteristic elements 301 of the semantic road map 300, an area 303 is determined in the semantic road map 300. The determined area 303 is characterized in that the characteristic elements 301 located in the determined area 303 at least partially match the characteristic elements 201 of the map display 200. Thus, the thus determined area 303 includes at least one characteristic element 301 that matches the characteristic element 201 of the map display 200. In this case, the match of the characteristic elements may be brought about by the fact that the respective characteristic elements are from the same semantic class and are located in comparable positions in the map display 200 and the semantic road map 300.

[0039] Subsequently, to determine the position of the map display 200 within the semantic road map 300, the position P of the area 303 within the semantic road map 300 is identified as the positioning of the map display 200 within the semantic road map 300. The position P of the map display 200 within the semantic road map 300 thus determined may therefore be interpreted as the area 303 determined by comparison of the characteristic elements 201, 203 depicting the surrounding environment 501 of the vehicle 500 displayed by the map display 200.

[0040] To compare the characteristic elements 201 of the map representation 200 with the characteristic elements 301 of the semantic road map 300, descriptors can be determined for each keypoint of the characteristic elements 301 of the semantic road map 300 and for each keypoint of the characteristic elements 201 of the map representation 200 and compared with each other. In this respect, the descriptors can be formed as neighbor binary landmark descriptors. For this purpose, a grid structure with a number of spatial regions is determined for each keypoint, and a value of the property of the considered characteristic element is determined for each spatial region of the grid structure. To create the descriptor, the determined values ​​of the property are compared with each other for a number of spatial regions in the grid structure, and the spatial regions of the grid structure in which the property of the characteristic element has a higher value than the directly adjacent spatial regions are assigned the value 1, whereas the other spatial regions are assigned the value 0 accordingly. The vectors resulting on this basis can be compared with each other for any characteristic elements of the map representation 200 or of the semantic road map 300.

[0041] For comparison, inter alia, an averaging of the descriptors for the characteristic elements of one semantic class may be performed, which may be effected, for example, by taking a weighted average value of multiple descriptors of multiple characteristic elements of one semantic class.

[0042] To compare different descriptors, a nearest neighbor search of the descriptors in a k-dim tree can be performed. To compare the descriptors of the various characteristic elements 201, 301, the characteristic elements 201, 301 can be considered in terms of their semantic class, so that only the descriptors of the characteristic elements 201, 301 whose characteristic elements are assigned to the same semantic class are compared. Alternatively, one averaged descriptor can be used for multiple characteristic elements of one semantic class as described above, so that the same averaged descriptor is used for each element of the same class. In this case, the semantic class can be brought about by the objects described above, for example the road boundary 305, the road markings 307, the street lamps 309 or the traffic lights 311. The key points on which the calculation of the descriptors is performed can be each point of one characteristic element in the map display 200 or the semantic road map 300.

[0043] In order to compare the characteristic elements 201, 301 or their respective descriptors, a rotation of the map display 200 relative to the semantic road map 300 may be performed. This rotation allows the map display and the semantic road map 300 to have the same orientation, which allows for an accurate comparison of the descriptors of the key points of the characteristic elements 201, 301 to be compared, and therefore an accurate positioning of the map display 200 within the semantic road map 300.

[0044] Rotation of the map view 200 relative to the semantic road map 300 may be performed around the center of a cluster 312 of corresponding location hypotheses 313 based on the major axes or average line directions of the map view 200 and the semantic road map 300 .

[0045] Depending on the road segment displayed by the map display 200, based on the high periodicity of the road extension related to road boundaries or road markings or related to the arrangement of road lighting masts or traffic lights, the number of possible positions of the road segment displayed by the map display 200 in the semantic road map 300 may be increased. This allows for a large number of possible position hypotheses to be made, which indicate with different probabilities the correct position of the road segment displayed by the map display 200 in the semantic road map 300, depending on the characteristic elements 201 of the map display 200.

[0046] In order to determine the location of the map display 200 in the semantic road map 300, a number of location hypotheses are created. In this regard, for those characteristic elements 301 of the semantic road map 300 for which a match of the descriptors with the descriptors of the characteristic elements 201 of the map display 200 has been determined, a location displacement of the keypoints can be performed with a displacement distance corresponding to the distance of each keypoint of the characteristic elements 201 of the map display 200 to the center point of the map display 200. A cluster 312 of displaced keypoints in the semantic road map 300 thus formed can be interpreted as a location hypothesis 313 from now on. In this regard, a geometric validation of the location hypothesis can be performed by executing a RANSAC algorithm. In order to determine the actual location of the map display 200 in the semantic road map 300 based on the created location hypotheses 313, a location hypothesis 315 based on the point cluster 312 with the highest point density is determined based on the point density of the individual clusters 312 of displaced keypoints. In this case, the point density of each cluster 312 corresponds to the number of displaced keypoints of the characteristic elements 301 that respectively match the characteristic elements 201 of the map representation 200. Thus, the point cluster 312 with the highest point density and the location hypothesis 315 based thereon corresponds to the area 303 in the semantic road map in which the largest number of characteristic elements 301 that match the characteristic elements 201 of the map representation 200 are located. Thus, the location hypothesis 315 thus determined represents the correct location of the map representation in the semantic road map 300 with the greatest probability.

[0047] Instead, the actual location of the map view 200 within the semantic road map 300 can be determined based on a number of location hypotheses 313, by converting the map view 200 into the semantic road map 300 for each location hypothesis and testing the map view 200 thus converted for a match with the respective areas 303 of the location hypotheses 313 of the semantic road map 300.

[0048] For the determination of the position of the map display 200 in the semantic road map 300, all characteristic elements 201 of the map display 200 and all characteristic elements 301 of the semantic road map may be taken into account. In order to narrow down the number of characteristic elements 301 to be considered in the semantic road map 300, a pre-positioning of the map display 200 in the semantic road map 300 may be performed. In this regard, based on the geographical position of the surroundings 501 represented by the surroundings sensor data of the vehicle 500 and depicted in the map display 300, a sub-area 317 of the semantic road map 300 including the geographical position of the surroundings 501 may be determined. The geographical position may be determined, for example, based on data of the satellite positioning system GNSS indicating the geographical position of the vehicle 501 at the time when the surroundings sensor data on the basis of which the map display 200 is generated is received. Due to the consideration of the geographical position, the position of the map display 200 in the semantic road map 300 may be pre-positioned to an accuracy of about 50 meters. That is, in order to precisely determine the position of the map display 200 within the road map 300 by comparing the characteristic elements 201, 301, with respect to the semantic road map 300, only those characteristic elements 301 that are located within the partial area 317 of the semantic road map 300 determined by pre-positioning are taken into account.

[0049] FIG. 3 shows a schematic diagram of a descriptor 400 . The depicted descriptor 400 is formed as a neighbor binary landmark descriptor and includes a grid structure 401 with a number of spatial regions 403. The grid structure 401 is formed in the shape of a cylinder with a rounded base in the illustrated embodiment. The spatial regions 403 are arranged in the radial and azimuth directions as well as in the vertical direction of the cylindrical grid structure 401. The three-dimensional formation of the descriptor 400 allows for a three-dimensional consideration of the characteristic elements 201, 301 of the three-dimensional map representation 200 or the three-dimensional semantic road map 300.

[0050] FIG. 4 shows a schematic diagram of a descriptor 400 applied to a semantic road map 300 or map display 200 . The plan view shown shows a descriptor 400 according to the embodiment of Fig. 3 positioned on a roadway 203, 304 with the corresponding roadway boundary lines 205, 305 and road markings 207, 307. To calculate the descriptor 400, values ​​of properties of the considered characteristic elements 201, 301 are calculated for each spatial region 403 of the grid structure 401. In the shown embodiment, the roadway boundary lines 205, 305 and road markings 207, 307 are considered as characteristic elements 201, 301. Since these characteristic elements 201, 301 each have a linear shape, the length L that each characteristic element 201, 301 takes up in the considered spatial region 403, 404, respectively, is taken into account as the property of the considered characteristic elements 201, 301.

[0051] As mentioned above, the descriptors can be calculated separately for the various semantic classes of the characteristic elements 201, 301. In the illustrated example, the descriptor 400 can thus be calculated only for the road markings 207, 307 or only for the road boundary lines 205, 305. The consideration of the semantic classes of the characteristic elements 201, 301 in the calculation of the descriptor 400 is indicated by the different hatching of the various spatial regions 403, 404. To calculate the descriptor 400, the spatial regions 403, 404 are compared with each other, in particular the values ​​taken by the properties of the characteristic elements 201, 301 considered in the spatial regions 403, 404 with each other. The spatial regions 404 having a value of the considered property of the characteristic elements 201, 301 that is greater than the value of the spatial regions 403 directly adjacent to the respective spatial region 404 are assigned the numerical value 1. In contrast, each further spatial region 403 having a value of the property of the characteristic element 201, 301 that is not greater than the value of the respective directly adjacent spatial region 403 is assigned the numerical value 0. The descriptor 400 thus determined can therefore be displayed in a binary vector representation.

[0052] FIG. 5 shows a flow diagram of a method 100 for positioning a map representation 200 of a vehicle's 500 surroundings 501 within a semantic road map 300 . To position the map representation 200 of the surroundings 501 of the vehicle 500 within the semantic road map 300, first in process step 101, surroundings sensor data of at least one surroundings sensor 503 of the vehicle 500 is recorded, where the surroundings sensor data describes the surroundings 501 of the vehicle 500. The surroundings sensor data may include, for example, camera data, lidar data, or radar data.

[0053] In a further process step 103, a map representation 200 of the surroundings 501 of the vehicle 500 is created based on the received surroundings sensor data. The map representation 200 may in this case be formed as a digital three-dimensional map of the surroundings 501 of the vehicle 500 and may depict the surroundings 501 of the vehicle 500 within the reach of at least one surroundings sensor 503.

[0054] In a further process step 139, a pre-positioning of the map representation 200 with respect to the semantic road map 300 is performed, which in this case is based on the geographical position of the surroundings 501 of the vehicle 500 displayed by the map representation 200. The geographical position may be based on data of the satellite positioning system GNSS received at the time the surroundings sensor data on which the map representation 200 is based are received.

[0055] In a further process step 141, a sub-area 317 is determined based on the preliminary positioning of the map view 200 within the semantic road map 300. The sub-area 317 comprises the geographic location of the map view 200.

[0056] In a further process step 105, characteristic elements 201, 301 are determined in the map view 200 and in the semantic road map 300, respectively. The characteristic elements 201, 301 may be caused by objects in the surrounding environment 501 depicted in the map view 200 or the semantic road map 300. The characteristic elements 201, 301 may be caused by, for example, road boundary lines 205, 305, road markings 207, 307, street lights 209, 309 or traffic lights 311.

[0057] In a further process step 107 , the characteristic elements 201 of the map representation 200 are compared with the characteristic elements 301 of the semantic road map 300 . For this purpose, in a further process step 113 , a descriptor 400 is determined for each characteristic element 201 node of the map representation 200 and for each characteristic element 301 node in the sub-area 317 of the semantic road map 300 .

[0058] To this end, in process step 117, a lattice structure 401 is defined that has a number of spatial regions 403 for each node. In a further process step 119, for each spatial region 403 of the lattice structure 401, a value of a property of the respective considered characteristic element 201, 301 is determined. The considered property of the characteristic element 201, 301 can be for example the point density of the point-like characteristic elements 201, 301, the length of the line-like characteristic elements 201, 301, the area of ​​the surface-like characteristic elements or the volume of the three-dimensional characteristic elements 201, 301.

[0059] In a further process step 121, the values ​​of the properties of the characteristic elements 201, 301 of the individual spatial regions 403 of the lattice structure 401 of the descriptor 400 are compared with the values ​​taken by the respective properties in directly adjacent spatial regions 403 of the lattice structure 401 of the descriptor 400.

[0060] Based thereon, in a further process step 123, the numerical value 1 is assigned to each spatial region 403, 404 in which the respectively considered property takes a value greater than the value of the property in the directly adjacent spatial region 403, 404, while the numerical value 0 is assigned to spatial regions 403, 404 in which the property has a value not greater than the value of the property in the directly adjacent spatial region. The descriptor 400 can thus be represented in a binary vector representation.

[0061] In a further process step 115, the thus determined descriptors 400 of the various characteristic elements 201, 203 are compared with one another. In this case, the comparison of the descriptors 400 of the various characteristic elements 201, 301 can be performed in such a way that only the descriptors of the characteristic elements 201, 301 that are assigned to the same semantic class are compared. Thus, for example, only the descriptors 400 of the roadway boundaries 205, 305 can be compared with one another. As mentioned above, for each semantic class, one common averaged descriptor can be used for all characteristic elements of that semantic class.

[0062] In order to compare the descriptors 400 of multiple characteristic elements 201, 301, in a further process step 125 a nearest neighbour search of the descriptors 400 in the k-dimension tree is performed. In a further process step 109, an area 303 is determined within the sub-area 317 of the semantic road map 300, in which at least one characteristic element 301 is located that matches at least one characteristic element 201 of the map display 200.

[0063] To determine the area 303 , in a further process step 127 location hypotheses 313 are made as possible locations of the map view 200 within the semantic road map 300 .

[0064] For this purpose, in process step 129, key points of the characteristic elements 301 of the semantic road map 300 whose descriptors 400 match the descriptors 400 of the characteristic elements 201 of the map display 200 are displaced in the semantic road map 300 by a distance corresponding to the distance by which the respective characteristic elements 201 of the map display 200 are moved away from the center of the map display. This displacement of the key points of the characteristic elements 301 of the semantic road map 300 results in the formation of clusters 312 of nodes of the characteristic elements 301 within the sub-area 317 of the semantic road map 300.

[0065] In a further process step 131 , the clusters 312 thus formed are identified as location hypotheses 313 for the map representation 200 within the semantic road map 300 . In a further process step 133, the cluster 312 of displaced keypoints having the highest point density of keypoints within the cluster 312 is determined, and this respective cluster 312 is identified as the location hypothesis 313 having the highest probability of the actual location of the map display 200 within the semantic road map 300.

[0066] In a further process step 135, the location hypotheses 313 are geometrically validated by running a RANSAC algorithm. In a further process step 137, the location hypothesis 315 with the most inlier values ​​is identified as location P of the map display 200, where the inlier value of the location hypothesis 315 corresponds to the number of characteristic elements 301 that correspond to characteristic elements 201 of the map display 200 within the area 303 defined by the location hypothesis 315.

[0067] Following this, in a further process step 111, the location P of the area 303 in the semantic road map 300 that corresponds to the location hypothesis 315 having the most matches of the characteristic element 301 is identified as the location of the map display 200 in the semantic road map 300.

[0068] In a further process step 143 feature associations are further formed between features of the feature elements 301 of the semantic road map 300 and features of the feature elements 201 of the map representation 200 that match the feature elements 301 of the semantic road map 300, respectively.

[0069] FIG. 6 shows a flow diagram of a method 600 for supplementing a semantic road map 300 with information of a map representation 200 of a vehicle's 500 surroundings 501 . In order to supplement the semantic road map 300 with information of the map display 200 of the surrounding environment 501 of the vehicle 500, first in process step 601, the method 100 according to the present invention for positioning the map display 200 of the surrounding environment 501 of the vehicle 500 within the semantic road map 300 based on the above-mentioned embodiment is executed.

[0070] Then, in process step 603 , the information from the map view 200 is incorporated into the defined area 303 of the semantic road map 300 . FIG. 7 shows a schematic diagram of a computer program product 700 comprising instructions that, when executed by a computing unit, cause the program to perform the method 100 for positioning a map representation 200 of a surrounding environment 501 of a vehicle 500 within a semantic road map 300 and / or the method 600 for supplementing the semantic road map 300 with information of the map representation 200 of a surrounding environment 501 of a vehicle 500.

[0071] The computer program product 700 is in the shown embodiment stored on a memory medium 701. The memory medium 701 in this case can be any memory medium known from the state of the art.

Claims

1. A computer-implemented method (100) for positioning a map representation (200) of a surrounding environment (501) of a vehicle (500) within a semantic road map (300), comprising: - receiving (101) ambient environment sensor data (505) of at least one ambient environment sensor (503) of a vehicle (500); - creating (103) a map representation (200) of the surrounding environment (501) of said vehicle (500) based on said surrounding environment sensor data (505) of said vehicle (500); - determining (105) a feature element (201) in said map representation (200) and a feature element (301) in a semantic road map (300) depicting said surroundings (501); a step (107) of comparing said characteristic elements (201) of said map representation (200) with said characteristic elements (301) of said semantic road map (300); - determining (109) an area (303) of said semantic road map (300) for which at least one characteristic element (301) of said area (303) of said semantic road map (300) coincides with at least one characteristic element (201) of said map representation (200) and said map representation (200) and said semantic road map (300) depict the same area of ​​said surroundings (501) of said vehicle (500); - identifying (111) the location (P) of said area (303) in said semantic road map (300) as the location of said map representation (200) in said semantic road map (300); A computer implemented method (100) comprising: The step (107) of comparing the characteristic elements (201, 301) - determining (113) a descriptor (400) for the key points of each characteristic element (301) of said semantic road map (300) and a descriptor (400) for the key points of each characteristic element (201) of said map representation (200); a step (115) of comparing the descriptors (400) of the keypoints of the characteristic elements (301) of the semantic road map (300) with the descriptors (400) of the keypoints of the characteristic elements (201) of the map representation (200), in which case characteristic elements (201, 301) match if the descriptors (400) of the keypoints of the respective elements (201, 301) match, A step (113) of determining a descriptor (400) of a characteristic element (201, 301) - determining (117) a geometric grid structure (401) of said descriptors (400) at the locations of said keypoints of each of said characteristic elements (201, 301) in said map representation (200) or said semantic road map (300), said geometric grid structure (401) comprising a number of adjacent spatial regions (403); - determining (119) for each spatial region (403) of said geometric lattice structure (401) the nature of each of said characteristic elements (201, 301); a step (121) of comparing the value of said property in one spatial region (403) with the value of said property for the spatial regions (404) directly adjacent to said spatial region (403) of said geometric grid structure (401); a step (123) of assigning to spatial regions (403) of said geometric grid structure (401) the value 1 if the value of said property in each said spatial region (403) is greater than the value of said property in a directly adjacent spatial region (404) and of assigning to a spatial region (403) the value 0 if the value of said property in said spatial region (403) is not greater than the value of said property in said directly adjacent spatial region (404), A computer implemented method (100).

2. The step (121) of comparing the descriptors (400) comprises: - performing (125) a nearest neighbor search of said descriptor (400) in a k-dimension tree, The computer implemented method (100) of claim 1.

3. 2. The computer-implemented method of claim 1, wherein descriptors (400) for characteristic elements (201, 301) of different semantic classes are determined and only descriptors (400) for characteristic elements (201, 301) of the same semantic class are compared with each other.

4. 4. The computer-implemented method (100) of claim 3, wherein the semantic classes of the characteristic elements (201, 301) include road markings (207, 307), road boundary lines (205, 305), street lights (209, 309), traffic lights (311), and traffic signs.

5. 2. The computer implemented method (100) of claim 1, wherein the property is one from the following list: point density of points in the spatial domain (403, 404), length (L) of characteristic element (201, 301) displayed as a line in the spatial domain (403, 404), area of ​​characteristic element (201, 301) displayed as a surface in the spatial domain (403, 404), volume of characteristic element (201, 301) displayed as a volume in the spatial domain (403, 404).

6. The step of determining (109) the area (303) comprises: - generating (127) a number of location hypotheses (313) for a possible location (P) of said map representation (200) within said semantic road map (300), said location hypothesis generating step (127) comprising: performing (129) a position translation, in the semantic road map (300), of the keypoints of the characteristic elements (301) in the semantic road map (300) whose descriptors (400) match the descriptors (400) of the characteristic elements (201) of the map representation (200), by a distance corresponding to the distance of the respective corresponding characteristic elements (201) of the map representation (200) in the map representation (200) to the center point of the map representation (200); - identifying (131) the locations of clusters as location hypotheses (313) of said map representation (200); a step (133) of validating said location hypothesis (313) as a location (P) of said area (303), said step of validating comprising: determining (135) the number of keypoints displaced to the centers of said clusters (312) and identifying the cluster (312) with the largest number of keypoints displaced to the centers of said clusters (312) as the location hypothesis (315) for said area (305) having the greatest match with said map representation (200); or - depicting (137) said map representation (200) in said semantic road map (300) based on said location hypotheses (313), comparing said characteristic elements (301) at the location of said location hypotheses (313) with said characteristic elements (201) of said map representation (200) depicted at the location of said location hypotheses (313), and identifying said location hypotheses (315) with the greatest number of matching characteristic elements (201, 301), The computer implemented method (100) of claim 1.

7. 7. The computer-implemented method (100) of claim 6, wherein the comparison of the characteristic element (301) at the location of the location hypothesis (313) with the characteristic element (201) of the map representation (200) depicted at the location of the location hypothesis (313) comprises execution of a RANSAC algorithm.

8. moreover, - performing (139) a pre-positioning of said map representation (200) relative to said semantic road map (300) based on the geographical position of said surroundings (501) of said vehicle (500) displayed by said map representation (200); - determining (141) a sub-area (317) of the semantic road map (300) based on the determined geographical location, in which the characteristic elements (301) of the semantic road map (300) within the sub-area (317) are determined, The computer implemented method (100) of claim 1.

9. moreover, forming (143) feature associations between features of the feature elements (301) of the semantic road map (300) and features of the feature elements (201) of the map representation (200) for which there is a respective one match of each of the descriptors (400), The computer implemented method (100) of claim 1.

10. A method (600) for supplementing a semantic road map (300) with information of a map representation (200) of a surrounding environment (501) of a vehicle (500), comprising: - performing (601) a computer-implemented method (100) for positioning (200) a map representation (200) of an environment (501) of a vehicle (500) in a semantic road map (300) according to claim 1; - incorporating (603) information of said map representation (200) into said defined area (303) of said semantic road map (300); A computer implemented method (600) comprising:

11. A computing unit (505) adapted to carry out the method according to any one of the preceding claims.

12. A program (700) for carrying out the method according to any one of claims 1 to 10.

Images