Methods for training a classifier for map verification and methods for map verification
A classifier trained on environmental data efficiently verifies and corrects digital maps by determining input features and comparing them to ground truth labels, addressing inaccuracies in existing map generation systems.
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2024-11-18
- Publication Date
- 2026-05-21
AI Technical Summary
Existing digital map generation systems suffer from errors leading to inaccurate representations of reality, necessitating a method to detect and correct these errors efficiently without manual checks or additional data sources.
A classifier is trained using environmental datasets to determine input features for map sections, which are transformed into feature tensors, and compared with ground truth labels to assess map correctness, enabling efficient map verification and correction.
The method simplifies and enhances the quality assurance of digital maps by predicting the accuracy of map sections without additional data, reducing manual effort and improving correction processes.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for training a classifier for map verification, a method for map verification, a device, a computer program and a machine-readable storage medium. State of the art
[0002] The generation of a digital road map can, for example, be based on measurement data that is aggregated in various layers to create a (high-definition) map. It can be assumed, for instance, that the measurement data is already summarized and provided in a compactized format during acquisition, such as lane markings in vehicle-related coordinates instead of the original camera images. Errors can occur during map creation, leading to an inaccurate representation of reality. There is a need to avoid or detect such errors in order to correct them. Disclosure of the invention
[0003] The object underlying the invention is to provide a concept for training a classifier for map verification.
[0004] The object underlying the invention is also to provide a concept for map verification.
[0005] These problems are solved by means of the respective subject matter of the independent claims. Advantageous embodiments of the invention are the subject matter of dependent claims.
[0006] Following an initial aspect, a procedure for training a classifier for map verification is provided, comprising the following steps: receiving multiple environment datasets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through the same geographical area, Creating a digital map of the geographic area based on the environmental datasets, Dividing the digital map into several map sections, Determining input features for the multiple map sections based on the received environmental data sets, wherein the input features represent additional information for the multiple map sections, Determining a training dataset comprising multiple training data points based on the input features, wherein the training data points each comprise a pair of a feature tensor associated with a position of the digital map and a ground truth label, which is a measure of the correctness of a map section of the multiple map sections encompassing the corresponding position, Training the classifier based on the determined training dataset to obtain a trained classifier, wherein a training step of the training of the classifier includes that the classifier determines a measure of the correctness of the map section encompassing the training data point based on the feature tensor of the training data point, and compares the determined measure with the ground truth label of the training data point.
[0007] Following a second aspect, a procedure for map verification is provided, comprising the following steps: Receiving multiple environment data sets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through the same geographical area, Creating a digital map of the geographic area based on the environmental datasets, Dividing the digital map into several map sections, Determining input features for the multiple map sections based on the received environmental data sets, wherein the input features represent additional information for the multiple map sections, Determining a feature tensor assigned to a position on the digital map based on the input features, Determining one or more measures of the correctness of the map section of the digital map containing the corresponding position, based on the feature tensor, by a classifier in order to verify the digital map.
[0008] According to a third aspect, a device is provided which is set up to perform all steps of the procedure according to the first aspect and / or the second aspect.
[0009] According to a fourth aspect, a computer program is provided, comprising instructions which, when the computer program is executed by a computer, for example by the device according to the third aspect, cause it to execute a procedure according to the first aspect and / or according to the second aspect.
[0010] According to a fifth aspect, a machine-readable storage medium is provided on which the computer program is stored according to the fourth aspect.
[0011] The invention is based on the finding and includes the fact that the above problem is solved by determining input features for several map sections of the digital map, wherein these input features represent additional information for the several map sections. This additional information is used for training the classifier for map verification or for verifying the digital map.
[0012] This allows for efficient training and card verification without the need for manual checks, for example. In particular, there is no need to use any additional data beyond this supplementary information.
[0013] To train the classifier and verify the map, information and data derived from the mapping pipeline are used. For example, input features are determined based on the created digital map. This means that the input features are determined indirectly, for instance, based on the surrounding datasets. Alternatively, the input features can be created directly based on the surrounding datasets.
[0014] In other words, it may be intended, for example, that an input characteristic is determined based on the digital map, i.e., indirectly based on the environmental data sets. Alternatively, it may be intended that an input characteristic is determined directly based on the environmental data sets, i.e., immediately based on the environmental data sets.
[0015] The concept described here can thus advantageously complement a known system for generating digital maps with a methodology that predicts the accuracy of map sections or specific map features, such as lane markings or road geometry, based on features extracted from the mapping system—the input features. This can be achieved, in particular, without the need for additional data sources like separate video images. The prediction is performed by the classifier, which can be trained according to the concept described here. This simplifies and improves the quality assurance of the digital map through an enhanced correction process.
[0016] In one embodiment of the method according to the first aspect, it is provided that the respective feature tensor of the training data points is determined based on the input features assigned to the respective position.
[0017] This results, for example, in the technical advantage that the respective feature tensor can be determined efficiently.
[0018] In one embodiment of the method according to the first aspect, it is provided that the input features assigned to the respective position are each transformed into a tensor of fixed length in order to obtain the feature tensor.
[0019] This results, for example, in the technical advantage that the feature tensor can be efficiently obtained.
[0020] In one embodiment of the method according to the first aspect, it is provided that the respective Ground Truth Label is determined based on a comparison of the corresponding map section with a Ground Truth map of the geographical area.
[0021] This results, for example, in the technical advantage that the respective Ground Truth Label can be determined efficiently.
[0022] In one embodiment of the method according to the first aspect, it is provided that for the corresponding map section a completeness measure is determined based on the corresponding map section and the Ground Truth map, wherein the completeness measure indicates a measure of the completeness of the map contents of the map section with respect to the Ground Truth map, wherein the respective Ground Truth Label is determined based on the respective completeness measure.
[0023] This results, for example, in the technical advantage that the respective Ground Truth Label can be determined efficiently.
[0024] In one embodiment of the method according to the first aspect, it is provided that for the corresponding map section, a respective deviation from one or more positions of one or more map contents of the map section is determined with reference to the Ground Truth map, wherein the respective Ground Truth Label is determined based on the respective deviation(s) determined.
[0025] This results, for example, in the technical advantage that the respective Ground Truth Label can be determined efficiently.
[0026] In one embodiment of the method according to the first aspect, it is provided that for the corresponding map section a Jaccard index is determined between one or more map contents of the map section with reference to the Ground Truth map, wherein the respective Ground Truth Label is determined based on the respective Jaccard index determined.
[0027] This results, for example, in the technical advantage that the respective Ground Truth Label can be determined efficiently.
[0028] The Jaccard index, also known as the Jaccard coefficient, is a measure of the similarity of sets. It can also be described by its definition as IoU, which stands for "Intersection over Union".
[0029] In one embodiment of the method according to the first aspect, it is provided that the input features are each an element selected from the following group of input features: landmark of the digital map, additional attribute to a map feature of the digital map, number of environment data records used to determine the digital map at a location in the geographic area, metadata, in particular metadata of a source of an environment data record, error measure which indicates a consistency of the digital map compared with the environment data records used to create the digital map.
[0030] This results, for example, in the technical advantage that particularly suitable input features can be used.
[0031] An error measure is, for example, the chi-square error of an optimization of a large number of observations or data collections. Another error measure is, for example, the number of outliers (unusual observations) in the creation process.
[0032] In one embodiment of the method according to the first aspect, it is provided that the landmark is an element selected from the following group of landmarks: road marking, traffic signal system, traffic sign, upright structure.
[0033] This results, for example, in the technical advantage that particularly suitable landmarks can be provided.
[0034] An environment dataset includes, for example, an environment model and / or raw environmental sensor data and / or evaluated raw environmental sensor data and / or landmark(s).
[0035] An environmental sensor is, for example, an environmental sensor of the corresponding motor vehicle.
[0036] An environmental dataset is created, for example, based on the detection of a vehicle's surroundings by one or more of the vehicle's environmental sensors. Thus, for instance, a landmark was detected by one or more of the vehicle's environmental sensors, and through appropriate analysis of the raw sensor data, the landmark was identified in the vehicle's surroundings, so that the environmental dataset includes this information.
[0037] In one embodiment of the method according to the first aspect, it is provided that the additional attribute specifies what information is provided by the traffic sign, or wherein the additional attribute specifies a type of road marking.
[0038] This results, for example, in the technical advantage that particularly suitable additional attributes can be used.
[0039] In one embodiment of the method according to the first aspect, the metadata, in particular metadata of a source of an environment data set, includes one or more of the following information: vehicle type, vehicle speed, environment sensor modality and / or environment sensor version of an environment sensor of the corresponding vehicle used to create the corresponding environment data set, date of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, time of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, weather at the time of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, type,especially city or motorway, information, in particular a number of motor vehicles and / or pedestrians and / or cyclists and / or other road users, about the surroundings.
[0040] This results, for example, in the technical advantage that particularly suitable metadata can be used.
[0041] In one embodiment of the method according to the first aspect, it is provided that the Ground Truth Label and / or the measure of correctness determined by the classifier each indicate a class membership in two or more classes with regard to correctness.
[0042] Correctness encompasses several different aspects. Therefore, a measure of correctness can indicate which of these aspects are correct and, in particular, which are not. For example, a class might be complete (one aspect), but there might be, for instance, a positional inaccuracy (another aspect) and / or missing road markings (another aspect), especially lines.
[0043] This results, for example, in the technical advantage that a particularly suitable ground truth label or a particularly suitable classifier can be provided.
[0044] In one embodiment of the method according to the second aspect, it is provided that a confidence of the corresponding map section is determined as a measure, wherein the confidence is compared with a threshold value, and depending on the comparison, a binary truth value is determined as a measure indicating whether the corresponding map section is correct or incorrect.
[0045] This results, for example, in the technical advantage of determining a particularly suitable dimension.
[0046] In one embodiment of the method according to the second aspect, it is provided that a variance of a confidence of the corresponding map section is determined as a measure.
[0047] This results, for example, in the technical advantage of determining a particularly suitable dimension.
[0048] In one embodiment of the method according to the second aspect, it is provided that the classifier was or is trained according to the method according to the first aspect.
[0049] This results, for example, in the technical advantage that a particularly suitable classifier is used for the procedure according to the second aspect.
[0050] Features of the method according to the first aspect are derived analogously from features of the method according to the second aspect, and vice versa. Statements made in connection with the method according to the first aspect apply analogously to embodiments of the method according to the second aspect, and vice versa.
[0051] For example, the procedure according to the second aspect includes one or more steps of the procedure according to the first aspect, and vice versa.
[0052] The procedure according to the first aspect is, for example, a computer-implemented procedure.
[0053] The method according to the second aspect is, for example, a computer-implemented method.
[0054] The device is, for example, programmed to execute the computer program.
[0055] The embodiments and examples described here can be combined in any way, even if this is not explicitly described.
[0056] A tensor in the sense of the description is, for example, a vector or includes, for example, a vector.
[0057] The invention is explained in more detail below with reference to preferred embodiments. These include: Fig. 1. A flowchart of a procedure for training a classifier for map verification, Fig. 2 a flowchart of a procedure for map verification, Fig. 3 a device, Fig. 4 a machine-readable storage medium and Fig. 5 a block diagram.
[0058] Fig. Figure 1 shows a flowchart of a procedure for training a map verification classifier, comprising the following steps: Received 101 from multiple environment datasets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through a similar geographical area, Create 103 a digital map of the geographic area based on the environment datasets, Divide 105 of the digital map into several map sections, Determine 107 input features for the multiple map sections based on the received environment data sets, where the input features represent additional information for the multiple map sections. Determine 109 of a training dataset comprising multiple training data points based on the input features, wherein the training data points each comprise a pair of a feature tensor associated with a position of the digital map and a ground truth label, which is a measure of the correctness of a map section of the multiple map sections comprising the corresponding position, Training 111 of the classifier based on the determined training data set to obtain a trained classifier, wherein a training step of the training of the classifier includes that the classifier determines a measure of the correctness of the map section encompassing the training data point for a training data point based on the feature tensor of the training data point, 113 comparing the determined measure with the ground truth label of the training data point.
[0059] A training epoch comprises, for example, N such training steps, where N depends on the size of the training dataset. For example, N = number of samples / batch size. Multiple samples, i.e., training data points, are grouped together for a training dataset into a fixed batch size, and, for example, only one weight update is performed for the entire batch.
[0060] Fig. Figure 2 shows a flowchart of a map verification procedure, comprising the following steps: Received 201 from multiple environment datasets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through a same geographical area, Create 203 a digital map of the geographic area based on the environment datasets, Divide 205 of the digital map into several map sections, Determine 207 input features for the multiple map sections based on the received environment data sets, where the input features represent additional information for the multiple map sections, Determine 209 a feature tensor assigned to a position on the digital map based on the input features, Determine 211 one or more measures for the correctness of the map section of the digital map containing the corresponding position based on the feature tensor by a classifier in order to verify the digital map.
[0061] Fig. Figure 3 shows a device 301 which is set up to perform all steps of the method according to the first aspect and / or according to the second aspect.
[0062] For example, device 301 includes a communication device which is configured to receive multiple environmental data sets via a communication network.
[0063] For example, the device 301 includes a processor unit which is configured to perform the further steps of the method according to the first aspect and / or according to the second aspect, for example steps 103 to 115 and / or steps 203 to 211.
[0064] The processor setup includes, for example, one or more processors.
[0065] Device 301, for example, is implemented in a cloud infrastructure.
[0066] Fig. Figure 4 shows a machine-readable storage medium 401 on which a computer program 403 is stored. The computer program 403 comprises instructions that are executed by a computer, for example by the device 301 of the Fig. 3, induce them to carry out a procedure in accordance with the first aspect and / or the second aspect.
[0067] Fig.Figure 5 shows a block diagram 501, which illustrates the concept(s) described here by way of example.
[0068] Reference symbol 503 points to a map section of a digital map as described. Reference symbol 505 points to a corresponding reference map section of a ground truth map as described.
[0069] Map section 503 represents a road 507 comprising two lanes 509 and 511, separated by a dashed center line 513. Road 507 is laterally bounded by lane markings 515.
[0070] Furthermore, map section 503 represents elements, identified by the reference symbols 517, 519, 521, which can symbolically represent, for example, a vertical structure, i.e., an upright structure, a traffic signal system or a traffic sign.
[0071] Reference map section 505 represents a road 523 with two lanes 525 and 527, separated by a solid center line with reference sign 531. Lane boundaries with reference sign 529 define the sides of road 523.
[0072] Furthermore, elements with reference symbols 533 and 535 are shown, which are represented by reference map section 505. These elements could be, for example, a vertical (i.e., upright) structure, a traffic signal system, or a traffic sign.
[0073] A legend with reference number 536 is shown, which includes a solid arrow with reference number 537 and a dashed arrow with reference number 539.
[0074] The solid arrow with reference numeral 537 indicates the sequence of block diagram 501 in an embodiment of a method according to the second aspect, i.e., during inference. The dashed arrow with reference numeral 539 indicates a sequence of block diagram 501 in an embodiment of the method according to the first aspect, i.e., during training.
[0075] The concept(s) described here specifically involve training a classifier for map verification. Training data, which is determined according to this concept, is required for training the classifier. A data point in the training dataset, i.e., a training data point, consists in particular of a pair (X, y).
[0076] Here, X describes a generic data vector or tensor, the feature tensor in the sense of description, and y describes a corresponding ground truth label. For example, y describes a class membership, i.e., the correctness of the corresponding map segment. Thus, y can be defined, for example, as follows: y ∈ {0, 1}.
[0077] In block diagram 501, X is labelled as block with reference 541 and y is labelled as block with reference 543.
[0078] To determine the feature tensor, the surrounding data sets on which the digital map from which map section 503 was created are used directly or indirectly.
[0079] Input features can therefore include, for example, information from the created digital map itself, metadata, or intermediate results from the mapping pipeline. Such input features include, for example, one or more of the following characteristics or information: Map features (landmarks) such as road markings or geometry, traffic lights, road signs, upright structures such as posts / trees,
[0080] Additional attributes of the map features, such as not only information about the existence of a traffic sign at a location, but also the specific speed limit or the type of lane marking (solid or dashed),
[0081] Redundancy in the form of the number of raw data sets used for mapping a location (e.g., number of passes),
[0082] Metadata of the source, e.g. vehicle type, speed, sensor modality and version, date of acquisition,
[0083] Error measures that measure the consistency of the generated map compared to the raw data used, i.e., error measures that arise in the mapping pipeline.
[0084] The input features described above are transformed, for example, into a feature vector or feature tensor of fixed length, X. Scalars, for instance, can be directly imported as entries into a vector component or tensor component. Feature positions can be transformed into a vector or tensor, for example, by applying a fixed grid and then rolling it. Normalization of the input feature value ranges is also possible. For example, methods analogous to those used in Point Cloud Registration can be employed. Ground truth labels can be determined, for example, by comparing the generated map sections with a reference map, the so-called ground truth map, and / or by manual annotations.A ground truth map can be generated, for example, using dedicated fleets of survey vehicles, and additional data such as video images can be used, either separately or instead of traditional ground truth data. Crucially, the resulting map accurately reflects reality, and each map segment to be evaluated is assigned a ground truth label, such as a ground truth label y ∈ {0, 1}. This can involve using criteria such as: completeness of the map content, deviation in the position of individual map features, and the Jaccard index between map features in the reference map and the generated or created digital map.
[0085] Thus, X and y are determined and generated as described above. A training dataset is then compiled in such a way that feature tensors and corresponding ground truth labels are determined and generated for a sufficiently large number of map positions to advantageously cover a data distribution of the expected application data.
[0086] For training, for example, (deep) neural networks or SVMs (SVM: "Support Vector Machine") can be used and trained with standard loss functions known from the literature, such as cross-entropy. The training dataset created or obtained within the framework of the concept described here can, for example, be divided into a training and a validation part.
[0087] Once the classifier, identified in block diagram 501 by a block with the reference symbol 545, has been trained, it can be used for the map verification process. Using classifier 545, predictions can be made at new data points. For these new data points, it is ensured that the features X can be extracted from the mapping system. After training classifier 545, it can thus be advantageously used to evaluate the mapping results, i.e., the comparison results. For this purpose, classification is performed, for example, at the end of the mapping pipeline, and its input data X, i.e., the input features, are processed analogously to the setup of the training dataset. Ground truth labels are, of course, not required here.
[0088] The output of the map verification process is one or more measures of correctness. In block diagram 501, an output of the classifier, i.e., the measure or measures, is indicated by a block with the reference number 547.
[0089] Therefore, the following outputs can be provided: a confidence c normalized to [0, 1] a binary truth value, determined by applying a threshold to c, in an advantageous form further measures of the uncertainty or variance of c by standard procedures from the literature, such as ensemble formation.
[0090] In summary, the classifier predicts whether a digital (HD) map at a given location matches a reference or whether an error exists. For example, it only considers maps with a higher level of detail than navigation maps, such as those containing landmarks or lane-specific information. A particular advantage arises from the data used and the application of the classifier in the context of evaluating generated maps and with the goal of enabling a more reliable overall system using maps. The concept(s) described here thus complement an existing digital map-generating system by adding the methodology to predict the accuracy of map sections or specific map features (e.g., lane markings, road geometry, etc.) based on features extracted from the mapping system.In particular, this is possible without supplementing other data sources such as separate video images. This prediction is performed by a classifier. Thus, ensuring the quality of the map can be simplified and / or improved through an enhanced correction process.
[0091] According to the concepts described here, the use of specific features from the mapping pipeline or metadata of the underlying raw data, the input features, is particularly intended.
[0092] For example, the (machine learning) classifier is trained. Features, i.e., input characteristics, such as the content of the created digital map and the corresponding features extracted from the mapping system are used for this purpose. Ground truth is, for example, a statement about the correctness of a map section (binary label), which was determined, for instance, through manual labeling and / or automated comparison with a ground truth map previously created for specific regions of the world. A classifier (e.g., a neural network, SVM) can then be trained on the resulting dataset.
[0093] When applying the classifier, for example, a generalization to the entire map occurs, extending beyond locations with available training data. The classifier predicts the accuracy of map sections, individual map layers, or contained landmarks. The features of the examined map sections are transformed, for example, analogously to the training process, into feature vectors, specifically feature tensors or feature vectors. The result is, for example, a binary prediction determined by applying a threshold (e.g., 0.5), or the underlying confidence level between 0 and 1.
[0094] The advantage of the concepts described here lies particularly in the fact that the accuracy of a generated map or its sub-content can be checked and evaluated with significantly reduced (manual) effort and without the need for additional data. Map sections identified as erroneous can be excluded from use, corrected (e.g., through human review), or suggested for further data collection. Furthermore, confidence levels can be embedded in the map content to enable improved decision-making for functions accessing it.
[0095] The classification results can be used to evaluate the quality of the generated map. For example, it can be used to... in a downstream function that can decide based on confidence whether the map quality is sufficient, for example, to reduce the manual effort of map verification and correction (through direct use or as a guide in a manual review), for example, to evaluate and improve the mapping pipeline, through improved automatic analysis of error cases or detection of faulty input data, for example, to assess sufficient input data for the mapping pipeline and, if necessary, to collect additional data (of improved quality or scope) for areas with insufficient input data.
[0096] The concepts described here primarily involve the application of the principle of binary classification. This problem category deals with data where each data point can be assigned to exactly one of two possible classes. These two classes are defined by the correctness or incorrectness of the map in a given section. The goal of binary classification is to learn a classification rule, particularly using machine learning methods, that assigns each data point to one of the two classes. Summary
[0097] The invention relates to a method for training a classifier for map verification. Several environmental datasets, each representing a specific environment of motor vehicles during a journey through the same geographical area, are used to create a digital map of the geographical area. In particular, information and data resulting from the mapping pipeline are used to train the classifier and verify the map.
[0098] The invention relates to a method for card verification, a device, a computer program and a machine-readable storage medium.
Claims
[1] Method for training a classifier (545) for map verification, comprising the following steps: Receiving (101) multiple environment data sets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through a same geographical area, Creating (103) a digital map of the geographic area based on the environment datasets, Dividing (105) the digital map into several map sections (503), Determining (107) input features for the multiple map sections (503) based on the received environment data sets, wherein the input features represent additional information for the multiple map sections (503), Determining (109) a training data set comprising multiple training data points based on the input features, wherein the training data points each comprise a pair of a feature tensor (541) associated with a position of the digital map and a ground truth label (543), which is a measure (547) of the correctness of a map section (503) comprising the corresponding position of the multiple map sections (503), Training (111) of the classifier (545) based on the determined training data set to obtain a trained classifier (545), wherein a training step of the training of the classifier (545) comprises determining (113) by the classifier (545) for a training data point, based on the feature tensor (541) of the training data point, a measure (547) for the correctness of the map section (503) encompassing the training data point, wherein the determined measure (547) is compared (115) with the ground truth label (543) of the training data point. [2] Method according to claim 1, wherein the respective feature tensor (541) of the training data points is determined based on the input features assigned to the respective position. [3] Method according to claim 2, wherein the input features assigned to the respective position are each transformed into a tensor of fixed length to obtain the feature tensor (541). [4] Method according to one of the preceding claims, wherein the respective Ground Truth Label (543) is determined based on a comparison of the corresponding map section (503) with a Ground Truth map of the geographical area. [5] Method according to claim 4, wherein for the corresponding map section (503) a completeness measure is determined based on the corresponding map section (503) and the Ground Truth map, wherein the completeness measure indicates a measure for the completeness of the map contents of the map section (503) with respect to the Ground Truth map, wherein the respective Ground Truth Label (543) is determined based on the respective completeness measure. [6] Method according to claim 4 or 5, wherein for the corresponding map section (503) a respective deviation from one or more positions of one or more map contents of the map section (503) is determined with reference to the Ground Truth map, wherein the respective Ground Truth Label (543) is determined based on the respective deviation(s) determined. [7] Method according to any one of claims 4 to 6, wherein for the corresponding map section (503) a Jaccard index is determined between one or more map contents of the map section (503) with reference to the Ground Truth map, wherein the respective Ground Truth Label (543) is determined based on the respective determined Jaccard index. [8] Method according to any of the preceding claims, wherein the input features are each an element selected from the following group of input features: landmark of the digital map, additional attribute to a map feature of the digital map, number of environment data sets used to determine the digital map at a location in the geographic area, metadata, in particular metadata of a source of an environment data set, error measure which indicates a consistency of the digital map compared with the environment data sets used to create the digital map. [9] Method according to claim 8, wherein the landmark is an element selected from the following group of landmarks: road marking (513, 515), traffic signal system, traffic sign, upright structure. [10] Method according to claim 9, wherein the additional attribute specifies which information is provided by the traffic sign, or wherein the additional attribute specifies a type of road marking (513, 515), [11] Method according to any one of claims 8 to 10, wherein the metadata, in particular the metadata of a source of an environment data set, comprises one or more of the following information: vehicle type, vehicle speed, environment sensor modality and / or environment sensor version of an environment sensor of the corresponding vehicle used to create the corresponding environment data set, date of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, time of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, weather at the time of acquisition of the corresponding environment by one or more environment sensors of the corresponding vehicle to create the corresponding environment data set, type, in particular city or motorway,Information, in particular the number of motor vehicles and / or pedestrians and / or cyclists and / or other road users, about the surroundings. [12] Method according to any of the preceding claims, wherein the Ground Truth Label (543) and / or the measure (547) determined by the classifier (545) indicates a class membership in two or more classes with respect to correctness. [13] Methods for map verification, comprising the following steps: Receiving (201) multiple environment data sets, each representing a respective environment of motor vehicles during a respective journey of the motor vehicles through the same geographical area, Creating (203) a digital map of the geographic area based on the environment datasets, Dividing (205) the digital map into several map sections (503), Determining (207) input features for the multiple map sections (503) based on the received environment data sets, wherein the input features represent additional information for the multiple map sections (503), Determine (209) a feature tensor (541) associated with a position on the digital map based on the input features, Determine (211) one or more measures (547) for the correctness of the map section (503) containing the corresponding position of the digital map based on the feature tensor (541) by a classifier (545) in order to verify the digital map. [14] Method according to claim 13, wherein a confidence of the corresponding map section (503) is determined as a measure, wherein the confidence is compared with a threshold value, wherein, depending on the comparison, a binary truth value is determined as a measure which indicates whether the corresponding map section (503) is correct or incorrect. [15] Method according to claim 13 or according to claim 14, wherein a measure is determined as a variance of a confidence of the corresponding map section (503). [16] Method according to any one of claims 13 to 15, wherein the classifier (545) was trained according to the method according to any one of claims 1 to 12. [17] Device (301) which is configured to perform all steps of the method according to any of the preceding claims. [18] Computer program (403) comprising instructions which, when executed by a computer, cause the computer to execute a method according to any one of claims 1 to 16. [19] Machine-readable storage medium (401) on which the computer program (403) according to claim 18 is stored.