Map element association method, computer-readable storage medium and intelligent device

Abstract

A map element association method, a computer-readable storage medium, and an intelligent device. The method comprises: on the basis of a first data source of a road scene, acquiring attribute information of a first map element and, on the basis of a second data source of the road scene, acquiring attribute information of a second map element, the attribute information comprising semantic information and / or geometric information (S11); on the basis of the semantic information and / or the geometric information, calculating an attribute error of the first map element and the second map element (S12); and, according to the attribute error, determining an association relationship between the first map element and the second map element (S13). According to the attribute error obtained on the basis of the semantic information and / or the geometric information, the method associates the map elements in different data sources, thus helping to improve the accuracy of association, and effectively avoiding the situation of false association.

Description

Map element association method, computer-readable storage medium, and intelligent device

[0001] This application claims priority to Chinese patent application No. 202311737691.9 filed on December 14, 2023, entitled “A method for associating map elements, computer-readable storage medium and intelligent device”. The entire contents of the above Chinese patent application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of image processing technology, and specifically provides a map element association method, a computer-readable storage medium, and an intelligent device. Background Art

[0003] Current advanced driver assistance solutions rely heavily on HD maps (High Definition Maps). However, errors are inevitably introduced during the HD map creation process, and untimely HD map updates can also lead to issues with freshness, which can affect vehicle safety during driving. Furthermore, this can compromise the effectiveness of assisted driving, often requiring multiple manual interventions and resulting in a poor user experience. To ensure safety and enhance the user experience, related technologies are integrating multi-source data to compensate for map errors. For example, perception results and map data are supplemented to construct information about the local driving environment.

[0004] Existing systems for fusing multi-source data typically require a data association module to obtain association information between different data sources and perform subsequent fusion optimization based on this association information. Related technologies typically obtain association information between different data sources based on the data's ID (identity document) attributes. However, this method suffers from poor accuracy and is prone to misassociation, which in turn affects the accuracy of constructing local driving environment information.

[0005] Summary of the Invention

[0006] This application aims to solve the above technical problem, that is, to solve the problem of poor accuracy of existing data association methods.

[0007] In a first aspect, the present application provides a map element association method, which includes:

[0008] Acquiring attribute information of a first map element based on a first data source of a road scene; and acquiring attribute information of a second map element based on a second data source of the road scene; the attribute information including semantic information and / or geometric information;

[0009] Calculating an attribute error between the first map element and the second map element based on the semantic information and / or geometric information;

[0010] An association relationship between the first map element and the second map element is determined according to the attribute error.

[0011] In some embodiments, calculating the attribute error between the first map element and the second map element based on the geometric information includes:

[0012] Calculating a Euclidean distance based on the geometric information of the first map element and the geometric information of the second map element;

[0013] The attribute error between the first map element and the second map element is determined based on the Euclidean distance.

[0014] In some embodiments, determining the association relationship between the first map element and the second map element based on the attribute error includes:

[0015] According to the attribute error, a Hungarian matching algorithm is used to determine an association relationship between the first map element and the second map element.

[0016] In some embodiments, before calculating the attribute error between the first map element and the second map element based on the semantic information and / or geometric information, the method further includes:

[0017] A preliminary association relationship between the first map element and the second map element is obtained.

[0018] In some embodiments, after obtaining the preliminary association relationship between the first map element and the second map element, the method further includes:

[0019] Verifying the preliminary association relationship;

[0020] When the verification result is that the association fails, the attribute error between the first map element and the second map element is calculated based on the semantic information and / or geometric information.

[0021] In some embodiments, verifying the preliminary association relationship includes:

[0022] Obtaining the type of the first map element or the second map element;

[0023] When the type is a target type, similarity is calculated based on semantic information of the first map element and the second map element, and the preliminary association relationship is verified according to the similarity.

[0024] In some embodiments, obtaining attribute information of a first map element based on a first data source of a road scene includes:

[0025] Based on the map data of the road scene, attribute information of the first map element is obtained.

[0026] In some embodiments, obtaining attribute information of a second map element based on a second data source of the road scene includes:

[0027] Based on the predicted data of the road scene, attribute information of the second map element is obtained.

[0028] In a second aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned map element association methods.

[0029] In a third aspect, the present application provides a smart device comprising:

[0030] at least one processor;

[0031] and, a memory communicatively coupled to the at least one processor;

[0032] The memory stores a computer program, and when the computer program is executed by the at least one processor, any one of the above-mentioned map element association methods is implemented.

[0033] By adopting the above technical solution, the present application can obtain attribute information of a first map element from a first data source based on a road scene; and obtain attribute information of a second map element from a second data source based on the road scene; the attribute information includes semantic information and / or geometric information; the attribute error between the first map element and the second map element is calculated based on the semantic information and / or geometric information, and the association relationship between the first map element and the second map element is determined based on the attribute error. This method combines the attribute error obtained based on the semantic information and / or geometric information to associate map elements from different data sources, which is conducive to improving the accuracy of the association and effectively avoiding misassociation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The preferred embodiments of the present application are described below with reference to the accompanying drawings, in which:

[0035] FIG1 is a flow chart showing the main steps of a method for associating map elements provided in an embodiment of the present application;

[0036] FIG2 is a schematic diagram of the structure of a map element association system provided in an embodiment of the present application;

[0037] FIG3 is a flowchart of a method for associating map elements provided by another embodiment of the present application;

[0038] Figure 4 is a schematic diagram showing the correlation between the lane lines in the HD map data and the predicted data at time i-1;

[0039] Figure 5 is a schematic diagram showing the preliminary correlation between the lane lines in the HD map data and the predicted data at time i;

[0040] FIG6 is a schematic diagram of lane line association after deleting the incorrect association relationship provided by the present application;

[0041] FIG7 is a schematic diagram of the final correlation relationship between the high-precision map data and the lane lines in the predicted data provided by this application;

[0042] FIG8 is a schematic diagram of the structure of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the scope of protection of the present application.

[0044] Referring to FIG. 1 , FIG. 1 is a flow chart showing the main steps of a map element association method provided in an embodiment of the present application, which may include:

[0045] Step S11: acquiring attribute information of a first map element based on a first data source of a road scene; and acquiring attribute information of a second map element based on a second data source of the road scene; the attribute information includes semantic information and / or geometric information;

[0046] Step S12: calculating the attribute error between the first map element and the second map element based on the semantic information and / or geometric information;

[0047] Step S13: determining the association relationship between the first map element and the second map element according to the attribute error.

[0048] In an embodiment of the present application, a road scene may include multiple map elements. Both a first data source and a second data source may include attribute information of multiple map elements in the road scene. Road elements in the first data source are represented by first road elements, and road elements in the second data source are represented by second road elements. When the first road element and the second road element are associated, it can be determined that the first road element and the second road element correspond to the same real map element in the road scene. The first data source and the second data source are different and are obtained using different methods.

[0049] In some embodiments, map elements may include at least one of lane lines, road signs, and traffic lights; in other embodiments, map element attributes may also include other elements in the road scene. In some embodiments, the first data source may be map data, which may be directly acquired. For example, the first data source may be high-precision map data. The first data source may include attribute information of each map element in the current road scene, such as the location information or color of the map element. The second data source may be predicted data, which may be data predicted based on historical data. For example, the second data source may be data obtained using a Kalman filter algorithm based on historical data, such as location information. The second data source may include predicted attribute information of each map element in the current road scene.

[0050] In some embodiments, in step S11, obtaining attribute information of the first map element based on the first data source of the road scene may specifically be:

[0051] Based on the map data of the road scene, attribute information of the first map element is obtained.

[0052] In some embodiments, in step S11, obtaining the attribute information of the second map element based on the second data source of the road scene may specifically be:

[0053] Based on the predicted data of the road scene, attribute information of the second map element is obtained.

[0054] There may be multiple first map elements and multiple second map elements.

[0055] In an embodiment of the present application, attribute information may include semantic information and / or geometric information. The semantic information may include the type and / or color of the map element, and the geometric information may include at least one of the shape, area, position, and contour curvature of the map element.

[0056] It should be noted that the attribute information of the first map element and the second map element can be obtained simultaneously or successively, and the acquisition order is not particularly limited in the embodiment of the present application.

[0057] In some embodiments, step S12 may specifically include:

[0058] calculating a semantic error between the first map element and the second map element based on the semantic information;

[0059] and / or,

[0060] A geometric error between the first map element and the second map element is calculated based on the geometric information, and a semantic error and / or a geometric error is used as an attribute error.

[0061] In some embodiments, semantic information can be represented as a vector and a semantic model, such as a semantic parser or knowledge graph, can be used to calculate the semantic similarity between two pieces of semantic information; and the semantic error can then be determined based on the semantic similarity. In other embodiments, semantic similarity can also be determined using other methods known in the art, such as using Euclidean distance or Manhattan distance. In some embodiments, semantic similarity can be expressed as a percentage, and the semantic error can be obtained by calculating the difference between 1 and the semantic similarity.

[0062] In some embodiments, calculating the geometric error between the first and second map elements based on geometric information may include calculating a Euclidean distance based on the geometric information of the first and second map elements; determining the geometric similarity between the first and second map elements based on the Euclidean distance; and determining the geometric error based on the geometric similarity. The geometric similarity can be expressed as a percentage, and the geometric error can be obtained by calculating the difference between 1 and the geometric similarity. It should be noted that in other embodiments, other methods in the art may also be used to determine the geometric similarity.

[0063] In some embodiments, step S13 may specifically include:

[0064] According to the attribute error, the Hungarian matching algorithm is used to determine the association relationship between the first map element and the second map element.

[0065] In some embodiments, determining the association relationship between the first map element and the second map element using the Hungarian matching algorithm based on the attribute error may include:

[0066] Each first map element in the first data source is treated as a pair to be matched with each second map element in the second data source. An attribute error matrix is ​​constructed based on the attribute error of each pair to be matched. The attribute error matrix can be a two-dimensional matrix, where each element in the two-dimensional matrix corresponds to the attribute error of each pair to be matched, and different columns of the two-dimensional matrix can correspond to different pairs to be matched.

[0067] The attribute errors are converted into weight values, and the Hungarian matching algorithm is used to find the maximum number of matches in the attribute error matrix. Specifically, a maximum weight value and the corresponding matching pair are initialized. Each element in the attribute error matrix (i.e., each matching pair) is traversed. For each matching pair, its weight value is calculated, which can be defined as the negative value or inverse of the attribute error. If the weight value of the current matching pair is greater than the maximum weight value, the maximum weight value and the corresponding matching pair are updated. Repeat these steps until all matching pairs are traversed and the maximum number of matches is achieved. A larger number of matches indicates a smaller attribute error between the matching pairs and a better matching effect.

[0068] When the maximum number of matches is achieved, the matching pair corresponding to the maximum weight value, that is, the minimum attribute error, can be obtained and the matching pair is taken as the optimal matching pair; and the association between the first map element and the second map element corresponding to the optimal matching pair is determined.

[0069] In some embodiments, a matching threshold may be set according to actual needs, and the attribute error of the optimal matching pair may be compared with the matching threshold. When the comparison result is less than 0.05, the first map element and the second map element corresponding to the optimal matching pair may be associated.

[0070] The above describes a map element association method provided in an embodiment of the present application. This method uses a first data source based on a road scene to obtain attribute information for a first map element; and a second data source based on the road scene to obtain attribute information for a second map element. The attribute information includes semantic information and / or geometric information. The attribute error between the first and second map elements is calculated based on the semantic and / or geometric information, and the association relationship between the first and second map elements is determined based on the attribute error. This method combines the attribute error obtained based on the semantic and / or geometric information to associate map elements from different data sources, which helps improve association accuracy and effectively avoids misassociations.

[0071] In some embodiments, the map element association method provided in the embodiments of the present application can also correct the associated map element pairs, as described below for details.

[0072] 2 and 3 , FIG2 is a schematic diagram of the structure of a map element association system provided in an embodiment of the present application, and FIG3 is a schematic diagram of the flow of a map element association method provided in another embodiment of the present application, which can be implemented based on the map element association system shown in FIG2 , including:

[0073] Step S31: obtaining a preliminary association relationship between the first map element and the second map element;

[0074] Step S32: verifying the preliminary association relationship;

[0075] When the verification result is that the association fails, execute steps S33-S35;

[0076] Step S33: acquiring attribute information of a first map element based on a first data source of the road scene; and acquiring attribute information of a second map element based on a second data source of the road scene; the attribute information includes semantic information and / or geometric information;

[0077] Step S34: Calculating the attribute error between the first map element and the second map element based on the semantic information and / or geometric information;

[0078] Step S35: determining the association relationship between the first map element and the second map element according to the attribute error.

[0079] Among them, steps S33-S35 can be implemented in the same manner as the above steps S11-S13. For the sake of brevity, they are not repeated here. For details, please refer to the description above.

[0080] The following description will be made based on the implementation of the map element association system shown in FIG2 as an example.

[0081] In some embodiments, step S31 may specifically include:

[0082] Obtaining an association relationship between a first map element in a first data source and a second map element in a second data source at a historical moment; as shown in FIG2 , ID information of the first map element and the second map element may also be obtained, and the association relationship may be represented by the ID information of the first map element and the second map element;

[0083] Obtaining an ID association relationship between a first map element in a first data source and a historical moment and a current moment; and obtaining an ID association relationship between a second map element in a second data source and a historical moment and a current moment;

[0084] Based on the above two steps, a preliminary association relationship between the first map element and the second map element at the current moment is determined.

[0085] The first data source can be observational data, for example, high-precision map data. Therefore, the IDs of the first map elements at the historical and current moments are the same, allowing for direct association. The second data source can be forecasted data. The current moment's data can be derived based on the data at the historical moment, thus also determining the association between the second map elements at the historical and current moments. Based on this, by combining the association between the first and second map elements obtained at the historical moment, the association between the first and second map elements at the current moment can be derived, i.e., a preliminary association.

[0086] In some embodiments, step S32 may specifically include:

[0087] Calculating similarity based on semantic information and / or geometric information of the first map element and the second map element;

[0088] Based on the similarity calculation results and the preset threshold, the preliminary association relationship is verified.

[0089] The similarity can be calculated by calculating the Euclidean distance. When the similarity calculation result is greater than a preset threshold, the verification result is an association failure. When the similarity calculation result is less than or equal to the preset threshold, the verification result is an association success, and the current association relationship between the first map element and the second map element can be saved. The successfully associated first map element and the second map element are determined to be an association pair. The preset threshold can be set as needed.

[0090] In some other embodiments, step S32 may specifically include:

[0091] Get the type of the first map element or the second map element;

[0092] When the type is a target type, similarity is calculated based on semantic information of the first map element and the second map element, and the preliminary association relationship is verified according to the similarity.

[0093] Among them, the target type can be pre-set according to needs. As an example, the target type can be a road fork. The map elements of this type have complex geometric shapes, and the verification accuracy through geometric information is relatively poor. Based on this, the similarity can be calculated based on semantic information to improve the verification accuracy and effectively identify the misassociation.

[0094] In some embodiments, similarity is calculated based on semantic information of the first map element and the second map element, and the preliminary association relationship is verified based on the similarity as follows:

[0095] A Euclidean distance is calculated based on semantic information of the first map element and the second map element, and similarity is obtained based on the Euclidean distance; the similarity is compared with a preset threshold value, and when the similarity is greater than the preset threshold value, the verification result is that the association fails; when the similarity is less than or equal to the preset threshold value, the verification result is that the association succeeds, the first map element and the second map element are determined to be an associated pair, and the current association relationship between the first map element and the second map element can be saved.

[0096] In some embodiments, when the verification result is that the association fails, the preliminary association relationship between the current first map element and the second map element may be deleted and used as the first map element to be associated and the second map element to be associated, and the association relationship between the first map element to be associated and the second map element to be associated may be re-determined using steps S33-S35.

[0097] 4 to 7 , which are schematic diagrams of association at various stages of a map element association method provided in a specific embodiment of the present application, wherein the map element may be a lane line, the first data source may be high-precision map data, and the second data source may be predicted data. Lane lines L1, L2, L3, and L4 and the ID information of each lane line may be obtained from the high-precision map data, and the ID information of L1, L2, L3, and L4 are 1, 2, 3, and 4, respectively. Lane lines L1, L2, L3, and L4 are represented by black solid lines in the figure; lane lines m1, m2, m3, and m4 and the ID information of each lane line may be obtained from the predicted data, and the ID information of m1, m2, m3, and m4 are 101, 102, 103, and 104, respectively. Lane lines m1, m2, m3, and m4 are represented by black dotted lines in the figure, and the curves with arrows represent the association relationship. See Figure 4, which is a schematic diagram of the association between the lane lines in the HD map data and the predicted data at time i-1. i is a positive integer. Lane lines L1, L2, L3, and L4 are associated with lane lines m1, m2, m3, and m4 in sequence. The corresponding ID association relationship table is as follows:

[0098] Figure 5 shows the preliminary association diagram of lane lines in the HD map data and predicted data at time i. Lane lines L1, L2, L3, and L4 are associated with lane lines m2, m1, m3, and m4 in sequence. The corresponding ID association table is as follows:

[0099] Among them, through verification, it can be seen that the association relationship between lane lines L1, L2 and lane lines m1, m2 is wrong. The erroneous association information of lane lines L1, L2 and lane lines m1, m2 can be deleted, and lane lines L1, L2 and lane lines m1, m2 can be used as lane lines to be associated, as shown in Figure 6. Figure 6 is a schematic diagram of the association of lane lines after deleting the erroneous association relationship provided by this application.

[0100] The method of steps S33 to S35 of this application is executed for the lane lines to be associated, and the final association relationship between lane lines L1, L2 and lane lines m1, m2 is obtained based on the attribute information, as shown in Figure 7, which is a schematic diagram of the final association relationship between lane lines in the high-precision map data and predicted data provided by this application.

[0101] It should be noted that in the embodiment of the present application, the two lane lines that are successfully associated correspond to the same lane line in the real road scene. This is to reflect that the lane lines are obtained based on different data sources. For example, lane lines L1 and m1 show a larger interval between the two lane lines. In other examples, the two associated lane lines may also at least partially overlap.

[0102] The above is a map element association method provided in another embodiment of the present application, which can achieve the same beneficial effects as the corresponding embodiment of Figure 1. In addition, the association relationship of the associated map elements can be verified and corrected, which is conducive to improving the accuracy of the association.

[0103] It will be understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiments of the present application can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0104] In another aspect, the present application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the map element association method described in any of the above embodiments. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, in embodiments of the present application, the computer-readable storage medium is non-transitory.

[0105] Another aspect of the present application provides an intelligent device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, implements the map element association method described in any of the above embodiments.

[0106] The smart devices described in this application may include driving devices, smart cars, robots and other devices.

[0107] Referring to FIG. 8 , FIG. 8 exemplarily shows a structure in which a memory 81 and a processor 82 are connected via a bus, and only one memory 81 and only one processor 82 are provided.

[0108] In other embodiments, the smart device may include multiple memories 81 and multiple processors 82. The program for executing the map element association method of any of the above-described embodiments may be divided into multiple subroutines, each of which may be loaded and executed by the processor 82 to execute the map element association method or different steps of the map element association method of the above-described method embodiment. Specifically, each subroutine may be stored in a different memory 81, and each processor 82 may be configured to execute the programs in one or more memories 81 to jointly implement the map element association method of the above-described method embodiment.

[0109] In some embodiments of the present application, the smart device further includes at least one sensor configured to sense information. The at least one sensor is communicatively coupled to any of the processors described herein. Optionally, the smart device further includes an autonomous driving system configured to guide the smart device in autonomous or assisted driving. The at least one processor communicates with the at least one sensor and / or the autonomous driving system to implement the map element association method described in any of the aforementioned embodiments of the present application.

[0110] Thus far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present application.

Claims

1. A method for associating map elements, characterized in that, Including: Obtaining attribute information of a first map element based on a first data source of a road scenario; And obtaining attribute information of a second map element based on a second data source of the road scenario; The attribute information includes semantic information and / or geometric information; Calculating an attribute error between the first map element and the second map element based on the semantic information and / or geometric information; Determining an association relationship between the first map element and the second map element according to the attribute error.

2. The method according to claim 1, characterized in that, Calculating the attribute error between the first map element and the second map element based on geometric information includes: Calculating an Euclidean distance based on the geometric information of the first map element and the geometric information of the second map element; Determining the attribute error between the first map element and the second map element based on the Euclidean distance.

3. The method according to claim 1, characterized in that, The determining the association relationship between the first map element and the second map element according to the attribute error includes: Determining the association relationship between the first map element and the second map element by using a Hungarian matching algorithm according to the attribute error.

4. The method according to any one of claims 1 to 3, characterized in that, Before calculating the attribute error between the first map element and the second map element based on the semantic information and / or geometric information, the method further includes: Obtaining a preliminary association relationship between the first map element and the second map element.

5. The method according to claim 4, characterized in that, After obtaining the preliminary association relationship between the first map element and the second map element, the method further includes: Verifying the preliminary association relationship; When the verification result is an association failure, calculating the attribute error between the first map element and the second map element based on the semantic information and / or geometric information.

6. The method according to claim 5, characterized in that, The verifying the preliminary association relationship includes: Obtaining the type of the first map element or the second map element; When the type is a target type, calculating a similarity based on the semantic information of the first map element and the second map element, and verifying the preliminary association relationship according to the similarity.

7. The method according to claim 1, characterized in that, The obtaining attribute information of a first map element based on a first data source of a road scenario includes: Obtaining the attribute information of the first map element based on the map data of the road scenario.

8. The method according to claim 1, characterized in that, The obtaining attribute information of a second map element based on a second data source of the road scenario includes: Obtaining the attribute information of the second map element based on the prediction data of the road scenario.

9. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the map element association method according to any one of claims 1 to 8 is implemented.

10. An intelligent device, characterized in that, Including: At least one processor; And a memory communicatively connected to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the map element association method according to any one of claims 1 to 8 is implemented.

Images