Semantic map-based vehicle localization methods, devices, and electronic equipment

Abstract

The present disclosure relates to a semantic map-based vehicle positioning method, a semantic map-based vehicle positioning device, a storage medium and an electronic device. The semantic map-based vehicle positioning method comprises obtaining an original map used by a vehicle during automatic driving and environment information of an environment in which the vehicle is currently driving; wherein the original map is a global map containing a starting point and a destination; constructing a local semantic map based on the environment information of the environment in which the vehicle is currently driving; determining a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving existing in the original map and the local semantic map; and performing location positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map, thereby achieving accurate positioning of the vehicle and enhancing robustness of the positioning system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, in particular to a vehicle positioning method based on a semantic map, a vehicle positioning device based on a semantic map, a storage medium and an electronic device. BACKGROUND

[0002] With the development of intelligent technology, cars begin to accelerate towards the direction of unmanned intelligence. Among them, the car automatic driving system is a kind of unmanned intelligent driving. The car automatic driving system, also known as automatic driving car, is also called unmanned car, computer driving car, or wheeled mobile robot, which is a kind of intelligent car system that realizes unmanned driving through vehicle computer system. But with the development of automatic driving technology, the dependence of automatic driving vehicles on positioning system is getting higher and higher, especially the high robustness of positioning system. This will affect the universality of automatic driving vehicle automatic driving application scenarios. SUMMARY

[0003] Therefore, the embodiments of the present disclosure expect to provide a vehicle positioning method based on a semantic map, a vehicle positioning device based on a semantic map, a storage medium and an electronic device.

[0004] The technical solution of the present disclosure is implemented as follows:

[0005] In a first aspect, the present disclosure provides a vehicle positioning method based on a semantic map.

[0006] The vehicle positioning method based on a semantic map provided by the embodiments of the present disclosure comprises:

[0007] Obtaining an original map used by a vehicle during automatic driving, and environment information of an environment in which the current driving state of the vehicle is located; wherein the original map is a global map containing a starting point and a destination;

[0008] Constructing a local semantic map based on the environment information of the environment in which the current driving state of the vehicle is located;

[0009] Determining a conversion relationship of the local semantic map corresponding to the original map based on the geometric relationship of the semantic elements in the environment in which the current driving state of the vehicle is located respectively existing in the original map and the local semantic map;

[0010] Based on the position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map, the position of the vehicle in the original map is positioned.

[0011] In some embodiments, the determining, based on geometric relations of semantic elements in an environment where the vehicle is currently driving, which respectively exist in the original map and the local semantic map, a transformation relation of the local semantic map corresponding to the original map, comprises:

[0012] determining, according to a global position trajectory of the vehicle in the original map and a trajectory of the local semantic map, a position error E of the local semantic map to the position trajectory of the original map;

[0013] selecting, in the original map and the local semantic map, semantic elements within the position error E;

[0014] selecting, based on the semantic elements within the position error E, a semantic matching pair for geometric relation matching, and performing geometric matching between the semantic matching pair;

[0015] performing, based on a correlation matrix and the semantic matching pair, correlation matrix solving to obtain a semantic matching pair set with a maximum matching degree;

[0016] determining, based on the semantic matching pair set, a transformation relation of the local semantic map corresponding to the original map.

[0017] In some embodiments, the performing, based on a correlation matrix and the semantic matching pair, correlation matrix solving to obtain a semantic matching pair set with a maximum matching degree, comprises:

[0018] determining, based on the semantic matching pair, the correlation matrix;

[0019] solving the correlation matrix to obtain the semantic matching pair set with the maximum matching degree based on solving a maximum clique problem of the correlation matrix.

[0020] In some embodiments, the determining, based on the semantic matching pair set, a transformation relation of the local semantic map corresponding to the original map, comprises:

[0021] determining, based on the semantic matching pair set, a centroid position of any one group of semantic matching pairs;

[0022] performing, based on the centroid position of the any one group of semantic matching pairs, Euclidean transformation to determine a de-centroid coordinate of each semantic matching pair;

[0023] determining, based on the de-centroid coordinate of each semantic matching pair, a rotation matrix;

[0024] determining, based on the centroid position of the any one group of semantic matching pairs and the rotation matrix, a transformation relation of the local semantic map corresponding to the original map.

[0025] In some embodiments, the semantic matching pair at least comprises one of:

[0026] The point-point matching corresponding to the semantic elements, the point-line matching corresponding to the semantic elements, and the line-line matching corresponding to the semantic elements.

[0027] In some embodiments, the geometric matching between the semantic matching pairs comprises:

[0028] Determining the geometric consistency of the point-point distance in the local semantic map and the point-point distance in the original map;

[0029] Determining the geometric consistency of the line-line angle in the local semantic map and the line-line angle in the original map;

[0030] Determining the geometric consistency of the point-line distance in the local semantic map and the point-line distance in the original map.

[0031] In a second aspect, the present disclosure provides a vehicle positioning device based on a semantic map, comprising:

[0032] An information acquisition module is configured to acquire an original map used by a vehicle during automatic driving, and environment information of an environment in which the vehicle is currently driving; wherein the original map is a global map containing a starting point and a destination;

[0033] A map construction module is configured to construct a local semantic map based on the environment information of the environment in which the vehicle is currently driving;

[0034] A relationship determination module is configured to determine a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving in the original map and the local semantic map, respectively;

[0035] A vehicle positioning module is configured to perform position positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map.

[0036] In some embodiments, the relationship determination module is configured to

[0037] According to the global position trajectory of the vehicle in the original map and the trajectory of the local semantic map, a position error E of the local semantic map to the position trajectory of the original map is determined;

[0038] Selecting semantic elements within the position error E in the original map and the local semantic map;

[0039] selecting a semantic matching pair for geometric relationship matching based on the semantic elements within the position error E, and performing geometric matching between the semantic matching pair;

[0040] solving the correlation matrix based on the correlation matrix and the semantic matching pair, to obtain a semantic matching pair set with the largest matching degree;

[0041] determining a conversion relationship of the local semantic map corresponding to the original map based on the semantic matching pair set.

[0042] In a third aspect, the present disclosure provides a computer readable storage medium having a semantic map based vehicle positioning program stored thereon, which, when executed by a processor, implements the semantic map based vehicle positioning method of the first aspect.

[0043] In a fourth aspect, the present disclosure provides an electronic device comprising a memory, a processor, and a semantic map based vehicle positioning program stored on the memory and executable on the processor, wherein the processor implements the semantic map based vehicle positioning method of the first aspect when executing the semantic map based vehicle positioning program.

[0044] The semantic map based vehicle positioning method according to the embodiments of the present disclosure comprises obtaining an original map used by a vehicle during automatic driving, and environment information of an environment in which the vehicle is currently driving; wherein the original map is a global map containing a starting point and a destination; constructing a local semantic map based on the environment information of the environment in which the vehicle is currently driving; determining a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving existing in the original map and the local semantic map respectively; and performing position positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map. In the present application, the global map containing the starting point and the destination is combined with the local semantic map, the conversion relationship of the local semantic map corresponding to the original map is determined through the geometric relationships of the semantic elements in the environment in which the vehicle is currently driving existing in the original map and the local semantic map respectively, and the position positioning of the vehicle in the original map is performed based on the position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map, so as to realize accurate positioning of the vehicle and enhance robustness of the positioning system.

[0045] Additional aspects and advantages of the present disclosure will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1is a semantic map based vehicle positioning method flow chart shown according to an exemplary embodiment;

[0047] Figure 2 is a local semantic map and original map matching schematic diagram shown according to an exemplary embodiment;

[0048] Figure 3 is a point-point matching schematic diagram shown according to an exemplary embodiment;

[0049] Figure 4 is a line-line matching schematic diagram shown according to an exemplary embodiment;

[0050] Figure 5 is a point-line matching schematic diagram shown according to an exemplary embodiment;

[0051] Figure 6 is a semantic map based vehicle positioning flow chart shown according to an exemplary embodiment;

[0052] Figure 7 is a semantic map based vehicle positioning device structure schematic diagram shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] The embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0054] With the development of intelligent technology, cars begin to accelerate in the direction of unmanned intelligent. Among them, the car automatic driving system is a kind of unmanned intelligent driving. The car automatic driving system, also known as automatic driving car, is also called unmanned car, computer driving car or wheeled mobile robot, which is a kind of intelligent automobile system realizing unmanned driving through vehicle computer system. However, with the development of automatic driving technology, the automatic driving vehicle depends more and more on the positioning system, especially the high robustness positioning system. This will affect the universality of automatic driving application scene of automatic driving vehicle.

[0055] In view of the above situation, the present disclosure provides a semantic map based vehicle positioning method. Figure 1 is a semantic map based vehicle positioning method flow chart shown according to an exemplary embodiment. As Figure 1 shown, the semantic map based vehicle positioning method comprises:

[0056] Step 10, obtaining an original map used by the vehicle when the vehicle is automatically driving, and environment information of an environment where the vehicle is currently driving; wherein the original map is a global map containing a starting point and a destination;

[0057] Step 11, constructing a local semantic map based on the environment information of the environment where the vehicle is currently driving;

[0058] Step 12, determining a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment where the vehicle is currently driving existing in the original map and the local semantic map, respectively;

[0059] Step 13, performing location positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map.

[0060] In an example embodiment, the environment information of the environment where the vehicle is currently driving includes a speed bump, an arrow, a zebra crossing, a stop line, a parking space, and the like in the environment where the vehicle is currently driving. The original map is a global map containing a starting point and a destination, which can be a high-precision map such as a Baidu map, a Gaode map, or a non-high-precision map. The environment information of the environment where the vehicle is currently driving can be obtained in real time through a vehicle-mounted camera. Figure 2 is a matching schematic diagram between a local semantic map and an original map according to an example embodiment. As shown in Figure 2 M1 is an original map, and M2 is a local semantic map. As shown in Figure 2 the matching of the semantic matching pair between the local semantic map and the original map is performed.

[0061] In an example embodiment, the local semantic map can be constructed according to the information of the speed bump, the arrow, the zebra crossing, the stop line, the parking space, and the like in the environment where the vehicle is currently driving.

[0062] According to the vehicle positioning method based on the semantic map, the global map containing the starting point and the destination is combined with the local semantic map, the conversion relationship of the local semantic map corresponding to the global map is determined through the geometric relationship of the semantic elements in the environment where the vehicle is currently driving existing in the global map and the local semantic map respectively, the position of the vehicle in the global map is positioned based on the position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the global map, so as to realize the accurate positioning of the vehicle in the global map according to the specific scene in the local semantic map, and further to enhance the robustness of the vehicle in the global map.

[0063] In some embodiments, the conversion relationship of the local semantic map corresponding to the global map is determined based on the geometric relationship of the semantic elements in the environment where the vehicle is currently driving existing in the global map and the local semantic map respectively, including:

[0064] According to the global position trajectory of the vehicle in the global map and the trajectory of the local semantic map, the position error E of the position trajectory of the local semantic map to the global map is determined;

[0065] The semantic elements within the position error E are selected in the global map and the local semantic map;

[0066] Based on the semantic elements within the position error E, the semantic matching pairs for geometric relationship matching are selected, and the geometric matching between the semantic matching pairs is performed;

[0067] Based on the correlation matrix and the semantic matching pairs, the correlation matrix is solved to obtain a semantic matching pair set with the largest matching degree;

[0068] Based on the semantic matching pair set, the conversion relationship of the local semantic map corresponding to the global map is determined.

[0069] In an example embodiment, a position error E of the local semantic map to a position trajectory of the original map is determined according to a global position trajectory of the vehicle in the original map and a trajectory of the local semantic map, including:

[0070] A weighted average of absolute values of position differences between matched elements in the local semantic map and matched elements in the original map after T conversion is the position error E.

[0071] Based on semantic elements within the position error E, a semantic matching pair for geometric relationship matching is selected, and geometric matching between the semantic matching pairs is performed. For example, the semantic elements within the position error E include a speed bump, an arrow, a zebra crossing, a stop line, and a parking space, and any one or more of the features in the speed bump, the arrow, the zebra crossing, the stop line, and the parking space can be selected as a semantic matching pair for geometric matching between the semantic matching pairs. For example, two edge lines of the speed bump are selected as a semantic matching pair, and a line-line included angle between the two edge lines is determined.

[0072] In some embodiments, the correlation matrix is solved based on the correlation matrix and the semantic matching pair to obtain a semantic matching pair set with the largest matching degree, including:

[0073] The correlation matrix is determined based on the semantic matching pair;

[0074] The maximum clique problem is solved based on the correlation matrix, and the correlation matrix is solved to obtain a semantic matching pair set with the largest matching degree.

[0075] In an example embodiment, the speed bump, the arrow, the zebra crossing, the stop line, and the parking space can be matched in advance. For example, a line-line distance between two edge lines of the speed bump, a vertex-to-bottom line distance of the arrow, a distance between two edge lines of the zebra crossing, a line-line distance between adjacent stop lines, and a line-line distance between two edge lines of the parking space. When the correlation matrix is constructed, the matching result based on the geometric consistency of the semantic matching pair can be used for construction. For example, the correlation matrix S,

[0076] If there are a total of 4 semantic matching pairs, the correlation matrix S is 4*4, and if there are a total of n semantic matching pairs, the correlation matrix S is n*n. Here, only an example is provided, and it is not intended to be limiting.

[0077] In some embodiments, the conversion relationship of the local semantic map to the original map is determined based on the semantic matching pair set, including:

[0078] The centroid position of the semantic matching pair set is determined based on the semantic matching pair set.

[0079] Based on the centroid position of the set of semantic matching pairs, a Euclidean transformation is performed to determine the decentroid coordinates of the set of semantic matching pairs.

[0080] Based on the centroid coordinates of the set of semantic matching pairs, determine the rotation matrix;

[0081] Based on the centroid position of the set of semantic matching pairs and the rotation matrix, the transformation relationship between the local semantic map and the original map is determined.

[0082] In an exemplary embodiment,

[0083] p i p′ represents the semantic elements that constitute semantic matching pairs in the local semantic map. i The semantic elements that constitute semantic matching pairs in the original map; p 为 The centroid position of the semantic elements that constitute a semantic matching pair in the local semantic map, p′ 为 The centroid positions of the semantic elements that constitute a semantic matching pair in the original map;

[0084] q i =p i -p, q′ i =p′ i -p′;

[0085] q i为 Centroid-free coordinates of semantic elements that constitute semantic matching pairs in a local semantic map.

[0086] q′ i These are the centroid coordinates of the semantic elements that constitute semantic matching pairs in the original map.

[0087] The rotation matrix R is determined by optimizing formula ().

[0088]

[0089] R is the rotation matrix, and t is the transformation relationship between the local semantic map and the original map;

[0090] Based on the rotation matrix R in Formula (I), determine the transformation relation t between the local semantic map and the original map; where t * For the solution of t, t * =p-Rp′.

[0091] In some embodiments, the semantic matching pair includes at least one of the following:

[0092] Point matching corresponding to semantic elements, point-line matching corresponding to semantic elements, and line-line matching corresponding to semantic elements.

[0093] In an example embodiment, the point-point matching corresponding to the semantic element, the point-line matching corresponding to the semantic element, and the line-line matching corresponding to the semantic element in the present application can all be used as a semantic matching pair.

[0094] In some embodiments, geometric matching between the semantic matching pairs is performed, including:

[0095] Determining geometric consistency of point-point distance in the local semantic map and point-point distance in the original map;

[0096] Determining geometric consistency of line-line angle in the local semantic map and line-line angle in the original map;

[0097] Determining geometric consistency of point-line distance in the local semantic map and point-line distance in the original map.

[0098] In an example embodiment, Figure 3 is a point-point matching schematic diagram according to an example embodiment. As shown in Figure 3 , point and point: candidate matching pairs are [Pt1 P1] and [Pt2 P2]; wherein Pt1 and Pt2 are semantic points in the original map, and P1 and P2 are semantic points in the local semantic map;

[0099] Determining the distance between Pt1 and Pt2 as Dr, and the distance between P1 and P2 as D, determining the absolute value of the difference between Dr and D as dd, and then verifying geometric consistency using dd; wherein the scale factor threshold is Sp, the point consistency basic threshold is Tol_p, and the consistency threshold can solve the matching problem in the case of inconsistent scales of the two maps;

[0100] If dd >= Sp*max(Dr, D) + Tol_p, the matching score of the matching pair is 0;

[0101] If dd < Sp*max(Dr, D) + Tol_p, the smaller dd is, the higher the matching score is, and if dd is 0, the matching score is 1, a relationship can be set so that the matching score and the size of dd are negatively related.

[0102] Figure 4 is a line-line matching schematic diagram according to an example embodiment. As shown in Figure 4 , line and line: candidate matching pairs are [P1P2 Pt1Pt2] and [P3P4 Pt3Pt4];

[0103] Determining the distance D1 between the midpoints of line segments P1P2 and P3P4, and the distance D2 between the midpoints of line segments Pt1Pt2 and Pt3Pt4;

[0104] Determining the absolute value of the difference between D1 and D2 as dd;

[0105] dd verifies geometric consistency with "point to point";

[0106] determine the angle theta1 of the line P1P2 and P3P4, determine the angle theta2 of the line Pt1Pt2 and Pt3Pt4;

[0107] obtain the absolute value of the difference between theta1 and theta2 as dtheta, and then verify geometric consistency through dtheta; wherein,

[0108] the scale factor threshold is Sl, the angle consistency basic threshold is Tol_l, and the consistency threshold can solve the matching problem in the case of two map scale inconsistency.

[0109] If dtheta >= Sl*max(D1, D2) + Tol_l, the matching score of the matching pair is 0, and the verification is terminated;

[0110] If dd < Sl*max(D1, D2) + Tol_l, the smaller dtheta is, the higher the matching score is, and when dtheta is 0, the matching score is 1. A relationship can be set so that the matching score and the size of dtheta are negatively related.

[0111] If max(abs(theta1), abs(theta2)) is close to 0, it can be considered that P1P2 and P3P4 are parallel, and Pt1Pt2 and Pt3Pt4 are parallel;

[0112] Then verify the geometric consistency by the distance between the parallel lines. If the angle difference between the map Mr and the map M is known to be less than theta, the geometric consistency can be further verified;

[0113] Figure 5 is a point-line matching schematic diagram according to an exemplary embodiment. As Figure 5 shown, point to line: candidate matching pairs are [P1 Pt1], [P3P4 Pt3Pt4];

[0114] Determine the distance D1 between the point P1 and the midpoint of P3P4, and the distance D2 between the point Pt1 and the midpoint of Pt3Pt4;

[0115] Determine the absolute value of the difference between D1 and D2 as dd;

[0116] dd verifies geometric consistency with "point to point";

[0117] Determine the perpendicular line vector L1 from the point P1 to P3P4, and the perpendicular line vector L2 from the point Pt1 to Pt3Pt4;

[0118] Determine the geometric consistency through L1 and L2, such as the length and direction difference of L1 and L2.

[0119] If the angle difference between the map Mr and the map M is known to be less than theta, the angle between the vector Pt1->Pt2 and the vector P1-> is If the angle is greater than theta, the matching score of the matching pair is 0.

[0120] Figure 6 is a flow chart of vehicle positioning based on semantic map according to an exemplary embodiment. As shown in Figure 6 the flow chart of vehicle positioning based on semantic map comprises:

[0121] Step 60, obtaining an original map M1;

[0122] Step 61, constructing a semantic map M2 in real time;

[0123] Step 62, intercepting / downsampling;

[0124] Step 63, positioning initialization;

[0125] Step 64, judging whether the positioning initialization is successful, if the positioning initialization is not successful, executing Step 63, positioning initialization;

[0126] Step 65, if the positioning initialization is successful, performing semantic map matching;

[0127] Step 66, judging whether the semantic map matching is successful, if not, executing Step 65, semantic map matching;

[0128] Step 67, if the semantic map matching is successful, performing position determination.

[0129] The present disclosure provides a vehicle positioning device based on semantic map. Figure 7 is a structural schematic diagram of a vehicle positioning device based on semantic map according to an exemplary embodiment. As shown in Figure 7 the structural schematic diagram of the vehicle positioning device based on semantic map comprises:

[0130] An information obtaining module 70 is configured to obtain an original map used by a vehicle during automatic driving, and environment information of an environment in which the vehicle is currently driving; wherein the original map is a global map containing a starting point and a destination;

[0131] A map constructing module 71 is configured to construct a local semantic map based on the environment information of the environment in which the vehicle is currently driving;

[0132] A relationship determining module 72 is configured to determine a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving respectively existing in the original map and the local semantic map;

[0133] The vehicle positioning module 73 is configured to position the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship between the local semantic map and the original map.

[0134] In an example embodiment, the environment information of the environment where the current driving state of the vehicle is located includes a speed bump, an arrow, a zebra crossing, a stop line, a parking space, and the like in the environment where the current driving state of the vehicle is located. The original map is a global map containing a starting point and a destination, which can be a high-precision map such as a Baidu map or a Gaode map, or a non-high-precision map. The environment information of the environment where the current driving state of the vehicle is located can be obtained in real time by a vehicle-mounted camera.

[0135] In an example embodiment, the local semantic map can be constructed according to the information of the speed bump, the arrow, the zebra crossing, the stop line, and the parking space in the environment where the current driving state of the vehicle is located.

[0136] The vehicle positioning device based on a semantic map according to the embodiments of the present disclosure is configured to obtain an original map used by a vehicle during automatic driving, and environment information of an environment where a current driving state of the vehicle is located. The original map is a global map containing a starting point and a destination. A local semantic map is constructed based on the environment information of the environment where the current driving state of the vehicle is located. A conversion relationship between the local semantic map and the original map is determined based on geometric relationships of semantic elements in the environment where the current driving state of the vehicle is located in the original map and the local semantic map, respectively. Positioning of the vehicle in the original map is performed based on position information of the vehicle in the local semantic map and the conversion relationship between the local semantic map and the original map. In the present disclosure, the global map containing the starting point and the destination is combined with the local semantic map. The conversion relationship between the local semantic map and the original map is determined based on the geometric relationships of the semantic elements in the environment where the current driving state of the vehicle is located in the original map and the local semantic map, respectively. Positioning of the vehicle in the original map is performed based on the position information of the vehicle in the local semantic map and the conversion relationship between the local semantic map and the original map. Thus, accurate positioning of the vehicle in the original map according to a specific scene in the local semantic map is realized, which is conducive to enhancing the robustness of positioning of the vehicle in the original map.

[0137] In some embodiments, the relationship determining module is configured to

[0138] The position error E of the position trajectory of the local semantic map to the position trajectory of the original map is determined according to the global position trajectory of the vehicle in the original map and the trajectory of the local semantic map.

[0139] The semantic elements within the position error E are selected in the original map and the local semantic map.

[0140] selecting semantic matching pairs for geometric relationship matching based on the semantic elements within the position error E, and performing geometric matching between the semantic matching pairs;

[0141] solving the association matrix based on the association matrix and the semantic matching pairs to obtain a semantic matching pair set with the largest matching degree;

[0142] determining a conversion relationship of the local semantic map corresponding to the original map based on the semantic matching pair set.

[0143] In an example embodiment, the position error E of the local semantic map to the position trajectory of the original map is determined according to the global position trajectory of the vehicle in the original map and the trajectory of the local semantic map, including:

[0144] The weighted average of the absolute value of the position difference between the matching elements in the local semantic map after T conversion and the matching elements in the original map is the position error E.

[0145] Selecting semantic matching pairs for geometric relationship matching based on the semantic elements within the position error E, and performing geometric matching between the semantic matching pairs. For example, the semantic elements within the position error E include speed bumps, arrows, zebra crossings, stop lines, and parking spaces, etc., and any one or more of the features in the speed bumps, arrows, zebra crossings, stop lines, and parking spaces can be selected as semantic matching pairs for geometric matching between the semantic matching pairs. For example, two edge lines of a speed bump are selected as semantic matching pairs to determine the line-line included angle between the two edge lines.

[0146] In some embodiments, the relationship determining module is configured to

[0147] determine the association matrix based on the semantic matching pairs;

[0148] Solving the maximum clique problem based on the association matrix, solving the association matrix to obtain a semantic matching pair set with the largest matching degree.

[0149] In an example embodiment, speed bumps, arrows, zebra crossings, stop lines, and parking spaces, etc. can be matched in advance. For example, the line-line distance between the two edge lines of a speed bump, the distance between the vertex of an arrow and the bottom line, the distance between the two edge lines of a zebra crossing, the line-line distance between adjacent stop lines, and the line-line distance between the two edge lines in a parking space.

[0150] In some embodiments, the relationship determining module is configured to

[0151] determining the centroid position of the semantic matching pair set based on the semantic matching pair set;

[0152] Based on the centroid position of the set of semantic matching pairs, a Euclidean transformation is performed to determine the barycentric coordinates of the set of semantic matching pairs;

[0153] Based on the barycentric coordinates of the set of semantic matching pairs, a rotation matrix is determined.

[0154] Based on the centroid position of the set of semantic matching pairs and the rotation matrix, a transformation relationship of the local semantic map corresponding to the original map is determined.

[0155] In exemplary embodiments,

[0156] p i p′ i p′ 为 p′ 为 p′

[0157] q i q′ i q′ i q′ i q′

[0158] q i为 q′

[0159] q′ i q′

[0160] The rotation matrix R is determined by optimizing the formula ().

[0161] wherein,

[0162] R is a rotation matrix, and t is the transformation relationship of the local semantic map corresponding to the original map.

[0163] According to the rotation matrix R in formula (I), the transformation relationship t of the local semantic map corresponding to the original map is determined; wherein t * is the solution of t, t * = p-Rp′.

[0164] In some embodiments, the semantic matching pairs at least include one of the following:

[0165] Point-to-point matching corresponding to semantic elements, point-to-line matching corresponding to semantic elements, and line-to-line matching corresponding to semantic elements.

[0166] In the exemplary embodiments, the point-point matching corresponding to the semantic elements, the point-line matching corresponding to the semantic elements, and the line-line matching corresponding to the semantic elements can all be used as the semantic matching pairs.

[0167] In some embodiments, geometric matching between the semantic matching pairs is performed, including:

[0168] determining geometric consistency of point-point distances in the local semantic map and point-point distances in the original map;

[0169] determining geometric consistency of line-line angles in the local semantic map and line-line angles in the original map;

[0170] determining geometric consistency of point-line distances in the local semantic map and point-line distances in the original map.

[0171] In the exemplary embodiments, for example, as shown in the figure, point and point: candidate matching pairs are [Pt1 P1] and [Pt2 P2]; wherein Pt1 and Pt2 are semantic points in the original map, and P1 and P2 are semantic points in the local semantic map.

[0172] determining the distance between Pt1 and Pt2 as Dr, and the distance between P1 and P2 as D, determining the absolute value of the difference between Dr and D as dd, and then verifying geometric consistency using dd; wherein the scale factor threshold is Sp, the point consistency basic threshold is Tol_p, and the consistency threshold can solve the matching problem in the case of inconsistent scales of the two maps.

[0173] if dd >= Sp*max(Dr, D) + Tol_p, then the matching score of the matching pair is 0;

[0174] if dd < Sp*max(Dr, D) + Tol_p, then the smaller dd is, the higher the matching score is, and if dd is 0, the matching score is 1; a relationship can be set so that the matching score and the size of dd are negatively correlated.

[0175] line and line: candidate matching pairs are [P1P2 Pt1Pt2] and [P3P4 Pt3Pt4];

[0176] determining the distance D1 between the midpoints of line segments P1P2 and P3P4, and the distance D2 between the midpoints of line segments Pt1Pt2 and Pt3Pt4;

[0177] determining the absolute value of the difference between D1 and D2 as dd;

[0178] dd verifies geometric consistency as in "point and point";

[0179] Determine the angle theta1 between the lines P1P2 and P3P4, and determine the angle theta2 between the lines Pt1Pt2 and Pt3Pt4.

[0180] Obtain the absolute value of the difference between theta1 and theta2 as dtheta, and then verify the geometric consistency through dtheta; wherein,

[0181] The scale factor threshold is Sl, the angle consistency basic threshold is Tol_l, and the consistency threshold can solve the matching problem in the case of inconsistent scales of two maps.

[0182] If dtheta >= Sl*max(D1, D2) + Tol_l, the matching score of the matching pair is 0, and the verification is terminated;

[0183] If dd < Sl*max(D1, D2) + Tol_l, the smaller dtheta is, the higher the matching score is, and if dtheta is 0, the matching score is 1. A relationship can be set so that the matching score is negatively related to the size of dtheta.

[0184] If max(abs(theta1), abs(theta2)) is close to 0, it can be considered that P1P2 and P3P4 are parallel, and Pt1Pt2 and Pt3Pt4 are parallel;

[0185] Then verify the geometric consistency by the distance between the parallel lines. If the angle difference between the map Mr and the map M is known to be less than theta, the geometric consistency can be further verified;

[0186] Point and line: the candidate matching pairs are [P1 Pt1] and [P3P4 Pt3Pt4];

[0187] Determine the distance D1 between the point P1 and the midpoint of P3P4, and the distance D2 between the point Pt1 and the midpoint of Pt3Pt4;

[0188] Determine the absolute value of the difference between D1 and D2 as dd;

[0189] dd verifies the geometric consistency as "point and point";

[0190] Determine the perpendicular line vector L1 from the point P1 to P3P4, and the perpendicular line vector L2 from the point Pt1 to Pt3Pt4;

[0191] Determine the geometric consistency through L1 and L2, such as the length and direction difference of L1 and L2.

[0192] If the angle difference between the map Mr and the map M is known to be less than theta, the angle between the vector Pt1->Pt2 and the vector P1-> is greater than theta, and the matching score of the matching pair is 0.

[0193] In some embodiments, the relationship determining module is configured to

[0194] determine a position error E of the local semantic map to a position trajectory of the original map according to a global position trajectory of the vehicle in the original map and a trajectory of the local semantic map;

[0195] select semantic elements within the position error E in the original map and the local semantic map;

[0196] select a semantic matching pair for geometric relationship matching based on the semantic elements within the position error E, and perform geometric matching between the semantic matching pair;

[0197] perform correlation matrix solving based on the correlation matrix and the semantic matching pair to obtain a semantic matching pair set with the largest matching degree;

[0198] determine a conversion relationship of the local semantic map to the original map based on the semantic matching pair set.

[0199] The present disclosure provides a computer readable storage medium having stored thereon a semantic map based vehicle positioning program, which, when executed by a processor, implements the semantic map based vehicle positioning method of any of the above embodiments.

[0200] The present disclosure provides an electronic device comprising a memory, a processor, and a semantic map based vehicle positioning program stored on the memory and executable on the processor, wherein the processor, when executing the semantic map based vehicle positioning program, implements the semantic map based vehicle positioning method of any of the above embodiments.

[0201] It should be noted that the logical and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination of the above. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing, and / or an article of manufacture. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electronic connection having one or more wires (electronic devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical devices), and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0202] It should be understood that portions of the present disclosure can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, a number of steps or methods can be implemented in software or firmware that is stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0203] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0204] In the description of the present disclosure, it needs to be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the purpose of facilitating the description of the present disclosure and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present disclosure.

[0205] In addition, the terms "first", "second", and the like used in the embodiments of the present disclosure are only for descriptive purposes, and cannot be understood as indicating or implying relative importance, or implicitly indicating the number of technical features referred to in the embodiments. Therefore, the features defined with the terms "first", "second" and the like in the embodiments of the present disclosure can explicitly or implicitly indicate that at least one such feature is included in the embodiments. In the description of the present disclosure, the meaning of the word "multiple" is at least two or two or more, such as two, three, four, etc., unless otherwise specifically limited in the embodiments.

[0206] In the present disclosure, unless otherwise specifically defined or limited in the embodiments, the terms "mounting", "connecting", "connecting" and "fixing" and the like appearing in the embodiments should be understood broadly, for example, the connection can be a fixed connection, or a detachable connection, or integrated, which can be understood, or can be a mechanical connection, an electrical connection, etc. Of course, it can also be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements, or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present disclosure can be understood according to the specific implementation situation.

[0207] In the present disclosure, unless otherwise specifically defined or limited, the first feature "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature "above", "above" and "above" the second feature can be that the first feature is directly above or obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "below" and "below" the second feature can be that the first feature is directly below or obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0208] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for vehicle positioning based on semantic map, characterized in that, The method comprises the following steps: acquiring an original map used by a vehicle for automatic driving and environment information of an environment in which the vehicle is currently driving, wherein the original map is a global map containing a starting point and a destination; constructing a local semantic map based on the environment information of the environment in which the vehicle is currently driving; determining a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving existing in the original map and the local semantic map, including determining a position error E of a position trajectory of the local semantic map to a position trajectory of the original map according to a global position trajectory of the vehicle in the original map and a trajectory of the local semantic map, selecting semantic elements within the position error E in the original map and the local semantic map, selecting a semantic matching pair for geometric relationship matching based on the semantic elements within the position error E, and performing geometric matching between the semantic matching pairs, solving a correlation matrix based on the correlation matrix and the semantic matching pair, and obtaining a semantic matching pair set with the largest matching degree, and determining the conversion relationship of the local semantic map corresponding to the original map based on the semantic matching pair set; performing position positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map.

2. The semantic map based vehicle positioning method according to claim 1, characterized in that, Solving the correlation matrix based on the correlation matrix and the semantic matching pair to obtain a semantic matching pair set with the largest matching degree comprises: determining the correlation matrix based on the semantic matching pair; solving the correlation matrix to obtain a semantic matching pair set with the largest matching degree by solving a maximum clique problem based on the correlation matrix.

3. The semantic map based vehicle positioning method of claim 1, wherein, The determination of the conversion relationship of the local semantic map corresponding to the original map based on the semantic matching pair set comprises: determining a centroid position of an arbitrary group of semantic matching pairs based on the semantic matching pair set; determining a de-centroid coordinate of each semantic matching pair based on the centroid position of the arbitrary group of semantic matching pairs by performing Euclidean transformation; determining a rotation matrix based on the de-centroid coordinate of each semantic matching pair; determining the conversion relationship of the local semantic map corresponding to the original map based on the centroid position of the arbitrary group of semantic matching pairs and the rotation matrix.

4. The semantic map based vehicle positioning method of claim 1, wherein, The semantic matching pair comprises at least one of the following: a point-point matching corresponding to semantic elements, a point-line matching corresponding to semantic elements, and a line-line matching corresponding to semantic elements.

5. The semantic map based vehicle positioning method of claim 1, wherein, The geometric matching between the semantic matching pairs comprises: determining geometric consistency of point-point distances in the local semantic map and point-point distances in the original map; determining geometric consistency of line-line angles in the local semantic map and line-line angles in the original map; determining geometric consistency of point-line distances in the local semantic map and point-line distances in the original map.

6. A vehicle positioning apparatus based on a semantic map, characterized by, ​ The information acquisition module is configured to acquire an original map used by a vehicle in automatic driving and environment information of an environment in which the vehicle is currently driving, wherein the original map is a global map containing a starting point and a destination. The map construction module is configured to construct a local semantic map based on the environment information of the environment in which the vehicle is currently driving. The relationship determination module is configured to determine a conversion relationship of the local semantic map corresponding to the original map based on geometric relationships of semantic elements in the environment in which the vehicle is currently driving in the original map and the local semantic map, including determining a position error E of a position trajectory of the local semantic map to a position trajectory of the original map according to a global position trajectory of the vehicle in the original map and a trajectory of the local semantic map; selecting semantic elements within the position error E in the original map and the local semantic map; selecting a semantic matching pair for geometric relationship matching based on the semantic elements within the position error E, and performing geometric matching between the semantic matching pair; performing correlation matrix solving based on a correlation matrix and the semantic matching pair to obtain a semantic matching pair set with the largest matching degree; and determining the conversion relationship of the local semantic map corresponding to the original map based on the semantic matching pair set. The vehicle positioning module is configured to perform position positioning of the vehicle in the original map based on position information of the vehicle in the local semantic map and the conversion relationship of the local semantic map corresponding to the original map.

7. A computer readable storage medium characterized by A computer readable storage medium having stored thereon a vehicle positioning program based on a semantic map, which, when executed by a processor, implements the vehicle positioning method based on a semantic map in any one of claims 1-5.

8. An electronic device, comprising: A computer readable storage medium having stored thereon a vehicle positioning program based on a semantic map, which, when executed by a processor, implements the vehicle positioning method based on a semantic map in any one of claims 1-5.

Images