Feature matching method, device and unmanned vehicle

Abstract

The present disclosure provides a feature matching method, device and unmanned vehicle, and relates to the technical field of computer vision, in particular to the field of unmanned driving. The feature matching method comprises: taking each feature point in a first feature point set of a target as a reference point, constructing a reference deviation compensation vector according to the position deviation between the reference point and a candidate matching point of the reference point in a second feature point set of the target; performing deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector; determining the candidate matching points of the feature points after deviation compensation in the second feature point set to obtain a plurality of candidate matching point sets; and determining an optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the plurality of candidate matching point sets. Through the above steps, the matching effect of sparse feature points can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision, in particular to the field of unmanned driving, and especially to a feature matching method, device and unmanned vehicle. BACKGROUND

[0002] In the field of unmanned driving, a feature matching task is a basic task. For example, when detecting and positioning a traffic light, image data of the traffic light collected by a camera needs to be matched with point cloud data of the traffic light extracted from a map.

[0003] At present, feature matching algorithms mainly include dense point cloud matching algorithms, such as grid motion statistics (GMS) or vector field consistency (VFC) algorithms. Such matching algorithms rely on the relatively perfect feature distribution of dense point clouds and can achieve good feature matching results. However, if such matching algorithms are applied to a sparse point cloud matching scenario, such as a matching task between image detection of a traffic light and map projection of a traffic light, due to the small number of feature points and the limited feature expression of sparse point clouds, the matching algorithms cannot achieve good feature matching results.

[0004] In related technologies, when matching sparse feature points, a weighted bipartite graph optimal matching algorithm, such as a KM algorithm, is used. In this scenario, since there are still interference points, if such an algorithm is used, the interference points will be incorrectly matched. Using a limited distance combined with the KM algorithm can exclude matching points with relatively long distances, but it still cannot solve the problem of incorrect matching caused by factors such as the close distance between interference points and to-be-matched points, abnormal feature point sets, and matching deviations. SUMMARY

[0005] One of the technical problems to be solved by the present disclosure is to provide a solution that can improve the matching effect of sparse feature points and reduce the matching error rate.

[0006] According to a first aspect of the present disclosure, a feature matching method is provided, including: taking each feature point in a first feature point set of a target as a reference point, constructing a reference deviation compensation vector according to the reference point and the position deviation between the reference point and a candidate matching point in a second feature point set of the target; performing deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector; determining candidate matching points of the deviation-compensated feature points in the second feature point set to obtain a plurality of candidate matching point sets; and determining an optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the plurality of candidate matching point sets.

[0007] In some embodiments, determining the optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the plurality of candidate matching point sets comprises: constructing a plurality of bipartite graphs according to the first feature point set and the plurality of candidate matching point sets; determining a best complete matching result corresponding to each bipartite graph based on a bipartite graph matching algorithm; and determining the optimal feature matching result of the first feature point set and the second feature point set in the best complete matching result corresponding to each bipartite graph.

[0008] In some embodiments, constructing the bias compensation vector further comprises: searching for a feature point in the second feature point set within a maximum search distance range corresponding to the reference point with the reference point as the center, and taking the searched feature point as a candidate matching point of the reference point in the second feature point set.

[0009] In some embodiments, constructing the reference bias compensation vector further comprises: determining the maximum search distance corresponding to the reference point according to the depth information of the reference point.

[0010] In some embodiments, bias compensating other feature points in the first feature point set according to the reference bias compensation vector comprises: determining a dynamic bias compensation vector corresponding to the other feature points in the first feature point set according to the relationship between the other feature points and the depth information of the reference point, and the reference bias compensation vector; and bias compensating the other feature points according to the dynamic bias compensation vector to obtain bias-compensated feature points.

[0011] In some embodiments, determining the candidate matching points of the bias-compensated feature points in the second feature point set comprises: determining a maximum search distance corresponding to the bias-compensated feature points according to the depth information of the bias-compensated feature points; and searching for feature points in the second feature point set within the corresponding maximum search distance range with the bias-compensated feature points as the center, and taking the searched feature points as the candidate matching points corresponding to the bias-compensated feature points.

[0012] In some embodiments, determining the best complete matching result corresponding to each bipartite graph based on the bipartite graph matching algorithm comprises: for each bipartite graph, determining the matching weight of each edge in the bipartite graph according to the distance between the bias-compensated feature points and their candidate matching points corresponding to the bipartite graph; and taking the complete matching with the smallest sum of matching weights as the best complete matching result corresponding to the bipartite graph.

[0013] In some embodiments, determining the matching weight of each edge in the bipartite graph according to the distance between the bias-compensated feature point and its candidate matching point comprises: determining the matching weight of each edge in the bipartite graph according to the distance between the bias-compensated feature point and its candidate matching point corresponding to the bipartite graph, and a weight adjustment parameter, wherein the weight adjustment parameter is determined according to the depth information corresponding to the feature point.

[0014] In some embodiments, determining the optimal feature matching result of the first feature point set and the second feature point set from the optimal complete matching results corresponding to the respective bipartite graphs comprises: determining a score of the optimal complete matching result corresponding to each bipartite graph according to the matching accuracy and matching completeness indicators; and selecting the optimal feature matching result from the optimal complete matching results corresponding to the respective bipartite graphs according to the score.

[0015] In some embodiments, determining the score of the optimal complete matching result corresponding to each bipartite graph according to the matching accuracy and matching completeness indicators comprises: for each optimal complete matching result corresponding to a bipartite graph, calculating a sum of the maximum search distances of the respective bias-compensated feature points in the optimal complete matching result as a first parameter value; calculating a sum of the distances between the respective bias-compensated feature points and their matching points in the optimal complete matching result as a second parameter value; and determining the matching accuracy indicator according to the first parameter value and the second parameter value, wherein the matching accuracy indicator is positively correlated with the first parameter value, and the matching accuracy indicator is negatively correlated with the second parameter value.

[0016] In some embodiments, determining the score of the optimal complete matching result corresponding to each bipartite graph according to the matching accuracy and matching completeness indicators comprises: for each optimal complete matching result corresponding to a bipartite graph, calculating a number of successfully matched feature points of the second feature point set in the optimal complete matching result as a fourth parameter value; calculating a maximum value of the number of feature points in the first feature point set and the number of feature points in the second feature point set as a fifth parameter value; and determining the matching completeness indicator according to the fourth parameter value and the fifth parameter value, wherein the matching completeness indicator is positively correlated with the fourth parameter value, and the matching completeness indicator is negatively correlated with the fifth parameter value.

[0017] In some embodiments, determining the score of the optimal complete matching result corresponding to each bipartite graph according to the matching accuracy and matching completeness indicators comprises: calculating an F-score of the optimal complete matching result corresponding to each bipartite graph according to the matching accuracy and matching completeness indicators; and determining the score of the optimal complete matching result corresponding to each bipartite graph according to the F-score.

[0018] In some embodiments, the determining the score of the optimal complete matching result corresponding to each bipartite graph according to the F-score comprises: determining the score of the optimal complete matching result according to the F-score, and at least one of a matching penalty term and a matching reward term, wherein the matching penalty term is set according to the overall deviation of the first set of feature points in the matching result, and the matching reward term is set according to the matching priority of the successfully matched feature points in the second set of feature points in the matching result.

[0019] In some embodiments, the method further comprises: obtaining first source data and second source data of the target; determining the first set of feature points of the target according to the first source data, and determining the second set of feature points of the target according to the second source data.

[0020] In some embodiments, the first source data of the target is a map, and the second source data of the target is an image captured by a camera.

[0021] In some embodiments, the feature data of the first source of the target is feature data of the target inherent in the map, and the feature data of the second source of the target is feature data of the target detected in the image captured by the camera.

[0022] According to a second aspect of the present disclosure, a feature matching device is provided, comprising: a construction module configured to take each feature point in a first set of feature points of a target as a reference point, and construct a reference deviation compensation vector according to the position deviation between the reference point and its candidate matching point in a second set of feature points of the target; a deviation compensation module configured to perform deviation compensation on other feature points in the first set of feature points according to the reference deviation compensation vector; a first determination module configured to determine candidate matching points of the deviation-compensated feature points in the second set of feature points to obtain a plurality of candidate matching point sets; and a second determination module configured to determine an optimal feature matching result of the first set of feature points and the second set of feature points according to the first set of feature points and the plurality of candidate matching point sets.

[0023] According to a third aspect of the present disclosure, a feature matching device is further provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the feature matching method as described above based on instructions stored in the memory.

[0024] According to a fourth aspect of the present disclosure, a computer readable storage medium is further provided, which stores computer program instructions, the instructions being executed by a processor to implement the feature matching method as described above.

[0025] According to a fifth aspect of the present disclosure, an unmanned vehicle is provided, comprising the feature matching device as described above.

[0026] Compared with the related art, in the embodiment of the present disclosure, by taking each feature point in the first feature point set of the target as a reference point, a reference deviation compensation vector is constructed, other feature points in the first feature point set are compensated for deviation according to the reference deviation compensation vector, candidate matching points of the feature points after deviation compensation in the second feature point set are determined to obtain a plurality of candidate matching point sets, and the optimal feature matching result of the first feature point set and the second feature point set is determined according to the first feature point set and the plurality of candidate matching point sets. These steps can improve the matching effect of sparse feature points, reduce the error matching rate, and effectively solve the error matching problem caused by factors such as the close distance of the interference point to the to-be-matched point, the existence of abnormal feature point set, and matching deviation. Further, it is helpful to improve the target detection and positioning accuracy, improve the safety of people and vehicles in the unmanned driving scene, and reduce the accident rate in the unmanned driving scene.

[0027] Other features of the present disclosure and its advantages will become more apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] The present disclosure can be understood moreappreciably with reference to the following detailed description in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a feature matching method according to some embodiments of the present disclosure.

[0031] Figure 2 A flowchart of deviation compensation on feature points according to some embodiments of the present disclosure.

[0032] Figure 3 A flowchart of determining a candidate matching point set according to some embodiments of the present disclosure.

[0033] Figure 4 A flowchart of determining an optimal feature matching result according to some embodiments of the present disclosure.

[0034] Figure 5a A flowchart of a feature matching method according to some other embodiments of the present disclosure.

[0035] Figure 5b A comparison diagram of the effect of the feature matching method according to some embodiments of the present disclosure and the feature matching method of the related art.

[0036] Figure 6 A structural diagram of a feature matching device according to some embodiments of the present disclosure.

[0037] Figure 7 Structure diagram of a feature matching device according to some embodiments of the present disclosure.

[0038] Figure 8 Structure diagram of a computer system according to some embodiments of the present disclosure.

[0039] Figure 9 Structure diagram of an unmanned vehicle according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0040] Various exemplary embodiments of the present disclosure will now be described in detail by referring to the drawings. It should be noted that the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0041] It should be understood that the sizes of the various portions shown in the drawings are not necessarily drawn to scale for ease of description.

[0042] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the scope of the present disclosure and its applications or uses.

[0043] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the description of the present disclosure.

[0044] In all examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0045] It should be noted that like numbers and letters refer to like items throughout the drawings, and once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0046] In order to make the purposes, technical solutions, and advantages of the present disclosure clearer, the present disclosure is further described in detail below with reference to specific embodiments and with reference to the drawings.

[0047] Figure 1 Flow diagram of a feature matching method according to some embodiments of the present disclosure. As shown in FIG. 6, the method includes: Figure 1

[0048] Step S110: Taking each feature point in the first feature point set of the target as a reference point, a reference deviation compensation vector is constructed according to the positional deviation between the reference point and its candidate matching point in the second feature point set of the target.

[0049] ​The elements in the first feature point set and the second feature point set of the target can be position coordinate points. For example, when the target is a traffic light, the feature point set is composed of position coordinate points of the traffic light.

[0050] In some embodiments, the first feature point set and the second feature point set of the target are obtained in the following manner: obtaining first source data and second source data of the target; determining the first feature point set of the target according to the first source data, and determining the second feature point set of the target according to the second source data.

[0051] For example, when the first source data of the target is a map and the second source data of the target is an image captured by a camera, the point cloud data of the target is extracted from the map, and the point cloud data of the target is projected into the image to obtain the first feature point set of the target, and the target is detected from the image captured by the camera to obtain the second feature point set of the target.

[0052] For another example, when the first source data and the second source data of the target are two images captured by a camera, the target is detected from the two images respectively to obtain the first feature point set and the second feature point set of the target.

[0053] In step S110, each feature point in the first feature point set of the target is taken as a reference point to determine a candidate matching point thereof in the second feature point set, and then a reference deviation compensation vector is constructed according to a position deviation between the reference point and the candidate matching point thereof.

[0054] For example, it is assumed that the first feature point set of the target is composed of points a1, b1, c1 and d1, the second feature point set of the target is composed of points a2, b2, c2, d2, e2 and f2, a1 is taken as a reference point, a2 is determined as a candidate matching point thereof in the second feature point set, a deviation compensation vector L1 is determined according to a position deviation between the point a1 and the point a2, and similarly, b1, c1 and d1 are taken as reference points respectively to determine deviation compensation vectors L2, L3 and L4, thereby obtaining a plurality of deviation compensation vectors.

[0055] In some embodiments, the candidate matching point of the reference point in the second feature point set is determined in the following manner: taking the reference point as the center, searching for a feature point in the second feature point set within a maximum search distance range corresponding to the reference point, and taking the searched feature point as the candidate matching point of the reference point. By limiting the maximum search distance corresponding to the reference point, the influence of false interference points can be excluded, and the accuracy of the determined candidate matching point of the reference point can be improved.

[0056] In some embodiments, the maximum search distance corresponding to the reference point is determined according to the following manner: the maximum search distance corresponding to the reference point is determined according to the depth information corresponding to the reference point. The depth information reflects the distance between the feature points of the target and the observation device (such as a camera). In this embodiment, different maximum search distances can be set according to different depth information of the reference points. For example, a first deviation threshold Δd1 allowed in the real world can be set, and then the maximum search distance corresponding to the reference point is determined according to the depth information of the reference point, the camera parameters, and the first deviation threshold Δd1.

[0057] Through the above processing, different maximum search distances can be set for reference points with different depths, effectively solving the problem that the overall deviation of the target feature points is not linearly consistent due to inconsistent projection depths. Compared with the way of setting the same maximum search distance for all reference points, the influence of false interference points can be better excluded when searching for candidate matching points of the reference points, and the feature matching accuracy can be further improved.

[0058] Step S130: performing deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector.

[0059] The other feature points in the first feature point set are relative to the reference point or the reference deviation compensation vector. When the reference point is different, the other feature points in the first feature point set are also different.

[0060] For example, assuming that the first feature point set of the target is composed of points a1, b1, c1, and d1, when the deviation compensation is performed according to the deviation compensation vector L1 determined according to the reference point a1, the other feature points in the first feature point set are specifically b1, c1, and d1, and the deviation compensation is performed on the three feature points according to the deviation compensation vector L1, to obtain the deviation-compensated feature points b1', c1', and d1'; when the deviation compensation is performed according to the deviation compensation vector L2 determined according to the reference point b1, the other feature points in the first feature point set are specifically a1, c1, and d1, and the deviation compensation is performed on the three feature points according to the deviation compensation vector L2, to obtain the deviation-compensated feature points a1'', c1'', and d1''.

[0061] In step S130, the deviation-compensated feature points are obtained by performing deviation compensation on the other feature points in the first feature point set according to the reference deviation compensation vector, and the feature matching is performed based on the deviation-compensated feature points, which can effectively solve the problem of false matching caused by matching deviation.

[0062] Step S150: determining the candidate matching points of the deviation-compensated feature points in the second feature point set, to obtain a plurality of candidate matching point sets.

[0063] In the embodiments of the present disclosure, there are multiple sets composed of the feature points other than the reference point in the first feature point set. After the deviation compensation and the operation of finding candidate matching points for the feature points in these sets, multiple candidate matching point sets are obtained.

[0064] In some embodiments, the candidate matching points in the candidate matching point set do not include the candidate matching point of the reference point. In these embodiments, the set composed of the candidate matching points corresponding to the feature points other than the reference point in the first feature point set is taken as a candidate matching point set. For example, assuming that the first feature point set of the target is composed of points a1, b1, c1 and d1, when a1 is taken as the reference point, the feature points other than the reference point in the first feature point set are b1, c1 and d1, and the candidate matching point set corresponding thereto is set 1. When b1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, c1 and d1, and the candidate matching point set corresponding thereto is set 2. When c1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, b1 and d1, and the candidate matching point set corresponding thereto is set 3. When d1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, b1 and c1, and the candidate matching point set corresponding thereto is set 4.

[0065] In other embodiments, the candidate matching points in the candidate matching point set include the candidate matching point of the reference point. In these embodiments, the set composed of the candidate matching point of the reference point and the candidate matching points of the feature points other than the reference point in the first feature point set is taken as a candidate matching point set, thereby multiple candidate matching point sets can be obtained.

[0066] Step S170: determining the optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the multiple candidate matching point sets.

[0067] After the multiple candidate matching point sets are determined from the second feature point set, the optimal feature matching result of the first feature point set and the second feature point set is determined based on the first feature point set and the multiple candidate matching point sets.

[0068] In the embodiments of the present disclosure, by taking each feature point in the first feature point set as the reference point, the reference deviation compensation vector is constructed, the deviation compensation is performed on the feature points other than the reference point in the first feature point set based on the reference deviation compensation vector, and the candidate matching points are determined from the second feature point set based on the deviation-compensated feature points, which can improve the matching effect of the sparse feature points and effectively solve the problem of false matching caused by factors such as the close distance between the interference points and the to-be-matched points, the existence of abnormal feature point set, and matching deviation. Furthermore, it is helpful to improve the target detection and positioning accuracy, improve the safety of people and vehicles in the unmanned driving scene, and reduce the accident rate in the unmanned driving scene.

[0069] Figure 2 Flowchart for bias compensation of feature points according to some embodiments of the present disclosure. Figure 2 The flowchart is an optional implementation of step S130. As shown in the flowchart, the flowchart includes: Figure 2

[0070] Step S131: determining a dynamic bias compensation vector corresponding to each of the other feature points in the first feature point set according to the relationship between the depth information of the other feature points and the reference point and the reference bias compensation vector.

[0071] In some embodiments, a corresponding adjustment coefficient is determined according to the relationship between the depth information of the other feature points and the reference point, and the modulus of the dynamic bias compensation vector corresponding to each of the other feature points is determined according to the modulus of the adjustment coefficient and the reference bias compensation vector. Optionally, the product of the modulus of the adjustment coefficient and the reference bias compensation vector is taken as the modulus of the dynamic bias compensation vector corresponding to each of the other feature points. In addition, the direction of the reference bias compensation vector is taken as the direction of the dynamic bias compensation vector corresponding to each of the other feature points.

[0072] For example, assuming that the first feature point set of the target is composed of points a1, b1, c1, and d1, when bias compensation is performed according to the reference bias compensation vector L1 determined according to the reference point a1, the other feature points are specifically b1, c1, and d1, an adjustment coefficient is determined according to the relationship between the depth information of point b1 and the reference point a1, the modulus of the dynamic bias compensation vector corresponding to point b1 is determined according to the modulus of the adjustment coefficient and the bias compensation vector L1, and the direction of the reference bias compensation vector L1 is taken as the direction of the dynamic bias compensation vector corresponding to point b1. In the same way, the dynamic bias compensation vector corresponding to point c1 and the dynamic bias compensation vector corresponding to point d1 can be determined.

[0073] Step S132: bias compensating the other feature points according to the dynamic bias compensation vectors to obtain bias-compensated feature points.

[0074] In some embodiments, the other feature points are translated according to the dynamic bias compensation vectors, thereby obtaining the bias-compensated feature points.

[0075] In the embodiments of the present disclosure, by steps S131 to S132, different bias compensation vectors can be set according to the different depth information of the feature points, effectively solving the problem that the overall bias of the target feature points is not linearly consistent due to inconsistent projection depth, and the bias compensation effect is better than the way of setting the same bias compensation vector for other feature points, which can further improve the feature matching accuracy.

[0076] Figure 3 Flowchart for determining a candidate matching point set according to some embodiments of the present disclosure.​Figure 3 As shown in the figure, it is an optional implementation of step S150. As Figure 3 As shown in the figure, the flow includes:

[0077] Step S151: determining the maximum search distance corresponding to the deviation-compensated feature point according to the depth information corresponding to the deviation-compensated feature point.

[0078] The depth information corresponding to the deviation-compensated feature point is the depth information of the feature points other than the reference point in the first feature point set. For example, assuming that the first feature point set of the target is composed of points a1, b1, c1, and d1, when the other feature points are deviation-compensated with the feature point a1 as the reference point, the depth information of the deviation-compensated feature points is specifically the depth information of the feature points b1, c1, and d1.

[0079] In some embodiments, a second deviation threshold Δd2 allowed in the real world can be set, and then the maximum search distance corresponding to the deviation-compensated feature point is determined according to the depth information of the deviation-compensated feature point, the camera parameter, and the second deviation threshold Δd2.

[0080] Step S152: searching for the feature points in the second feature point set within the corresponding maximum search distance range with the deviation-compensated feature point as the center, and taking the searched feature points as the candidate matching points corresponding to the deviation-compensated feature point.

[0081] Exemplarily, the feature points in the second feature point set are searched within the square neighborhood with the deviation-compensated feature point as the center and the corresponding maximum search distance as the side length, and the searched feature points are taken as the candidate matching points corresponding to the deviation-compensated feature point. In addition, in specific implementation, the shape of the search neighborhood is not limited to a square, but can also be other shapes.

[0082] In the embodiments of the present disclosure, by limiting the maximum search distance corresponding to the deviation-compensated feature point, the influence of the false interference points can be excluded, the accuracy of the determined candidate matching points is improved, and thus the overall matching effect is improved.

[0083] Figure 4 A flowchart for determining the optimal feature matching result according to some embodiments of the present disclosure is shown. As shown in the figure, the flow includes: Figure 4 As shown in the figure, the flow includes:

[0084] Step S171: constructing a plurality of bipartite graphs according to the first feature point set and the plurality of candidate matching point sets.

[0085] In some embodiments, the candidate matching points in the candidate matching point set do not include the reference point. In these embodiments, the feature points in the first feature point set other than the reference point, which find the candidate matching points, are taken as one vertex set in the bipartite graph, and the corresponding candidate matching point set is taken as the other vertex set in the bipartite graph, thereby obtaining a bipartite graph. Further, according to the first feature point set and different candidate matching point sets, multiple bipartite graphs can be obtained.

[0086] In other embodiments, the candidate matching points in the candidate matching point set include the reference point. In these embodiments, all the feature points in the first feature point set, which find the candidate matching points, are taken as one vertex set in the bipartite graph, and the corresponding candidate matching point set is taken as the other vertex set in the bipartite graph, thereby obtaining a bipartite graph. Further, according to the first feature point set and different candidate matching point sets, multiple bipartite graphs can be obtained. For example, assuming that all the feature points in the first feature point set have candidate matching points, and assuming that there are 5 candidate matching point sets in total, 5 bipartite graphs can be obtained by combining the first feature point set with the 5 candidate matching point sets.

[0087] For example, assuming that the first feature point set of the target is composed of points a1, b1, c1, and d1, after constructing the reference deviation compensation vector with point a1 as the reference point to compensate the deviations of the other feature points, candidate matching points of points b1 and d1 are found, but no candidate matching point of point c1 is found. Then, in constructing the bipartite graph, points a1, b1, and d1 are taken as one vertex set of the bipartite graph, and the candidate matching points corresponding to the three points are taken as the other vertex set of the bipartite graph.

[0088] Step S172: determining the best complete matching result corresponding to each bipartite graph based on a bipartite graph matching algorithm.

[0089] In some embodiments, step S172 specifically includes: for each bipartite graph, determining the matching weight of each edge in the bipartite graph according to the distance between the deviation-compensated feature points of the bipartite graph and their candidate matching points; and determining the complete matching with the smallest sum of matching weights based on a bipartite graph matching algorithm, such as the KM algorithm, and taking it as the best complete matching result corresponding to the bipartite graph.

[0090] In an optional example, the distance between the deviation-compensated feature points of the bipartite graph and their candidate matching points is taken as the matching weight of each edge in the bipartite graph. In particular, for the case where the candidate matching point set contains the candidate matching point of the reference point, the matching weight of the edge composed of the reference point and its candidate matching point in each bipartite graph can be set to 0 or the same value. By selecting the distance between the deviation-compensated feature points and their candidate matching points as the matching weight of the edge in the bipartite graph, the matching effect can be better measured, and the matching accuracy can be improved.

[0091] In another optional example, the matching weight of each edge in the bipartite graph is determined according to the distance between the feature point and its candidate matching point after deviation compensation, and the weight adjustment coefficient. The weight adjustment coefficient is determined according to the depth information corresponding to the feature point. By considering the influence of the weight adjustment coefficient on the matching weight of the edge in the bipartite graph, the matching difference problem caused by the inconsistency of the depth information of the feature points can be effectively solved, and the matching accuracy can be improved.

[0092] For example, the matching weight of each edge in the bipartite graph is determined according to the following formula:

[0093] diff expect = |p candidate -p expect | + σ

[0094] wherein diff expect is the matching weight of the edge in the bipartite graph, |p candidate -p expect | is the distance between the feature point and its candidate matching point after deviation compensation, and σ is the weight adjustment coefficient.

[0095] Step S173: In the best complete matching result corresponding to each bipartite graph, the optimal feature matching result of the first feature point set and the second feature point set is determined.

[0096] After obtaining the best complete matching result corresponding to each bipartite graph, the best complete matching result corresponding to each bipartite graph is scored. According to the score, the optimal feature matching result is selected from the best complete matching result corresponding to each bipartite graph.

[0097] In some embodiments, the score of the best complete matching result corresponding to each bipartite graph is determined according to the matching accuracy and matching completeness indicators.

[0098] In deep learning, precision and recall are commonly used indicators for evaluating model performance. In the embodiments of the present disclosure, the matching accuracy and matching completeness indicators are introduced according to the concept of model evaluation in deep learning, and the matching result is scored based on this.

[0099] The matching accuracy index can be determined according to an exemplary manner as follows: for each bipartite graph corresponding optimal complete matching result, the sum of the maximum search distances of each deviation-compensated feature point corresponding to the optimal complete matching result is calculated and taken as a first parameter value; the sum of distances between each deviation-compensated feature point corresponding to the optimal complete matching result and its matching point (the matching point is selected from the candidate matching point corresponding to the deviation-compensated feature point by a bipartite graph matching algorithm such as KM) is calculated and taken as a second parameter value; and the matching accuracy index is determined according to the first parameter value and the second parameter value, wherein the matching accuracy index is positively correlated with the first parameter value, and the matching accuracy index is negatively correlated with the second parameter value.

[0100] For example, the matching accuracy index can be determined according to the following formula:

[0101]

[0102] wherein Precision is the matching accuracy index; diff expect is the distance between the deviation-compensated feature point and its matching point in the optimal complete matching result of the bipartite graph; ∑diff expect is the sum of distances between each deviation-compensated feature point and its matching point in the optimal complete matching result of the bipartite graph, that is, the second parameter value; ∑s is the sum of the maximum search distances of each deviation-compensated feature point in the optimal complete matching result of the bipartite graph, that is, the first parameter value; ∑(s-diff expect is the difference between the first parameter value and the second parameter value.

[0103] The matching completeness index can be determined according to an exemplary manner as follows: for each bipartite graph corresponding optimal complete matching result, the number of successfully matched feature points of the second feature point set in the optimal complete matching result is calculated and taken as a fourth parameter value; the maximum value of the number of feature points in the first feature point set and the number of feature points in the second feature point set is taken as a fifth parameter value; and the matching completeness index is determined according to the fourth parameter value and the fifth parameter value, wherein the matching completeness index is positively correlated with the fourth parameter value, and the matching completeness index is negatively correlated with the fifth parameter value.

[0104] For example, the matching completeness index can be determined according to the following formula:

[0105]

[0106] wherein Recall is the matching completeness index, size(K match) is the number of successfully matched feature points of the second feature point set in the best complete matching result of the bipartite graph, and max(size(PDeteet, PMatch)) is the maximum of the number of feature points in the first feature point set and the number of feature points in the second feature point set.

[0107] In an optional example, an F-score of the best complete matching result corresponding to each bipartite graph is calculated according to the matching precision and the matching completeness indicators, and the F-score is taken as the score of the best complete matching result corresponding to each bipartite graph.

[0108] In deep learning, the F-score is a commonly used indicator for evaluating the performance of a model. In the embodiments of the present disclosure, the matching precision and the matching completeness indicators are introduced according to the concept of model evaluation in deep learning, and the F-score is calculated based on the matching precision and the matching completeness indicators to score the matching result. Specifically, the F-score can be calculated according to the following formula, and taken as the score of the best complete matching result corresponding to each bipartite graph:

[0109]

[0110] wherein Score is the score of the best complete matching result corresponding to the bipartite graph, Precision is the matching precision indicator, Recall is the matching completeness indicator, and Measure is an adjustment coefficient.

[0111] In another optional example, an F-score of the best complete matching result corresponding to each bipartite graph is calculated according to the matching precision and the matching completeness indicators, and the score of the best complete matching result is determined according to the F-score and at least one of a matching penalty term and a matching reward term. The matching penalty term is set according to the overall deviation of the first feature point set in the matching result, and the matching reward term is set according to the matching priority of the successfully matched feature points of the second feature point set in the matching result. By scoring the matching result based on the F-score and at least one of the matching penalty term and the matching reward term, the matching effect can be better measured, the accuracy of the selected optimal matching feature matching result can be improved, and the sparse point feature matching effect can be improved, since factors such as matching deviation and whether the target with high priority is matched are considered.

[0112] For example, the score of the best complete matching result corresponding to each bipartite graph is calculated according to the following formula:

[0113]

[0114] wherein Score is the score of the best complete matching result corresponding to the bipartite graph, F-score, matching penalty term, To match the reward item, ∑limit is the sum of the maximum search distance corresponding to each reference point, ∑|V base |+|V drift | is the sum of the norm of the reference bias compensation vector involved in the best perfect matching result corresponding to the bipartite graph and the norm of the dynamic bias compensation vector, ε and μ are two coefficients set, Indicates that the successfully matched feature points in the second feature point set in the matching result are located in the priority matching point set B.

[0115] In the embodiments of the present disclosure, the optimal matching result of the first feature point set and the second feature point set is determined through the above steps. The feature matching method proposed in the embodiments of the present disclosure can well solve the problem of false matching caused by factors such as position deviation between feature point sets and close interference points, will not be affected by other points due to too far deviation, and will not match close interference points due to too close distance. Moreover, the feature matching method of the embodiments of the present disclosure also shows good matching effect when the first feature point set is abnormal or the second feature point set is abnormal.

[0116] Figure 5a The flowchart of the feature matching method according to some other embodiments of the present disclosure is shown. In the embodiments of the present disclosure, taking matching the projection point set of the inherent target in the map and the detection point set of the target in the image as an example, a detailed description is made. As shown in Figure 5a The method comprises:

[0117] Step S510: constructing a target attribute set.

[0118] In the embodiments of the present disclosure, the inherent target in the map is specifically a target known in advance in the map, such as a traffic light.

[0119] In some embodiments, the target attribute set constructed for the inherent target includes the projection point set M of the inherent target in the map, the projection depth set D of the target, and the priority matching point set B. The target attribute set is limited in the total field of view space of the camera.

[0120] Step S520: constructing a first feature point set and a second feature point set of the target.

[0121] In some embodiments, the projection point set M of the target focused in the field of view is taken as the first feature point set, and the projection point set M of the target focused in the field of view is taken as the first feature point set. The image detection of the target in the field of view generates a detection point set of the target, that is, a second feature point set of the target.

[0122] Step S530: taking each feature point in the first feature point set as a reference point, constructing a bias compensation vector.

[0123] In some embodiments, taking each feature point in the first feature point set as a reference point, a maximum search distance corresponding to the reference point is determined according to depth information corresponding to the reference point; a feature point in the second feature point set is searched within a range of the maximum search distance corresponding to the reference point with the reference point as the center, and the searched feature point is taken as a candidate matching point of the reference point in the second feature point set; and a reference deviation compensation vector is constructed according to a positional deviation between the reference point and the candidate matching point thereof.

[0124] For example, assuming that the first feature point set of the target is composed of points a1, b1, c1 and d1, the second feature point set of the target is composed of points a2, b2, c2, d2, e2 and f2, a1 is taken as a reference point, a2 is determined as a candidate matching point of a1 in the second feature point set, a deviation compensation vector L1 is determined according to a positional deviation between a1 and a2, and similarly, b1, c1 and d1 are taken as reference points respectively, deviation compensation vectors L2, L3 and L4 are determined, thereby obtaining a plurality of deviation compensation vectors.

[0125] Step S540: deviation compensation is performed on other feature points in the first feature point set.

[0126] In some embodiments, a dynamic deviation compensation vector corresponding to other feature points is determined according to a relationship between the other feature points in the first feature point set and depth information of the reference point, and the reference deviation compensation vector; and deviation compensation is performed on the other feature points according to the dynamic deviation compensation vector, to obtain feature points after deviation compensation.

[0127] For example, assuming that the first feature point set of the target is composed of points a1, b1, c1 and d1, and deviation compensation is performed on the first feature point set according to the deviation compensation vector L1 determined according to the reference point a1, the other feature points in the first feature point set are specifically b1, c1 and d1, and deviation compensation is performed on the three feature points respectively according to the deviation compensation vector L1, thereby obtaining feature points b1', c1' and d1' after deviation compensation.

[0128] wherein the deviation compensation performed on the other feature points according to the dynamic deviation compensation vector can be represented as:

[0129] p expect = p detect + v drift

[0130] wherein p expect is a feature point after deviation compensation, p detect is other feature points in the first feature point set except the reference point, and v drift is a dynamic deviation compensation vector.

[0131] Step S550: determining candidate matching points of the deviation-compensated feature point in the second feature point set, to obtain a plurality of candidate matching point sets.

[0132] In some embodiments, the candidate matching points of the deviation-compensated feature point in the second feature point set are determined according to the following manner: according to the depth information corresponding to the deviation-compensated feature point, a maximum search distance corresponding to the deviation-compensated feature point is determined; and within the range of the corresponding maximum search distance, the feature points in the second feature point set are searched with the deviation-compensated feature point as the center, and the searched feature points are taken as the candidate matching points corresponding to the deviation-compensated feature point.

[0133] In the embodiments of the present disclosure, there are a plurality of sets constituted by the feature points other than the reference point in the first feature point set, and after the deviation compensation and the operation of finding candidate matching points for the feature points in these sets, a plurality of candidate matching point sets are obtained.

[0134] In some embodiments, the candidate matching point set does not include the candidate matching point of the reference point. In these embodiments, the set composed of the candidate matching points corresponding to the feature points other than the reference point in the first feature point set is taken as a candidate matching point set. For example, assuming that the first feature point set of the target is composed of points a1, b1, c1, and d1, when a1 is taken as the reference point, the feature points other than the reference point in the first feature point set are b1, c1, and d1, and the candidate matching point set corresponding thereto is set 1, when b1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, c1, and d1, and the candidate matching point set corresponding thereto is set 2, when c1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, b1, and d1, and the candidate matching point set corresponding thereto is set 3, and when d1 is taken as the reference point, the feature points other than the reference point in the first feature point set are a1, b1, and c1, and the candidate matching point set corresponding thereto is set 4.

[0135] In other embodiments, the candidate matching point set includes the candidate matching point of the reference point. In these embodiments, the set composed of the candidate matching point of the reference point and the candidate matching points of the feature points other than the reference point in the first feature point set is taken as a candidate matching point set, so that a plurality of candidate matching point sets can be obtained.

[0136] Step S560: constructing a plurality of bipartite graphs according to the first feature point set and the plurality of candidate matching point sets.

[0137] In some embodiments, the candidate matching points do not include the reference point. In these embodiments, all feature points in the first feature point set except the reference point are taken as one vertex set in a bipartite graph, and one candidate matching point set is taken as another vertex set in the bipartite graph, so as to obtain a bipartite graph. By combining with different candidate matching point sets, multiple bipartite graphs can be obtained.

[0138] Step S570: determining the best complete matching result of each bipartite graph based on the KM algorithm.

[0139] In some embodiments, the matching weight of each edge in the bipartite graph is determined according to the distance between the feature point after deviation compensation and its candidate matching point, and the weight adjustment coefficient, the complete matching with the minimum sum of matching weights is determined based on the KM algorithm, and is taken as the best complete matching result corresponding to the bipartite graph.

[0140] For example, the matching weight of each edge in the bipartite graph is determined according to the following formula:

[0141] diff expect = |p candidate -p expect | + σ

[0142] Wherein, diff expect is the matching weight of the edge in the bipartite graph, |p candidate -p expect | is the distance between the feature point after deviation compensation and its candidate matching point, and σ is the weight adjustment coefficient.

[0143] Step S580: determining the optimal feature matching result of the first feature point set and the second feature point set in the best complete matching result of each bipartite graph according to the score.

[0144] In some embodiments, the F-score of the best complete matching result corresponding to each bipartite graph is calculated according to the matching accuracy and matching completeness indicators, and the F-score is taken as the score of the best complete matching result corresponding to each bipartite graph.

[0145] In other embodiments, the F-score of the best complete matching result corresponding to each bipartite graph is calculated according to the matching accuracy and matching completeness indicators, and the score of the best complete matching result is determined according to the F-score and at least one of the matching penalty term and the matching reward term. Wherein, the matching penalty term is set according to the overall deviation of the first feature point set in the matching result, and the matching reward term is set according to the matching priority of the successfully matched feature points in the second feature point set in the matching result.

[0146] For example, the score of the best complete matching result corresponding to each bipartite graph is calculated according to the following formula:

[0147]

[0148] wherein Score is the score of the optimal complete matching result of the bipartite graph, is the F-score, is the matching penalty term, is the matching reward term, and ∑limit is the sum of the maximum search distances corresponding to each reference point, ∑|V base |+|V drift | is the sum of the modulus of the reference bias compensation vector and the modulus of the dynamic bias compensation vector involved in the optimal complete matching result of the bipartite graph, and ε and μ are two coefficients set, indicates that the successfully matched feature points in the second feature point set in the matching result are located in the priority matching point set B.

[0149] In some embodiments, after determining the scores of the optimal complete matching results of the respective bipartite graphs, the optimal complete matching result with the highest score is selected as the optimal feature matching result of the first feature point set and the second feature point set.

[0150] In the embodiments of the present disclosure, the feature matching of the first feature point set and the second feature point set is realized through the above steps. Compared with the related art, the method of the embodiments of the present disclosure can improve the matching effect of sparse feature points and effectively solve the problem of false matching caused by factors such as the close distance between the interference points and the to-be-matched points, the existence of abnormal feature point sets, and matching bias.

[0151] Figure 5b is a schematic diagram for comparing the effects of the feature matching method according to some embodiments of the present disclosure and the feature matching method of the related art. In Figure 5b , the left graph is the matching result of the image detection traffic light and the map projection traffic light obtained based on the feature matching method of the related art, and the right graph is the matching result of the image detection traffic light and the map projection traffic light obtained based on the feature matching method in some embodiments of the present disclosure.

[0152] In Figure 5bIn the example, the correct matching should be: the image-detected red light L1 is matched with the map-projected red light a1, the image-detected red light L2 is matched with the map-projected red light a2, and the image-detected red light L3 is matched with the map-projected red light a3. However, due to factors such as projection deviation, the matching result is incorrect using the feature matching method of the related art, and the image-detected red light L2 is incorrectly matched to the map-projected red light a1 and the image-detected red light L3 is incorrectly matched to the map-projected red light a2. However, the feature matching method of the embodiment of the present disclosure obtains the correct matching result. As can be seen, compared with the related art, the feature matching method of the embodiment of the present disclosure has better matching accuracy. In addition, experiments prove that the embodiment of the present disclosure can exhibit good matching effect in more complex and special scenarios such as inaccurate calibration, inaccurate detection set or matching set, or falling into local optimum.

[0153] Figure 6 FIG. 1 is a structural schematic diagram of a feature matching device according to some embodiments of the present disclosure. As shown in FIG. 1, the device includes a construction module 610, a deviation compensation module 620, a first determination module 630, and a second determination module 640. Figure 6

[0154] The construction module 610 is configured to take each feature point in a first feature point set of a target as a reference point, and construct a reference deviation compensation vector according to a position deviation between the reference point and a candidate matching point of the reference point in a second feature point set of the target.

[0155] In some embodiments, the construction module 610 determines the candidate matching point of the reference point in the second feature point set according to the following manner: the construction module 610 takes the reference point as the center, searches for a feature point in the second feature point set within a maximum search distance range corresponding to the reference point, and takes the searched feature point as the candidate matching point of the reference point. By limiting the maximum search distance corresponding to the reference point, the influence of false interference points can be excluded, and the accuracy of the determined candidate matching point of the reference point can be improved.

[0156]

[0157] ​​In some embodiments, the construction module 610 determines the maximum search distance corresponding to the reference point according to the following manner: the construction module 610 determines the maximum search distance corresponding to the reference point according to the depth information corresponding to the reference point. The depth information reflects the distance between the feature points of the target and the observation device (such as a camera). In this embodiment, different maximum search distances can be set according to different depth information of the reference point. For example, a first deviation threshold Δd1 allowed in the real world can be set, and then the maximum search distance corresponding to the reference point is determined according to the depth information of the reference point, the camera parameters, and the first deviation threshold Δd1.

[0158] Through the above processing, different maximum search distances can be set for reference points with different depths, effectively solving the problem that the overall deviation of the target feature points is not linearly consistent due to inconsistent projection depths. Compared with the manner of setting the same maximum search distance for all reference points, the influence of false interference points can be better excluded when searching for candidate matching points of the reference points, and the feature matching accuracy is further improved.

[0159] The deviation compensation module 620 is configured to perform deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector.

[0160] The other feature points in the first feature point set are relative to the reference point or the reference deviation compensation vector. When the reference point is different, the other feature points in the first feature point set are also different.

[0161] In some embodiments, the deviation compensation module 620 determines a dynamic deviation compensation vector corresponding to the other feature points according to the relationship between the depth information of the other feature points in the first feature point set and the reference point, and the reference deviation compensation vector; and the deviation compensation module 620 performs deviation compensation on the other feature points according to the dynamic deviation compensation vector to obtain the deviation-compensated feature points.

[0162] For example, it is assumed that the first feature point set of the target is composed of points a1, b1, c1, and d1. When the deviation compensation is performed according to the deviation compensation vector L1 determined according to the reference point a1, the other feature points in the first feature point set are specifically b1, c1, and d1. The deviation compensation is performed on these three feature points according to the deviation compensation vector L1, and the deviation-compensated feature points b1', c1', and d1' are obtained. When the deviation compensation is performed according to the deviation compensation vector L2 determined according to the reference point b1, the other feature points in the first feature point set are specifically a1, c1, and d1. The deviation compensation is performed on these three feature points according to the deviation compensation vector L2, and the deviation-compensated feature points a1'', c1'', and d1'' are obtained.

[0163] The first determining module 630 is configured to determine candidate matching points of the feature points after deviation compensation in the second feature point set, so as to obtain multiple candidate matching point sets.

[0164] In some embodiments, the first determining module 630 determines the maximum search distance corresponding to the feature point after deviation compensation based on the depth information corresponding to the feature point after deviation compensation; with the feature point after deviation compensation as the center, it searches for feature points in the second feature point set within the corresponding maximum search distance range, and uses the searched feature points as candidate matching points corresponding to the feature point after deviation compensation, thereby obtaining multiple candidate matching point sets.

[0165] The second determining module 640 is configured to determine the optimal feature matching result of the first feature point set and the second feature point set based on the first feature point set and multiple candidate matching point sets.

[0166] In some embodiments, the second determining module 640 constructs multiple bipartite graphs based on the first feature point set and the multiple candidate matching point sets; determines the best complete matching result corresponding to each bipartite graph based on the bipartite graph matching algorithm; and determines the optimal feature matching result of the first feature point set and the second feature point set among the best complete matching results corresponding to each bipartite graph.

[0167] In this embodiment of the disclosure, feature matching of the first feature point set and the second feature point set is achieved through the above-described apparatus. Compared with related technologies, the apparatus of this embodiment can improve the matching effect of sparse feature points and effectively solve the problem of incorrect matching caused by factors such as interference points being close to the point to be matched, anomalies in the feature point set, and matching deviation.

[0168] Figure 7 This is a block diagram illustrating a feature matching apparatus according to other embodiments of the present disclosure.

[0169] like Figure 7 As shown, the feature matching apparatus 700 includes a memory 710 and a processor 720 coupled to the memory 710. The memory 710 is used to store instructions for executing embodiments of the feature matching method. The processor 720 is configured to execute the feature matching method in any of the embodiments of this disclosure based on the instructions stored in the memory 710.

[0170] Figure 8 This is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0171] like Figure 8 As shown, the computer system 800 can be represented in the form of a general computing device. The computer system 800 includes a memory 810, a processor 820, and a bus 830 connecting different system components.

[0172] The memory 810 may, for example, include system memory, non-volatile memory, and / or the like. The system memory may, for example, store operating systems, application programs, a Boot Loader, and other programs. The system memory may include volatile memory, such as random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, store instructions to perform at least one of the feature matching methods. The non-volatile memory includes, but is not limited to, magnetic storage, optical storage, flash memory, and / or the like.

[0173] The processor 820 can be implemented with a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, and / or the like. Accordingly, each module, such as the construction module, the bias compensation module, can be implemented by a central processing unit (CPU) running instructions in a memory to perform corresponding steps, or by a dedicated circuit performing the corresponding steps.

[0174] The bus 830 can use any of a variety of bus structures. For example, the bus structure includes, but is not limited to, an industry standard architecture (ISA) bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus.

[0175] The computer system 800 can further include an input / output interface 840, a network interface 850, a storage interface 860, and / or the like. These interfaces 840, 850, 860, and the memory 810 and the processor 820 can be connected through the bus 830. The input / output interface 840 can provide a connection interface for a display, a mouse, a keyboard, and / or the like input / output device. The network interface 850 provides a connection interface for various networking devices. The storage interface 860 provides a connection interface for an external storage device, such as a floppy disk, a U disk, an SD card, and / or the like.

[0176] Here, various aspects of the disclosure are described with reference to flowcharts and / or block diagrams of methods, apparatuses and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowcharts and / or block diagrams can be implemented by computer readable program instructions.

[0177] These computer readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, so that the instructions executed by the processor produce the device that implements the functions specified in one or more blocks of the flowcharts and / or block diagrams.

[0178] These computer readable program instructions can also be stored in computer readable memory that can direct computers to function in a certain manner, so as to generate a manufacture product, including instructions to implement the functions specified in one or more blocks in the flowchart and / or block diagram.

[0179] Figure 9 A structural schematic diagram of an unmanned vehicle according to some embodiments of the present disclosure. As shown, the unmanned vehicle 900 includes a feature matching device 910. Figure 9

[0180] The unmanned vehicle 900 is installed with at least one sensor for collecting data, so as to perform target detection and the like based on the collected data. Exemplarily, the sensor installed on the unmanned vehicle is one or more of a camera, a laser radar, or a millimeter wave radar.

[0181] When the unmanned vehicle 900 performs target detection, feature matching needs to be performed based on the feature matching device 910. For example, when a traffic light is detected and positioned, the image data of the traffic light collected by the camera needs to be matched with the point cloud data of the traffic light extracted from the map based on the feature matching device 910.

[0182] In some embodiments, the feature matching device 910 is configured to, taking each feature point in a first feature point set of a target as a reference point, construct a reference deviation compensation vector according to a position deviation between the reference point and a candidate matching point of the target in a second feature point set; perform deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector; determine the candidate matching point of the feature point after the deviation compensation in the second feature point set, to obtain a plurality of candidate matching point sets; and determine an optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the plurality of candidate matching point sets.

[0183] Exemplarily, when the target is a traffic light, the first and second feature point sets are composed of position coordinate points of the traffic light. Exemplarily, the first feature point set is a feature point of the traffic light projected on a map, and the second feature point set is a feature point set of the traffic light detected in an image.

[0184] In the embodiments of the present disclosure, the feature matching of the first feature point set and the second feature point set is realized by the above device. Compared with the related art, the device of the embodiments of the present disclosure can improve the matching effect of sparse feature points, and effectively solve the problem of false matching caused by factors such as that the distance between the interference point and the to-be-matched point is relatively close, that the feature point set is abnormal, and that the matching deviation is large. Further, it is helpful to improve the target detection and positioning accuracy, improve the safety of vehicles in the unmanned driving scene, and reduce the accident rate in the unmanned driving scene.

[0185] ​The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both software and hardware aspects.

[0186] By the feature matching method, device and unmanned vehicle in the above embodiments, the matching effect of sparse feature points can be improved, and the problem of false matching caused by factors such as that the distance between the interference point and the to-be-matched point is relatively close, the feature point set is abnormal, and the matching deviation is effectively solved.

[0187] Thus far, the feature matching method, device and unmanned vehicle according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details well known in the art are not described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein according to the above description.

Claims

1. A feature matching method, comprising: constructing, for each feature point in a first feature point set of a target, a reference deviation compensation vector for the feature point based on a positional deviation between the feature point and a candidate matching point of the feature point in a second feature point set of the target, wherein the first feature point set of the target is a position coordinate point set of the target obtained by projecting point cloud data of the target to an image, and the second feature point set of the target is a position coordinate point set of the target detected from an image captured by a camera, or the first feature point set of the target and the second feature point set of the target are position coordinate point sets of the target respectively detected from two images captured by the camera; performing deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector for the feature point; determining candidate matching points of the deviation-compensated feature points in the second feature point set to obtain a candidate matching point set for the feature point, and regarding candidate matching point sets for all feature points as a plurality of candidate matching point sets; determining an optimal feature matching result of the first feature point set and the second feature point set based on the first feature point set and the plurality of candidate matching point sets.

2. The feature matching method of claim 1, wherein, The determining of the optimal feature matching result of the first feature point set and the second feature point set based on the first feature point set and the plurality of candidate matching point sets comprises: constructing a plurality of bipartite graphs based on the first feature point set and the plurality of candidate matching point sets; determining a best complete matching result corresponding to each bipartite graph based on a bipartite graph matching algorithm; determining the optimal feature matching result of the first feature point set and the second feature point set from the best complete matching results corresponding to the bipartite graphs.

3. The feature matching method of claim 1, wherein, The constructing of the deviation compensation vector further comprises: searching for feature points in the second feature point set within a maximum search distance range corresponding to the feature point and taking the searched feature points as candidate matching points of the feature point in the second feature point set.

4. The feature matching method of claim 2, wherein, The constructing of the reference deviation compensation vector further comprises: determining the maximum search distance corresponding to the feature point based on depth information of the feature point.

5. The feature matching method of claim 1, wherein, The performing of the deviation compensation on other feature points in the first feature point set according to the reference deviation compensation vector comprises: determining a dynamic deviation compensation vector corresponding to each of the other feature points based on a relationship between the other feature points and depth information of the feature point and the reference deviation compensation vector; performing deviation compensation on the other feature points according to the dynamic deviation compensation vector to obtain deviation-compensated feature points.

6. The feature matching method of claim 5, wherein, The determining of candidate matching points of the deviation-compensated feature points in the second feature point set comprises: determining a maximum search distance corresponding to each of the deviation-compensated feature points based on depth information of the deviation-compensated feature points; searching for feature points in the second feature point set within the corresponding maximum search distance range with the deviation-compensated feature points as the center and taking the searched feature points as candidate matching points corresponding to the deviation-compensated feature points.

7. The feature matching method of claim 2, wherein, The determining of the best complete matching result corresponding to each bipartite graph based on the bipartite graph matching algorithm comprises: For each bipartite graph, a matching weight of each edge in the bipartite graph is determined according to a distance between the bias-compensated feature point corresponding to the bipartite graph and the candidate matching point thereof; The complete matching with the smallest sum of matching weights is taken as the optimal complete matching result corresponding to the bipartite graph.

8. The feature matching method of claim 7, wherein, The matching weight of each edge in the bipartite graph is determined according to a distance between the bias-compensated feature point corresponding to the bipartite graph and the candidate matching point thereof, and a weight adjustment parameter, wherein the weight adjustment parameter is determined according to depth information corresponding to the feature point. The optimal feature matching result of the first feature point set and the second feature point set is determined from the optimal complete matching result corresponding to each bipartite graph, including:

9. The feature matching method of claim 2, wherein, A score of the optimal complete matching result corresponding to each bipartite graph is determined according to a matching accuracy index and a matching completeness index; The optimal feature matching result is selected from the optimal complete matching result corresponding to each bipartite graph according to the score. The score of the optimal complete matching result corresponding to each bipartite graph is determined according to a matching accuracy index and a matching completeness index, including:

10. The feature matching method of claim 9, wherein, For the optimal complete matching result corresponding to each bipartite graph, a sum of maximum search distances of the bias-compensated feature points corresponding to the optimal complete matching result is calculated and taken as a first parameter value; A sum of distances between the bias-compensated feature points corresponding to the optimal complete matching result and the matching points thereof is calculated and taken as a second parameter value; The matching accuracy index is determined according to the first parameter value and the second parameter value, wherein the matching accuracy index is positively correlated with the first parameter value, and the matching accuracy index is negatively correlated with the second parameter value. The score of the optimal complete matching result corresponding to each bipartite graph is determined according to a matching accuracy index and a matching completeness index, including:

11. The feature matching method of claim 9, wherein, For the optimal complete matching result corresponding to each bipartite graph, a number of successfully matched feature points of the second feature point set in the optimal complete matching result is calculated and taken as a fourth parameter value; A maximum value of a number of feature points of the first feature point set and a number of feature points of the second feature point set is taken as a fifth parameter value; The matching completeness index is determined according to the fourth parameter value and the fifth parameter value, wherein the matching completeness index is positively correlated with the fourth parameter value, and the matching completeness index is negatively correlated with the fifth parameter value. The score of the optimal complete matching result corresponding to each bipartite graph is determined according to a matching accuracy index and a matching completeness index, including:

12. The feature matching method of claim 8, wherein, An F-score of the optimal complete matching result corresponding to each bipartite graph is calculated according to the matching accuracy index and the matching completeness index; The score of the optimal complete matching result corresponding to each bipartite graph is determined according to the F-score. The score of the optimal complete matching result corresponding to each bipartite graph is determined according to the F-score, including:

13. The feature matching method of claim 11, wherein, ​ determine a score of the optimal complete matching result according to the F-score, and at least one of a matching penalty term and a matching reward term, wherein the matching penalty term is set according to overall deviation of the first feature point set in the matching result, and the matching reward term is set according to matching priority of the successfully matched feature points in the second feature point set in the matching result.

14. The feature matching method of claim 1, further comprising: obtaining first source data and second source data of a target; determining a first feature point set of the target according to the first source data, and determining a second feature point set of the target according to the second source data.

15. The feature matching method of claim 14, wherein, The first source data of the target is a map, and the second source data of the target is an image captured by a camera.

16. A feature matching apparatus, comprising: a construction module configured to take each feature point in a first feature point set of a target as a reference point, and construct a reference deviation compensation vector under the reference point according to a position deviation between the reference point and a candidate matching point of the target in a second feature point set of the target, wherein the first feature point set of the target is a position coordinate point set of the target obtained by projecting point cloud data of the target to an image, and the second feature point set of the target is a position coordinate point set of the target detected from an image captured by a camera; or the first feature point set of the target and the second feature point set of the target are position coordinate point sets of the target respectively detected from two images captured by a camera; a deviation compensation module configured to compensate deviations of other feature points in the first feature point set according to the reference deviation compensation vector under the reference point; a first determination module configured to determine candidate matching points of the target in the second feature point set for the other feature points after the deviation compensation, to obtain a candidate matching point set under the reference point, and take all candidate matching point sets under the reference points as a plurality of candidate matching point sets; a second determination module configured to determine an optimal feature matching result of the first feature point set and the second feature point set according to the first feature point set and the plurality of candidate matching point sets.

17. A feature matching apparatus, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute a feature matching method according to any one of claims 1 to 15 based on instructions stored in the memory.

18. A computer readable storage medium having computer program instructions stored thereon, the instructions being executed by a processor to implement a feature matching method according to any one of claims 1 to 15.

19. An unmanned vehicle, comprising: the feature matching apparatus according to claim 16 or 17.

Images