Registration method of point cloud data

Abstract

The present application discloses a method for registration of point cloud data, which belongs to the field of computer technology. The method includes: acquiring first point cloud data and second point cloud data; based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, coarsely registering the first point cloud data and the second point cloud data, and outputting a matching point pair relationship; for the matching point pair relationship, filtering out external points according to confidence, the external points are unmatched points in the first point cloud data and the second point cloud data; performing precise registration of the first point cloud data and the second point cloud data after filtering out the external points, and determining the final relative pose of the second point cloud and the first point cloud. The method enables the deletion of unmatched points in the first point cloud data and the second point cloud data, and can improve the accuracy of point cloud registration of the first point cloud data and the second point cloud data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for registering point cloud data. Background Art

[0002] With the continuous development of computer technology, it is becoming more and more important to accurately determine the position and posture information of electronic devices. For example, during the movement of electronic devices, only by accurately determining the position and posture information of electronic devices can the electronic devices be controlled to reach the destination smoothly.

[0003] In the related art, the posture information of the electronic device is usually determined by the Iterative Closest Point (ICP) algorithm. Specifically, the first point cloud data obtained by scanning the electronic device is compared with the second point cloud data corresponding to the positioning position of the electronic device in the electronic map, and the matching point set between the first point cloud data and the second point cloud data is determined. Based on the matching point set, the first point cloud data is transformed into the coordinate system of the second point cloud data, and the error function between the transformed first point cloud data and the second point cloud data is estimated. If the error function value is greater than a threshold, the above operation is iterated until the error function value obtained is less than the threshold, and then the posture information corresponding to the first point cloud data is determined based on the current matching point set and the posture information corresponding to the second point cloud data. Summary of the invention

[0004] The embodiment of the present application provides a point cloud data registration method, which can improve the accuracy of point cloud registration. The technical solution is as follows:

[0005] In one aspect, a method for registering point cloud data is provided, the method comprising:

[0006] Acquire first point cloud data and second point cloud data;

[0007] Based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, coarsely aligning the first point cloud data with the second point cloud data, and outputting a matching point pair relationship;

[0008] For the matching point pair relationship, filtering out outliers according to confidence, wherein the outliers are points that do not match between the first point cloud data and the second point cloud data;

[0009] The first point cloud data and the second point cloud data after filtering out the external points are precisely aligned to determine the final relative pose of the second point cloud and the first point cloud.

[0010] In one aspect, a point cloud data registration device is provided, the device comprising:

[0011] An acquisition module, used for acquiring first point cloud data and second point cloud data;

[0012] A first registration module, configured to perform a rough registration on the first point cloud data and the second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, and output a matching point pair relationship;

[0013] A filtering module, configured to filter out outliers according to the confidence level of the matching point pair relationship, wherein the outliers are points that do not match between the first point cloud data and the second point cloud data;

[0014] The second registration module is used to perform precise registration on the first point cloud data and the second point cloud data after filtering out the external points, and determine the final relative position and posture of the second point cloud and the first point cloud.

[0015] In a possible implementation, the first registration module includes:

[0016] A key point determination unit, configured to determine a plurality of first key points from the first point cloud data and a plurality of second key points from the plurality of second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, wherein the key points are points in the point cloud data that meet preset conditions;

[0017] A registration unit is used to register the multiple first key points and the multiple second key points, and output the matching point pair relationship.

[0018] In a possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the key point determination unit is used to extract features from the multiple first points in the first point cloud data and the multiple second points in the second point cloud data to obtain first features of the multiple first points and second features of the multiple second points; determine a first key point feature from the obtained multiple first features, and determine a second key point feature from the obtained multiple second features; determine a first point corresponding to the determined first key point feature as a first key point, and determine a second point corresponding to the determined second key point feature as a second key point.

[0019] In a possible implementation, the key point determination unit is used to, for any first key point feature, determine a first point corresponding to the first key point feature as a first key point, and determine a second point corresponding to the second key point feature matching the first key point feature as a second key point, if there is a second key point feature matching the first key point feature among the determined second key point features; or,

[0020] The key point determination unit is used to determine, for any second key point feature, if there is a first key point feature matching the second key point feature in the determined first key point features, determine the second point corresponding to the second key point feature as the second key point, and determine the first point corresponding to the first key point feature matching the second key point feature as the first key point.

[0021] In a possible implementation, the first point cloud data is point cloud data obtained by scanning an automatic vehicle, and the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map; the automatic vehicle is equipped with a posture determination device, and the posture determination device is used to determine the posture of the automatic vehicle; the key point determination unit is used to obtain the first posture of the automatic vehicle and the second posture corresponding to the second point cloud data, and the first posture is the posture determined by the posture determination device of the automatic vehicle; for any first key point feature, based on the first posture and the second posture, determine the distance between the second point corresponding to each second key point feature and the first point corresponding to the first key point feature, and when there is a second key point feature similar to the first key point feature among multiple second key point features whose distance does not exceed the first distance threshold, the first point corresponding to the first key point feature is determined as the first key point.

[0022] In a possible implementation, the point cloud features of the first point cloud data include the first features of the multiple first key points, and the point cloud features of the second point cloud data include the second features of the multiple second key points; the registration unit is used to determine multiple candidate matching point pair relationships and matching parameters of the multiple candidate matching point pair relationships based on the first features of the multiple first key points and the second features of the multiple second key points, and the matching parameters are used to represent the feature similarity of the matched first key points and the second key points in the matching point pair relationships; based on the matching parameters of the multiple candidate matching point pair relationships, output the matching point pair relationships.

[0023] In a possible implementation, the registration unit is used to output the matching point pair relationship corresponding to the highest matching parameter; or,

[0024] The registration unit is used to determine the matching point pair relationship corresponding to the highest matching parameter; for each matched first key point and second key point in the determined matching point pair relationship, determine the first position relationship corresponding to the first key point, the first position relationship is used to represent the position relationship between the first key point and at least one other first key point; determine the second position relationship corresponding to the second key point, the second position relationship is used to represent the position relationship between the second key point and at least one other second key point, the at least one other second key point is a second key point that matches the at least one other first key point; when the first position relationship is the same as the second position relationship, retain the first key point and the second key point; when the first position relationship is different from the second position relationship, discard the first key point and the second key point; output the filtered matching point pair relationship.

[0025] In a possible implementation, the filtering module includes:

[0026] A determination unit, configured to determine a first relative position and posture of the first point cloud and the second point cloud based on the matching point pair relationship;

[0027] A mapping unit, configured to map the first point cloud data and the second point cloud data into the same coordinate system based on the first relative posture;

[0028] A filtering unit is used to filter out the external points according to the confidence level based on the mapped first point cloud data and the second point cloud data.

[0029] In a possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the filtering unit is used to determine the confidence of the multiple first points in the mapped first point cloud data based on the distance between the first point and a target second point, where the target second point is a second point among the multiple second points that matches the first point, and the distance is inversely proportional to the confidence; and filter out the first point in the first point cloud data whose confidence is lower than a first confidence threshold.

[0030] In a possible implementation manner, the device further includes:

[0031] The filtering module is further configured to retain, for any first point in the mapped first point cloud data, the first point in the first point cloud data if there is no second point matching the first point among the plurality of second points;

[0032] a determination module, configured to, after accurately registering the first point cloud data and the second point cloud data after filtering out the external points, determine a final relative pose between the second point cloud and the first point cloud, and then determine a third pose corresponding to the first point cloud data based on the final relative pose and the second pose corresponding to the second point cloud data in the electronic map;

[0033] An updating module is used to update the electronic map based on the first point cloud data after filtering the external points and the third posture.

[0034] In a possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the filtering unit is used to determine the confidence of the multiple second points in the mapped second point cloud data based on at least one of whether there is a first point matching the second point in the multiple first points, matching parameters between the first point matching the second point and the second point, and the number of first points whose distance from the second point does not exceed a first distance threshold; and filter out second points in the second point cloud data whose confidence is lower than a second confidence threshold.

[0035] In a possible implementation, the first point cloud data is point cloud data scanned by an automatic vehicle, the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map, and the device further includes:

[0036] An updating module is used to replace the second point cloud data in the electronic map with the filtered second point cloud data.

[0037] In a possible implementation, the second registration module is used to determine the matching point pair relationship between the first point cloud data and the second point cloud data after filtering out the external points; based on the matching point pair relationship, construct a covariance matrix, perform SVD decomposition on the covariance matrix, and output a rotation matrix and a translation matrix; based on the rotation matrix and the translation matrix, map the first point cloud data and the second point cloud data from which the external points are filtered out to the same coordinate system, and determine the error function value between the first point cloud data and the second point cloud data after mapping; if the error function value is greater than the target function value, iteratively execute the process of determining the matching point pair relationship and the error function value until the obtained error function value is not greater than the target function value, and then determine the relative posture corresponding to the current error function value as the final relative posture of the first point cloud and the second point cloud.

[0038] On the one hand, an automatic vehicle is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the point cloud data registration method in any possible implementation manner as described above.

[0039] On the one hand, a computer-readable storage medium is provided, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the operations performed by the point cloud data registration method in any possible implementation manner as described above.

[0040] On the one hand, a computer program or a computer program product is provided, wherein the computer program or the computer program product comprises: a computer program code, wherein when the computer program code is executed by a computer, the computer implements the operations performed by the point cloud data registration method in any possible implementation manner as described above.

[0041] The point cloud data registration method provided in the embodiment of the present application can first perform a coarse registration on the first point cloud data and the second point cloud data, filter out the unmatched points in the first point cloud data and the second point cloud data through the coarse registration result, and then perform a fine registration on the filtered first point cloud data and the second point cloud data, which can improve the accuracy of the point cloud registration of the first point cloud data and the second point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0044] Figure 2 is a flow chart of a point cloud data registration method provided in an embodiment of the present application;

[0045] Figure 3 is a flow chart of a point cloud data registration method provided in an embodiment of the present application;

[0046] Figure 4 is a flow chart of a point cloud data registration method provided in an embodiment of the present application;

[0047] Figure 5 It is a structural schematic diagram of a point cloud data registration device provided in an embodiment of the present application;

[0048] Figure 6 is a structural schematic diagram of another point cloud data registration device provided in an embodiment of the present application;

[0049] Figure 7 is a schematic diagram of the structure of an automatic vehicle provided in an embodiment of the present application;

[0050] Figure 8 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, a first order can be called a second order, and a second order can be called a first order without departing from the scope of this application.

[0053] The terms "at least one", "multiple", "each" and "any" used in this application, at least one includes one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple. For example, multiple orders include 3 orders, and each refers to each of the 3 orders, and any refers to any one of the 3 orders, which can be the first, the second or the third.

[0054] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the point cloud data, positioning location, posture information, etc. involved in this application are all obtained with full authorization. Moreover, the above information and data are processed and used in big data application scenarios, and cannot be identified to any natural person or have a specific association with him.

[0055] The point cloud data registration method provided in the embodiment of the present application can be applied to various scenarios such as autonomous driving, food delivery, express delivery, map update, etc. The embodiment of the present application does not limit the application scenarios, and only exemplifies the autonomous driving and map update scenarios.

[0056] During the driving process, the automatic vehicle can scan the surrounding environment to obtain the first point cloud data, obtain the second point cloud data corresponding to the positioning position from the electronic map based on the positioning position of the automatic vehicle, and determine the position and posture information of the automatic vehicle based on the first point cloud data and the second point cloud data. However, when the automatic vehicle scans to obtain the first point cloud data, it may introduce more noise points, or the electronic map sometimes fails to synchronize the changes in the driving environment in time, so that the environment indicated by the second point cloud data is inconsistent with the actual environment, resulting in the low accuracy of the position and posture information of the determined automatic vehicle. If the method provided in the embodiment of the present application is adopted, the first point cloud data and the second point cloud data can be roughly aligned first, and at least one of the introduced noise points and the scene change points in the electronic map can be filtered out based on the rough alignment result, and then the filtered first point cloud data and the second point cloud data can be precisely aligned, which can improve the accuracy of the point cloud alignment of the first point cloud data and the second point cloud data, so that the position and posture information of the automatic vehicle can be determined more accurately.

[0057] At present, a high-precision map can be constructed based on the position information of the automatic vehicle and the point cloud data scanned by the automatic vehicle. Only when the position information of the automatic vehicle is determined to be accurate can the accuracy of the constructed high-precision map be guaranteed. Therefore, how to accurately determine the position information of the automatic vehicle is an important part of the production of high-precision maps. If the method provided in the embodiment of the present application is adopted, not only can the position information of the automatic vehicle be accurately determined, but also the point cloud data scanned by the automatic vehicle can be filtered and processed, thereby ensuring the accuracy of the high-precision map constructed based on the position information of the automatic vehicle and the point cloud data scanned by the automatic vehicle.

[0058] In some embodiments, the registration method of point cloud data provided in the embodiments of the present application is performed by an electronic device. Optionally, the electronic device includes an automatic vehicle. In some embodiments, the automatic vehicle includes vehicles traveling on the ground (e.g., cars, trucks, buses, etc.), and may also include vehicles traveling in the air (e.g., drones, airplanes, helicopters, etc.), and may also include vehicles traveling on water or in water (e.g., ships, submarines, etc.). The automatic vehicle may or may not accommodate one or more passengers. In addition, the automatic vehicle can be applied to unmanned delivery fields, such as express logistics fields, takeaway food delivery fields, etc. Optionally, the electronic device includes an automatic vehicle and a server, which may be a server, or a server cluster consisting of several servers, or a cloud computing service center.

[0059] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes an automatic vehicle 101 and a server 102, and the automatic vehicle 101 and the server 102 are connected via a wireless or wired network.

[0060] In some embodiments, the automatic vehicle 101 obtains an electronic map from the server 102 and drives automatically based on the electronic map. In some embodiments, the point cloud data scanned by the automatic vehicle 101 and the position information of the automatic vehicle 101 when scanning the point cloud data are used to update the electronic map. In some embodiments, the server 102 is used to update the electronic map; in some embodiments, the server 102 is used to determine the position information of the automatic vehicle 101, etc. The embodiments of the present application do not limit the specific interaction between the automatic vehicle 101 and the server 102.

[0061] Figure 2 1 is a flow chart of a method for registering point cloud data provided by an embodiment of the present application. The present application embodiment is illustrated by taking an automatic vehicle as an example of an execution subject. The embodiment includes:

[0062] 201. The automatic vehicle obtains first point cloud data and second point cloud data.

[0063] Among them, the first point cloud data and the second point cloud data can be any two frames of point cloud data. In some embodiments, the electronic device includes a point cloud scanning component, and the first point cloud data and the second point cloud data are point cloud data obtained by the point cloud scanning component in the automatic vehicle scanning the surrounding environment. For example, the first point cloud data is a frame of point cloud data obtained at the current moment, and the second point cloud data is a frame of point cloud data obtained at the previous moment. In some embodiments, the electronic device includes a point cloud scanning component, and the first point cloud data is point cloud data obtained by the point cloud scanning component in the automatic vehicle scanning the surrounding environment. The second point cloud data is the point cloud data corresponding to the positioning position of the automatic vehicle in the electronic map.

[0064] Optionally, the point cloud scanning component is a radar. For example, a laser radar, a millimeter wave radar, etc. Among them, when the point cloud scanning component scans the surrounding environment, it can only scan the environment in front, or scan the environment in front and behind, etc. The embodiment of the present application does not limit the scanning range of the point cloud scanning component. In addition, the point cloud scanning component can only scan when the automatic vehicle is moving, avoiding scanning the surrounding environment multiple times at the same position to obtain multiple identical first point cloud data.

[0065] In some embodiments, the automatic vehicle includes a positioning component, which is used to locate the current geographical location of the automatic vehicle. The positioning component can be a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China, the Grenades system of Russia, or the Galileo system of the European Union. In the embodiment of the present application, the positioning position of the automatic vehicle is the position determined by the positioning component.

[0066] In some embodiments, the second point cloud data is point cloud data centered on the positioning position in the electronic map. The second point cloud data may be point cloud data within a rectangular area centered on the positioning position, or point cloud data within a circular area centered on the positioning position, etc. The second point cloud data is not limited in the embodiments of the present application.

[0067] In some embodiments, the first point cloud data and the second point cloud data include a plurality of points, each point being configured with information such as coordinates and intensity.

[0068] 202. The automatic vehicle performs a rough registration on the first point cloud data and the second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, and outputs a matching point pair relationship.

[0069] Among them, coarse registration refers to a simple registration of the first point cloud data and the second point cloud data, and this registration method is characterized by being fast. In addition, when the embodiment of the present application performs coarse registration on the first point cloud data and the second point cloud data, the point cloud features of the first point cloud data and the point cloud features of the second point cloud data are referred to, so that points with similar features in the first point cloud data and the second point cloud data are matched as a pair.

[0070] Among them, the matching point pair relationship is used to represent the matching relationship between multiple first points in the first point cloud data and multiple second points in the second point cloud data. The first point and the second point with a correct matching relationship refer to the points corresponding to the same position point in the environment. For example, the first point cloud data includes point A, point B, point C and point D obtained by scanning door 1. Among them, point A is the point corresponding to the upper left corner of door 1, point B is the point corresponding to the upper right corner of door 1, point C is the point corresponding to the lower left corner of door 1, and point D is the point corresponding to the lower right corner of door 1. The second point cloud data also includes point E, point F, point G and point H obtained by scanning door 1. Among them, point E is the point corresponding to the upper left corner of door 1, point F is the point corresponding to the upper right corner of door 1, point G is the point corresponding to the lower left corner of door 1, and point H is the point corresponding to the lower right corner of door 1. At this time, point A is point E is a matching point pair, point B and point F are a matching point pair, point C and point G are a matching point pair, and point D and point H are a matching point pair. The matching point pair relationship is used to indicate that point A is point E is a matching point pair, point B and point F are a matching point pair, point C and point G are a matching point pair, and point D and point H are a matching point pair.

[0071] 203. The automatic vehicle filters out outliers according to the confidence level for the matching point pair relationship, where the outliers are points that do not match between the first point cloud data and the second point cloud data.

[0072] Although the result of the rough registration is not necessarily optimal, it has a certain degree of accuracy. Therefore, the confidence of the points in the first point cloud data and the second point cloud data can be determined by referring to the relationship between the matching point pairs, and the external points can be filtered out according to the confidence. In an ideal situation where no noise points are introduced or the scene in the electronic map has not changed, the points in the first point cloud data and the second point cloud data can be matched one by one. However, due to the introduction of noise points or the change of the scene in the electronic map, there are unmatched points in the first point cloud data and the second point cloud data. Therefore, by filtering the unmatched points in the first point cloud data and the second point cloud data, at least some of the noise points and the points where the scene has changed can be filtered out.

[0073] 204. The automatic vehicle performs precise registration on the first point cloud data and the second point cloud data after filtering out the external points, and determines the final relative position of the second point cloud and the first point cloud.

[0074] Since in the above step 203, the unmatched points in the first point cloud data and the second point cloud data are deleted, the problem of mismatching between the first point cloud data and the second point cloud data is effectively alleviated, and the first point cloud data and the second point cloud data can be well registered. At this time, a precise registration method can be used to obtain a more accurate registration result, thereby obtaining a more accurate relative pose of the first point cloud and the second point cloud.

[0075] The point cloud data registration method provided in the embodiment of the present application can first perform a coarse registration on the first point cloud data and the second point cloud data, filter out the unmatched points in the first point cloud data and the second point cloud data through the coarse registration result, and then perform a fine registration on the filtered first point cloud data and the second point cloud data, which can improve the accuracy of the point cloud registration of the first point cloud data and the second point cloud data.

[0076] Figure 3 1 is a flow chart of a method for registering point cloud data provided by an embodiment of the present application. The present application embodiment is illustrated by taking an automatic vehicle as an example of an execution subject. The embodiment includes:

[0077] 301. An automatic vehicle obtains first point cloud data and second point cloud data, where the first point cloud data includes a plurality of first points, and the second point cloud data includes a plurality of second points.

[0078] The step 301 is the same as the above step 201 and will not be described in detail here.

[0079] 302. The automatic vehicle determines a plurality of first key points from the first point cloud data and a plurality of second key points from the second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data.

[0080] An autonomous vehicle is driving on the road, and the surrounding environment includes bushes, shoulders, street lights, billboards, roadside buildings, etc. Among them, the bushes and shoulders are continuous, and there is no obvious difference between bushes or shoulders at different locations. Therefore, the point cloud data obtained by scanning bushes or shoulders at different locations is not obviously different. Therefore, in terms of determining the position and posture of the unmanned vehicle based on the scanned point cloud data, the point cloud data obtained by scanning bushes or shoulders is not of reference value. Since buildings usually have large differences in size, shape, etc., the point cloud data obtained by scanning buildings at different locations are usually obviously different. Therefore, in terms of determining the position and posture of the unmanned vehicle based on the scanned point cloud data, the point cloud data obtained by scanning buildings is of reference value.

[0081] It can be seen that some points in the point cloud data have reference value, while others do not. In the above step 302, determining a plurality of first key points from the first point cloud data and determining a plurality of second key points from the second point cloud data is actually to select points with reference value from the point cloud data.

[0082] In some embodiments, the automatic vehicle selects key points from the point cloud data based on preset conditions, and the key points are points in the point cloud data that meet the preset conditions. The embodiments of the present application do not limit the preset conditions, and only use the following embodiments as examples for exemplary description.

[0083] In a possible implementation, the point features of the points with reference value are different from the point features of the points without reference value, and the key points are screened out from the point cloud data based on the point features of the points in the point cloud data. The automatic vehicle determines a plurality of first key points from the first point cloud data and a plurality of second key points from the second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, including: extracting features from the plurality of first points in the first point cloud data and the plurality of second points in the second point cloud data to obtain first features of the plurality of first points and second features of the plurality of second points; determining a first key point feature from the plurality of first features obtained, and determining a second key point feature from the plurality of second features obtained; determining a first point corresponding to the determined first key point feature as a first key point, and determining a second point corresponding to the determined second key point feature as a second key point.

[0084] Key point features can be regarded as features of key points, or features of points with reference value. Among them, the automatic car can be preset with key point features, and the automatic car determines the first key point feature from multiple first features, which can include: the automatic car selects a first feature similar to the key point feature from multiple first features and determines it as the first key point feature. Alternatively, the first key point feature is determined from multiple first features and the second key point feature is determined from multiple second features by a key point extraction model. Optionally, the automatic car performs the above-mentioned steps of "extracting features from multiple first points in the first point cloud data and multiple second points in the second point cloud data to obtain first features of multiple first points and second features of multiple second points; determining the first key point feature from the obtained multiple first features, and determining the second key point feature from the obtained multiple second features; determining the first point corresponding to the determined first key point feature as the first key point, and determining the second point corresponding to the determined second key point feature as the second key point" through the key point extraction model. The key point extraction model is obtained based on the training of annotated point cloud samples, and the points with reference value in the point cloud samples are annotated. Since the key point extraction model is obtained based on the labeled point cloud samples, the key point extraction model will pay attention to whether the point has reference value when extracting features from the points in the point cloud data. Therefore, the extracted features can indicate whether the point has reference value, and thus based on the multiple first features and the multiple second features, multiple first key points among the multiple first points and multiple second key points among the multiple second points can be determined.

[0085] It should be noted that after obtaining the first key point and the second key point, the automatic vehicle can perform a rough registration based on the first key point and the second key point, that is, determine the matching point pair relationship between the first key point and the second key point. Therefore, it is important to determine whether the first key point and the second key point are matched. In order to ensure that the first key point and the second key point are matched, when determining the first key point and the second key point, not only the condition of having a reference value must be met, but also the matching key point must be found in the key points of another point cloud data.

[0086] Optionally, the automatic vehicle determines the first point corresponding to the determined first key point feature as the first key point, and determines the second point corresponding to the determined second key point feature as the second key point, including: for any first key point feature, if there is a second key point feature matching the first key point feature in the determined second key point features, the automatic vehicle determines the first point corresponding to the first key point feature as the first key point, and determines the second point corresponding to the second key point feature matching the first key point feature as the second key point. Alternatively, for any second key point feature, if there is a first key point feature matching the second key point feature in the determined first key point features, the automatic vehicle determines the second point corresponding to the second key point feature as the second key point, and determines the first point corresponding to the first key point feature matching the second key point feature as the first key point.

[0087] Among them, the second key point feature that matches the first key point feature means that: the first key point feature is highly similar to the second key point feature. For example, the second key point feature that matches the first key point feature is determined, including: the second key point feature that has a similarity with the first key point feature that exceeds a first similarity threshold value. Among them, the first similarity threshold value can be any value, for example, 80%, 90%, 95%, etc., and the embodiment of the present application does not limit the first similarity threshold value.

[0088] In some embodiments, the automatic vehicle is equipped with a posture determination device, which is used to determine the posture information of the automatic vehicle. Optionally, the posture determination device includes a positioning component and an inertial sensor, wherein the positioning component is used to determine the position information of the automatic vehicle, and the inertial sensor is used to determine the posture information of the automatic vehicle. It should be noted that, under normal circumstances, there is a certain error in the posture information determined by the positioning component and the inertial sensor. Although there is a certain error in the posture information determined by the positioning component and the inertial sensor, we can use the posture information determined by them as a priori value to assist in finding matching key points.

[0089] Optionally, for any first key point feature, the automatic vehicle determines the first point corresponding to the first key point feature as the first key point if there is a second key point feature matching the first key point feature among the determined second key point features, including: obtaining the first pose of the automatic vehicle and the second pose corresponding to the second point cloud data, the first pose being the pose determined by a pose determination device of the automatic vehicle; for any first key point feature, determining the distance between the second point corresponding to each second key point feature and the first point corresponding to the first key point feature based on the first pose and the second pose, and determining the first point corresponding to the first key point feature as the first key point if there is a second key point feature similar to the first key point feature among multiple second key point features whose distance does not exceed a first distance threshold.

[0090] Wherein, based on the first posture and the second posture, determining the distance between the second point corresponding to each second key point feature and the first point corresponding to the first key point feature, including: based on the first posture and the second posture, mapping the first point corresponding to each first key point feature and the second point corresponding to each second key point feature to the same coordinate system, and determining the distance between the mapped first point and the second point corresponding to each second key point feature.

[0091] Optionally, the first point corresponding to each first key point feature is mapped to the coordinate system of the second point cloud data, and the distance between the mapped first point and the second point corresponding to each second key point feature is determined. Optionally, the second point corresponding to each second key point feature is mapped to the coordinate system of the first point cloud data, and the distance between the first point and the mapped second point corresponding to each second key point feature is determined. If the first pose is accurate, then the first point and the second point of the true match will be in the same position, but the first pose is inaccurate, therefore, there will be a certain distance between the first point and the second point of the true match, and the distance is caused by the error of the first pose, therefore, the distance will not be too large, and the distance between the matched first key point and the second key point will not exceed the first distance threshold.

[0092] The first distance threshold may be any threshold. Optionally, the first distance threshold is related to the error of the position and posture determination device, the larger the error of the position and posture determination device, the larger the first distance threshold, and the smaller the error of the position and posture determination device, the smaller the first distance threshold.

[0093] Since the distance between the matched first key point and the second key point does not exceed the first distance threshold, when determining the second key point that matches the first key point, a range can also be first determined based on the first distance threshold, and the second key point that matches the first key point can be found in the range. Optionally, the automatic vehicle maps the first point corresponding to each first key point feature to the coordinate system of the second point cloud data based on the first pose and the second pose, and for each mapped first point, a circular area is determined with the position of the first point as the center and the first distance threshold as the radius, and the second point located in the circular area is determined. If there is a second point in the circular area whose second key point feature is similar to the first key point feature of the first point, the first point is determined as the first key point, and the second point is determined as the second key point.

[0094] 303. The automatic vehicle aligns the multiple first key points and the multiple second key points, and outputs a relationship between matching point pairs.

[0095] The above step 303 can use any method to align multiple first key points and multiple second key points. The embodiment of the present application does not limit this, and only uses the following two examples for illustrative purposes.

[0096] In a possible implementation, the multiple first key points and the multiple second key points are aligned by an ICP algorithm. The automatic vehicle aligns the multiple first key points and the multiple second key points and outputs a matching point pair relationship, including: taking the first key point and the second key point with the smallest Euclidean distance as a matching point pair, and obtaining a matching point pair relationship between the multiple first key points and the multiple second key points; determining an error function of the matching point pair relationship, and if the error function value is greater than a threshold, iterating the above operation until the error function value obtained is less than the threshold, and outputting the current matching point pair relationship.

[0097] In another possible implementation, the matching point pair relationship is determined based on the similarity between the first key point and the second key point. The automatic vehicle determines the matching point set between the plurality of first key points and the plurality of second key points, including: determining the matching point pair relationship between the plurality of first key points and the plurality of second key points based on the similarity between the first key point features of the plurality of first key points and the second key point features of the plurality of second key points.

[0098] Among them, the higher the similarity between the first key point feature of the first key point and the second key point of the second key point, the more the first key point matches the second key point. Optionally, based on the similarity between the first key point features of the plurality of first key points and the second key point features of the plurality of second key points, a matching point set between the plurality of first key points and the plurality of second key points is determined, including: based on the first key point features of the plurality of first key points and the second key point features of the plurality of second key points, a similarity matrix is ​​constructed, the similarity matrix being used to represent the similarity between the first key feature of each first key point and the key point feature of each second key point; based on the similarity matrix, the matching point set is solved.

[0099] For example, the automatic car obtains 1000 first key points and 1000 second key points. The first key point features of the 1000 first key points form a first matrix of 1000×n dimensions, where n is the dimension of the first key point features. The second key point features of the 1000 second key points form a second matrix of n×1000 dimensions, where n is the dimension of the second key point features. The first matrix and the second matrix are subjected to operations such as dot multiplication to obtain a 1000×1000 dimensional similarity matrix. Among them, the first column of the similarity matrix is ​​the similarity between the first key point feature of the first first key point and the second key point feature of each second key point, that is, the similarity between the first first key point and each second key point. The second column is the similarity between the first key point feature of the second first key point and the second key point feature of each second key point, that is, the similarity between the second first key point and each second key point, and so on.

[0100] In one possible implementation, an automatic vehicle determines a set of matching points between a plurality of first key points and a plurality of second key points, including: determining a plurality of candidate matching point pair relationships and matching parameters of the plurality of candidate matching point pair relationships based on first features of the plurality of first key points and second features of the plurality of second key points, the matching parameters being used to represent feature similarities between the matched first key points and the second key points in the matching point pair relationships; and outputting the matching point pair relationships based on the matching parameters of the plurality of candidate matching point pair relationships.

[0101] The candidate matching point pair relationship may be determined by any method, such as an exhaustive method. The more similar the features of the first key point and the second key point matched in the matching point pair relationship are, the more the first key point matches the second key point. The matching parameter may be determined based on the feature similarity of the first key point and the second key point matched in the matching point pair relationship. The matching parameter is proportional to the similarity.

[0102] In some embodiments, the matching point pair relationship corresponding to the highest matching parameter may be directly used. The automatic vehicle outputs the matching point pair relationship based on the matching parameters of multiple candidate matching point pair relationships, including: the automatic vehicle outputs the matching point pair relationship corresponding to the highest matching parameter.

[0103] In some other embodiments, it is also necessary to screen the matching point pair relationships corresponding to the highest matching parameters to remove the wrong matching point pair relationships. The automatic vehicle outputs the matching point pair relationships based on the matching parameters of multiple candidate matching point pair relationships, including: the automatic vehicle determines the matching point pair relationship corresponding to the highest matching parameter; for each matched first key point and second key point in the determined matching point pair relationship, determines the first position relationship corresponding to the first key point, the first position relationship is used to represent the position relationship between the first key point and at least one other first key point, determines the second position relationship corresponding to the second key point, the second position relationship is used to represent the position relationship between the second key point and at least one other second key point, the at least one other second key point is the second key point that matches the at least one other first key point, when the first position relationship is the same as the second position relationship, retains the first key point and the second key point, when the first position relationship is different from the second position relationship, discards the first key point and the second key point; outputs the screened matching point pair relationships.

[0104] For example, the matching point pair relationship corresponding to the highest matching parameter indicates 4 matching point pairs, the first matching point pair includes the first key point A and the second key point E, the second matching point pair includes the first key point B and the second key point F, the third matching point pair includes the first key point C and the second key point G, and the fourth matching point pair includes the first key point D and the second key point H. Among them, the first key point A is the point corresponding to the upper left corner of door 1, the first key point B is the point corresponding to the upper right corner of door 1, the first key point C is the point corresponding to the lower left corner of door 1, and the first key point D is the point corresponding to the lower right corner of door 1. The second key point E is the point corresponding to the upper left corner of door 1, the second key point F is the point corresponding to the upper right corner of door 1, the second key point G is the point corresponding to the lower left corner of door 1, and the second key point H is the point corresponding to the lower right corner of door 2.

[0105] Determine the first positional relationship between the first key point A and the first key point B and the first key point C, determine the second positional relationship between the second key point E and the second key point F and the second key point G, the first positional relationship is the same as the second positional relationship, and the first matching point pair is the correct matching point pair. Determine the first positional relationship between the first key point D and the first key point A and the first key point B, determine the second positional relationship between the second key point H and the second key point E and the second key point F, the first positional relationship is different from the second positional relationship, the fourth matching point pair is an incorrect matching point pair, and delete the matching point pair.

[0106] 304. The automatic vehicle determines a first relative position and posture of the first point cloud and the second point cloud based on the matching point pair relationship.

[0107] The automatic vehicle may use any method to determine the first relative pose of the first point cloud and the second point cloud. For example, the matching point pair relationship may be processed by the SVD (Singular Value Decomposition) decomposition method to obtain the first relative pose between the first point cloud data and the second point cloud data, or the matching point pair relationship may be processed by the ICP algorithm to obtain the first relative pose of the first point cloud and the second point cloud, which is not limited in the present embodiment of the application.

[0108] 305. The automatic vehicle maps the first point cloud data and the second point cloud data into the same coordinate system based on the first relative posture.

[0109] Wherein, the automatic vehicle maps the first point cloud data and the second point cloud data to the same coordinate system based on the first relative posture, including: the automatic vehicle maps the first point cloud data to the coordinate system of the second point cloud data based on the first relative posture; or, the automatic vehicle maps the second point cloud data to the coordinate system of the first point cloud data based on the first relative posture; or, the automatic vehicle maps the first point cloud data and the second point cloud data to any coordinate system based on the first relative posture. The embodiment of the present application does not limit this, and it is only necessary to ensure that the first point cloud data and the second point cloud data are ultimately in the same coordinate system.

[0110] 306. The automatic vehicle filters out outliers based on the mapped first point cloud data and the second point cloud data according to confidence, where the outliers are points that do not match the first point cloud data and the second point cloud data.

[0111] In a possible implementation, the automatic vehicle filters out external points according to confidence based on the mapped first point cloud data and the second point cloud data, including: for multiple first points in the mapped first point cloud data, determining the confidence of the first point based on the distance between the first point and a target second point, the target second point being a second point among the multiple second points that matches the first point, and the distance is inversely proportional to the confidence; filtering out the first point in the first point cloud data whose confidence is lower than a first confidence threshold.

[0112] The first relative pose can be regarded as a preliminary estimated relative pose. If the first relative pose is accurate, then based on the first relative pose, after the first point cloud data and the second point cloud data are mapped to the same coordinate system, the points in the first point cloud data will coincide with the points in the second point cloud data. If there is an error in the first relative pose, then based on the first relative pose, after the first point cloud data and the second point cloud data are mapped to the same coordinate system, there will be a certain distance between the points in the first point cloud data and the corresponding points in the second point cloud data. However, if a point in the first point cloud data matches another point in the second point cloud data, and the distance between the two points is far, it means that the two points are not points of the same name, and it may be that the noise point in the first point cloud data is mistakenly considered to match a point in the second point cloud data. Therefore, the point in the first point cloud data can be deleted.

[0113] It should be noted that the embodiment of the present application is only an exemplary description of determining confidence based on distance and filtering external points based on confidence, while in another embodiment, external points can be filtered directly based on distance. In a possible implementation, for any point in the mapped first point cloud data, the automatic vehicle determines a second point that matches the first point among multiple second points, and deletes the first point in the first point cloud data when the distance between the first point and the determined second point exceeds a second distance threshold. The second distance threshold can be any value, and the embodiment of the present application does not limit the second distance threshold. Optionally, the second distance threshold is a set value, and the second distance threshold is 5 meters, 10 meters, etc. Optionally, the second distance threshold is determined based on the error of the first pose information, and the larger the error of the first pose information, the larger the second distance threshold.

[0114] It should be noted that the environment in which the autonomous vehicle is located may be somewhat different from the environment described by the electronic map. For example, when the electronic map is constructed, the roadside of Road 1 is empty, but when the autonomous vehicle is currently passing through Road 1, there is a billboard on the roadside of Road 1. Therefore, the second point cloud data of the electronic map does not include the point cloud data corresponding to the billboard, but the first point cloud data of the autonomous vehicle includes the point cloud data corresponding to the billboard. At this time, the second point cloud data does not include the second point that matches the first point corresponding to the billboard. In a possible implementation, the method also includes: for any first point in the mapped first point cloud data, if there is no second point matching the first point in the multiple second points, retaining the first point in the first point cloud data; after accurately aligning the first point cloud data and the second point cloud data after filtering out external points and determining the final relative pose of the second point cloud and the first point cloud, determining a third pose corresponding to the first point cloud data based on the final relative pose and the second pose corresponding to the second point cloud data in the electronic map; and updating the electronic map based on the first point cloud data after filtering out external points and the third pose.

[0115] In a possible implementation, the automatic vehicle filters out the external point according to the confidence level based on the mapped first point cloud data and the second point cloud data, including: for multiple second points in the mapped second point cloud data, determining the confidence level of the second point based on at least one of whether there is a first point matching the second point in the multiple first points, matching parameters between the first point matching the second point and the second point, and the number of first points whose distance from the second point does not exceed a first distance threshold; and filtering out the second points in the second point cloud data whose confidence level is lower than a second confidence threshold.

[0116] The environment in which the autonomous vehicle is located may be somewhat different from the environment described by the electronic map. For example, when the electronic map was constructed, a billboard was placed on the side of Road 1, so the electronic map included point cloud data obtained by scanning the billboard. However, the billboard was later removed, and when the autonomous vehicle passed through Road 1, the point cloud data obtained by scanning did not include the point cloud data of the billboard. Therefore, it is necessary to delete the point cloud data of the billboard from the second point cloud data. That is, delete the second point that does not find a matching first point, delete the second point that does not have a first point around it, and so on.

[0117] The first confidence threshold and the second confidence threshold may be any value, and the embodiment of the present application does not limit the first confidence threshold and the second confidence threshold.

[0118] It should be noted that the embodiment of the present application is only an illustrative explanation of determining confidence and filtering external points based on confidence. In another embodiment, the external points in the second point cloud data can be filtered directly based on at least one of whether there is a first point matching the second point among the multiple first points, the matching parameters between the first point matching the second point and the second point, and the number of first points whose distance from the second point does not exceed a first distance threshold.

[0119] For example, it is possible to directly determine whether there is a first point within a range where the distance from the second point does not exceed a third distance threshold. If so, it means that the second point is not a point where the scene changes, and the second point is retained. If not, it means that the second point is a point where the scene changes, and the second point is discarded.

[0120] The third distance threshold is any value, and the embodiment of the present application does not limit the third distance threshold. Optionally, the third distance threshold is an empirical value.

[0121] In some embodiments, the method further includes: the automatic vehicle replacing the second point cloud data in the electronic map with the filtered second point cloud data.

[0122] 307. The automatic vehicle performs precise matching on the first point cloud data after filtering out the external points and the second point cloud data to determine the final relative position and posture of the second point cloud and the first point cloud.

[0123] It should be noted that any precise matching method can be used in the embodiment of the present application to precisely match the first point cloud data with the second point cloud data after filtering out the external points, and determine the final relative position of the second point cloud and the first point cloud. The embodiment of the present application is only exemplified by taking the ICP algorithm as an example.

[0124] In a possible implementation, the first point cloud data and the second point cloud data after filtering out the external points are precisely aligned to determine the final relative pose of the second point cloud and the first point cloud, including: determining the matching point pair relationship between the first point cloud data and the second point cloud data after filtering out the external points; constructing a covariance matrix based on the matching point pair relationship, performing SVD decomposition on the covariance matrix, and outputting a rotation matrix and a translation matrix; mapping the first point cloud data and the second point cloud data after filtering out the external points to the same coordinate system based on the rotation matrix and the translation matrix, and determining the error function value between the first point cloud data and the second point cloud data after mapping; if the error function value is greater than the target function value, iteratively executing the process of determining the matching point pair relationship and the error function value until the obtained error function value is not greater than the target function value, and then determining the relative pose corresponding to the current error function value as the final relative pose of the first point cloud and the second point cloud.

[0125] In order to ensure the high freshness of the electronic map, the method provided in the embodiment of the present application can be applied to the electronic map update project. In some embodiments, in order to maintain the accuracy and consistency of the electronic map construction, it is necessary to eliminate the cumulative error as much as possible. Usually, in order to meet the high precision and high richness of the high-precision map, the construction of the electronic map will use a single batch of multiple passes or multiple batches of multiple passes of point cloud data to construct accurate closed-loop constraints, which can effectively eliminate cumulative errors. Therefore, the embodiment of the present application applies the posture information determination method to the optimization link of the electronic map, effectively ensuring the correctness of the closed-loop constraints, increasing the number of closed-loop constraints, and ultimately improving the global and local accuracy of the electronic map.

[0126] It should be noted that in the embodiment of the present application, the filtered first point cloud data can be considered to be point cloud data with noise points removed. The first point cloud data is data that can more accurately represent the scanned environment. Therefore, the electronic map can be updated based on the filtered first point cloud data.

[0127] In another embodiment, the embodiment of the present application can also filter out the point cloud data in the electronic map to update the electronic map. For example, for any second point in the second point cloud data in the electronic map, if there is no first point matching the second point among the plurality of first points, the automatic vehicle deletes the second point from the second point cloud data, so that the second point cloud data in the electronic map is updated.

[0128] The point cloud data registration method provided in the embodiment of the present application can first perform a coarse registration on the first point cloud data and the second point cloud data, filter out the unmatched points in the first point cloud data and the second point cloud data through the coarse registration result, and then perform a fine registration on the filtered first point cloud data and the second point cloud data, which can improve the accuracy of the point cloud registration of the first point cloud data and the second point cloud data.

[0129] The present application embodiment Figure 4 The registration method of point cloud data is illustrated as follows. Figure 4 As shown, first point cloud data obtained by scanning an automatic vehicle is obtained; based on the positioning position of the automatic vehicle, second point cloud data corresponding to the positioning position is obtained from an electronic map; based on the positioning component and inertial sensor of the automatic vehicle, an initial posture of the automatic vehicle is determined, and the first point cloud data, the second point cloud data and the initial posture are input into a key point extraction model, and the key point extraction model outputs multiple first key points, a first feature of each first key point, multiple second key points and a second feature of each second key point. Based on the first feature of each first key point and the second feature of each second key point, determine the matching point pair relationship between the multiple first key points and the multiple second key points; perform confidence screening on the matching point pair relationship, and delete the erroneous matching point pairs in the matching point pair relationship, for example, based on the positional relationship between the first key point and other first key points and the positional relationship between the second key point and other second key points in the matching point pair, find out the erroneous matching point pairs and delete them; based on the screened matching point pair relationship, determine the posture information of the automatic vehicle, and based on the posture information, screen out noise points in the first point cloud data or points in the scene change part in the second point cloud data; use the point cloud data with noise filtering and scene change part removal to determine the posture information of the automatic vehicle through precise matching methods such as ICP.

[0130] In addition, the embodiments of the present application also compare the method provided in the present application with the method in the related art, and the comparison is shown in Table 1.

[0131]

[0132] Table 1

[0133] Figure 5 is a schematic diagram of the structure of a point cloud data registration device provided in an embodiment of the present application, see Figure 5 , the device comprises:

[0134] An acquisition module 501 is used to acquire first point cloud data and second point cloud data;

[0135] A first registration module 502, configured to perform a rough registration on the first point cloud data and the second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, and output a matching point pair relationship;

[0136] A filtering module 503 is used to filter out outliers according to the confidence level for the matching point pair relationship, where the outliers are points that do not match between the first point cloud data and the second point cloud data;

[0137] The second registration module 504 is used to perform precise registration on the first point cloud data and the second point cloud data after filtering out the external points, and determine the final relative position and posture of the second point cloud and the first point cloud.

[0138] In a possible implementation, the first registration module 502 includes:

[0139] A key point determination unit 5021 is used to determine a plurality of first key points from the first point cloud data and a plurality of second key points from the plurality of second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, wherein the key points are points in the point cloud data that meet preset conditions;

[0140] The registration unit 5022 is used to register the multiple first key points and the multiple second key points, and output the matching point pair relationship.

[0141] In a possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the key point determination unit 5021 is used to extract features from the multiple first points in the first point cloud data and the multiple second points in the second point cloud data to obtain first features of the multiple first points and second features of the multiple second points; determine a first key point feature from the obtained multiple first features, and determine a second key point feature from the obtained multiple second features; determine a first point corresponding to the determined first key point feature as a first key point, and determine a second point corresponding to the determined second key point feature as a second key point.

[0142] In a possible implementation, the key point determination unit 5021 is used to, for any first key point feature, determine a first point corresponding to the first key point feature as a first key point, and determine a second point corresponding to the second key point feature matching the first key point feature as a second key point, when there is a second key point feature matching the first key point feature in the determined second key point features; or,

[0143] The key point determination unit 5021 is used to determine, for any second key point feature, if there is a first key point feature matching the second key point feature in the determined first key point features, the second point corresponding to the second key point feature as the second key point, and the first point corresponding to the first key point feature matching the second key point feature as the first key point.

[0144] In a possible implementation, the first point cloud data is point cloud data obtained by scanning an automatic vehicle, and the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map; the automatic vehicle is equipped with a posture determination device, and the posture determination device is used to determine the posture of the automatic vehicle; the key point determination unit 5021 is used to obtain the first posture of the automatic vehicle and the second posture corresponding to the second point cloud data, and the first posture is the posture information determined by the posture determination device of the automatic vehicle; for any first key point feature, based on the first posture and the second posture, determine the distance between the second point corresponding to each second key point feature and the first point corresponding to the first key point feature, and when there is a second key point feature similar to the first key point feature among multiple second key point features whose distance does not exceed the first distance threshold, the first point corresponding to the first key point feature is determined as the first key point.

[0145] In a possible implementation, the point cloud features of the first point cloud data include the first features of the multiple first key points, and the point cloud features of the second point cloud data include the second features of the multiple second key points; the alignment unit 5022 is used to determine multiple candidate matching point pair relationships and matching parameters of the multiple candidate matching point pair relationships based on the first features of the multiple first key points and the second features of the multiple second key points, and the matching parameters are used to represent the feature similarity of the matched first key points and the second key points in the matching point pair relationships; based on the matching parameters of the multiple candidate matching point pair relationships, output the matching point pair relationships.

[0146] In a possible implementation, the registration unit 5022 is used to output the matching point pair relationship corresponding to the highest matching parameter; or,

[0147] The registration unit 5022 is used to determine the matching point pair relationship corresponding to the highest matching parameter; for each matched first key point and second key point in the determined matching point pair relationship, determine the first position relationship corresponding to the first key point, the first position relationship is used to represent the position relationship between the first key point and at least one other first key point; determine the second position relationship corresponding to the second key point, the second position relationship is used to represent the position relationship between the second key point and at least one other second key point, the at least one other second key point is a second key point that matches the at least one other first key point; when the first position relationship is the same as the second position relationship, retain the first key point and the second key point; when the first position relationship is different from the second position relationship, discard the first key point and the second key point; output the filtered matching point pair relationship.

[0148] In a possible implementation, the filtering module 503 includes:

[0149] A determining unit 5031, configured to determine a first relative position and posture of the first point cloud and the second point cloud based on the matching point pair relationship;

[0150] A mapping unit 5032, configured to map the first point cloud data and the second point cloud data to the same coordinate system based on the first relative posture;

[0151] The filtering unit 5033 is used to filter out the external points according to the confidence level based on the mapped first point cloud data and the second point cloud data.

[0152] In one possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the filtering unit 5033 is used to determine the confidence of the multiple first points in the mapped first point cloud data based on the distance between the first point and a target second point, where the target second point is a second point among the multiple second points that matches the first point, and the distance is inversely proportional to the confidence; and filter out the first point in the first point cloud data whose confidence is lower than a first confidence threshold.

[0153] In a possible implementation manner, the device further includes:

[0154] The filtering module 503 is further configured to retain, for any first point in the mapped first point cloud data, the first point in the first point cloud data if there is no second point matching the first point in the plurality of second points;

[0155] A determination module 505 is used to perform precise registration on the first point cloud data and the second point cloud data after filtering out the external points, determine the final relative posture of the second point cloud and the first point cloud, and then determine a third posture corresponding to the first point cloud data based on the final relative posture and the second posture corresponding to the second point cloud data in the electronic map;

[0156] The updating module 506 is used to update the electronic map based on the first point cloud data after filtering the external points and the third posture.

[0157] In one possible implementation, the first point cloud data includes multiple first points, and the second point cloud data includes multiple second points; the filtering unit 5033 is used to determine the confidence of the multiple second points in the mapped second point cloud data based on at least one of whether there is a first point matching the second point in the multiple first points, matching parameters between the first point matching the second point and the second point, and the number of first points whose distance from the second point does not exceed a first distance threshold; and filter out second points in the second point cloud data whose confidence is lower than a second confidence threshold.

[0158] In a possible implementation, the first point cloud data is point cloud data scanned by an automatic vehicle, the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map, and the device further includes:

[0159] The updating module 506 is configured to replace the second point cloud data in the electronic map with the filtered second point cloud data.

[0160] In one possible implementation, the second registration module 504 is used to determine the matching point pair relationship between the first point cloud data and the second point cloud data after filtering out the external points; based on the matching point pair relationship, construct a covariance matrix, perform SVD decomposition on the covariance matrix, and output a rotation matrix and a translation matrix; based on the rotation matrix and the translation matrix, map the first point cloud data and the second point cloud data from which the external points are filtered out to the same coordinate system, and determine the error function value between the first point cloud data and the second point cloud data after mapping; if the error function value is greater than the target function value, iteratively execute the process of determining the matching point pair relationship and the error function value until the obtained error function value is not greater than the target function value, and then determine the relative posture corresponding to the current error function value as the final relative posture of the first point cloud and the second point cloud.

[0161] It should be noted that: the point cloud data registration device provided in the above embodiment only uses the division of the above functional modules as an example when registering the point cloud data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the automatic vehicle is divided into different functional modules to complete all or part of the functions described above. In addition, the point cloud data configuration device provided in the above embodiment and the point cloud data registration method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0162] Figure 7 The structure block diagram of an automatic vehicle 700 provided by an exemplary embodiment of the present application is shown. The automatic vehicle 700 includes: a processor 701 and a memory 702 .

[0163] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0164] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one program code, which is used to be executed by the processor 701 to implement the point cloud data registration method provided in the method embodiment of the present application.

[0165] In some embodiments, the automatic vehicle 700 may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera 706, an audio circuit 707, a positioning component 708 and a power supply 709.

[0166] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0167] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 704 can communicate with other automatic vehicles through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0168] The display screen 705 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to collect touch signals on the surface or above the surface of the display screen 705. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 705 can be one, and the front panel of the automatic vehicle 700 is set; in other embodiments, the display screen 705 can be at least two, which are respectively set on different surfaces of the automatic vehicle 700 or are folded; in some other embodiments, the display screen 705 can be a flexible display screen, which is set on the curved surface or folded surface of the automatic vehicle 700. Even, the display screen 705 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 705 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0169] The camera assembly 706 is used to collect images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. The front camera is arranged on the front panel of the automatic vehicle, and the rear camera is arranged on the back of the automatic vehicle. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0170] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing, or input them into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the automatic vehicle 700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0171] The positioning component 708 is used to locate the current geographical location of the automatic vehicle 700 to achieve navigation or LBS (Location Based Service). The positioning component 708 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0172] The power supply 709 is used to power various components in the automatic vehicle 700. The power supply 709 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0173] In some embodiments, the automatic vehicle 700 further includes one or more sensors 710 , including but not limited to: an acceleration sensor 711 , a gyroscope sensor 712 , a pressure sensor 713 , a fingerprint sensor 714 , an optical sensor 715 , and a proximity sensor 716 .

[0174] The acceleration sensor 711 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the automatic vehicle 700. For example, the acceleration sensor 711 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 701 can control the display screen 705 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 can also be used to collect motion data of games or users.

[0175] The gyro sensor 712 can detect the body direction and rotation angle of the automatic car 700, and the gyro sensor 712 can cooperate with the acceleration sensor 711 to collect the user's 3D actions on the automatic car 700. The processor 701 can implement the following functions based on the data collected by the gyro sensor 712: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0176] The pressure sensor 713 can be set on the side frame of the automatic vehicle 700 and / or the lower layer of the display screen 705. When the pressure sensor 713 is set on the side frame of the automatic vehicle 700, it can detect the user's grip signal of the automatic vehicle 700, and the processor 701 performs left and right hand recognition or quick operation according to the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is set on the lower layer of the display screen 705, the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0177] The fingerprint sensor 714 is used to collect the user's fingerprint, and the processor 701 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 714 can be set on the front, back, or side of the automatic car 700. When a physical button or a manufacturer logo is set on the automatic car 700, the fingerprint sensor 714 can be integrated with the physical button or the manufacturer logo.

[0178] The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the display screen 705 according to the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is reduced. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 according to the ambient light intensity collected by the optical sensor 715.

[0179] The proximity sensor 716, also called a distance sensor, is disposed on the front panel of the automatic vehicle 700. The proximity sensor 716 is used to collect the distance between the user and the front of the automatic vehicle 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front of the automatic vehicle 700 is gradually decreasing, the processor 701 controls the display screen 705 to switch from the screen-on state to the screen-off state; when the proximity sensor 716 detects that the distance between the user and the front of the automatic vehicle 700 is gradually increasing, the processor 701 controls the display screen 705 to switch from the screen-off state to the screen-on state.

[0180] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the automatic vehicle 700, and the automatic vehicle 700 may include more or less components than those shown in the figure, or combine certain components, or adopt a different arrangement of components.

[0181] Figure 8 This is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 801 and one or more memories 802, wherein at least one program code is stored in the memory 802, and the at least one program code is loaded and executed by the processor 801 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.

[0182] The server 800 is used to execute the steps executed by the server in the above method embodiment.

[0183] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a program code, and the program code can be executed by a processor in a computer device to complete the registration method of point cloud data in the above embodiment. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0184] In an exemplary embodiment, a computer program or a computer program product is further provided. The computer program or the computer program product includes a computer program code. When the computer program code is executed by a computer, the computer implements the point cloud data registration method in the above embodiment.

[0185] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0186] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud data registration method for autonomous driving, characterized in that: The method comprises: Acquire first point cloud data and second point cloud data; Based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data, determine a plurality of first key points from the first point cloud data, and determine a plurality of second key points from the plurality of second point cloud data, wherein the key points are points in the point cloud data that meet preset conditions; The plurality of first key points and the plurality of second key points are registered, and a matching point pair relationship is output; the point cloud features of the first point cloud data include the first features of the plurality of first key points, and the point cloud features of the second point cloud data include the second features of the plurality of second key points; The registering the plurality of first key points and the plurality of second key points and outputting the matching point pair relationship comprises: Based on the first features of the plurality of first key points and the second features of the plurality of second key points, determining a plurality of candidate matching point pair relationships and matching parameters of the plurality of candidate matching point pair relationships, the matching parameters being used to represent feature similarities between the first key points and the second key points matched in the matching point pair relationships; Outputting the matching point pair relationships based on the matching parameters of the multiple candidate matching point pair relationships; outputting the matching point pair relationships based on the matching parameters of the multiple candidate matching point pair relationships comprises: Output the matching point pair relationship corresponding to the highest matching parameter; or determine the matching point pair relationship corresponding to the highest matching parameter; for each matched first key point and second key point in the determined matching point pair relationship, determine the first position relationship corresponding to the first key point, the first position relationship is used to represent the position relationship between the first key point and at least one other first key point, determine the second position relationship corresponding to the second key point, the second position relationship is used to represent the position relationship between the second key point and at least one other second key point, the other at least one second key point is a second key point that matches the other at least one first key point, when the first position relationship is the same as the second position relationship, retain the first key point and the second key point, when the first position relationship is different from the second position relationship, discard the first key point and the second key point; output the filtered matching point pair relationship; For the matching point pair relationship, filtering out outliers according to confidence, wherein the outliers are points that do not match between the first point cloud data and the second point cloud data; The first point cloud data and the second point cloud data after filtering out the external points are precisely aligned to determine the final relative pose of the second point cloud and the first point cloud.

2. The method according to claim 1, characterized in that The first point cloud data includes a plurality of first points, and the second point cloud data includes a plurality of second points; and determining a plurality of first key points from the first point cloud data and a plurality of second key points from the plurality of second point cloud data based on the point cloud features of the first point cloud data and the point cloud features of the second point cloud data comprises: Performing feature extraction on a plurality of first points in the first point cloud data and a plurality of second points in the second point cloud data to obtain first features of the plurality of first points and second features of the plurality of second points; Determine a first key point feature from the obtained plurality of first features, and determine a second key point feature from the obtained plurality of second features; A first point corresponding to the determined first key point feature is determined as a first key point, and a second point corresponding to the determined second key point feature is determined as a second key point.

3. The method according to claim 2, characterized in that The step of determining the first point corresponding to the determined first key point feature as the first key point, and determining the second point corresponding to the determined second key point feature as the second key point, comprises: For any first key point feature, if there is a second key point feature matching the first key point feature among the determined second key point features, the first point corresponding to the first key point feature is determined as the first key point, and the second point corresponding to the second key point feature matching the first key point feature is determined as the second key point; or, for any second key point feature, if there is a first key point feature matching the second key point feature among the determined first key point features, the second point corresponding to the second key point feature is determined as the second key point, and the first point corresponding to the first key point feature matching the second key point feature is determined as the first key point.

4. The method according to claim 3, characterized in that The first point cloud data is point cloud data obtained by scanning an automatic vehicle, and the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map; the automatic vehicle is equipped with a posture determination device, and the posture determination device is used to determine the posture of the automatic vehicle; for any first key point feature, when there is a second key point feature matching the first key point feature in the determined second key point features, determining the first point corresponding to the first key point feature as the first key point includes: Acquire a first posture of the automatic vehicle and a second posture corresponding to the second point cloud data, wherein the first posture is a posture determined by a posture determination device of the automatic vehicle; For any first key point feature, based on the first posture and the second posture, determine the distance between the second point corresponding to each second key point feature and the first point corresponding to the first key point feature; if there is a second key point feature similar to the first key point feature among multiple second key point features whose distance does not exceed a first distance threshold, the first point corresponding to the first key point feature is determined as the first key point.

5. The method according to any one of claims 1 to 4, characterized in that: The filtering out the outliers according to the confidence level for the matching point pair relationship includes: Based on the matching point pair relationship, determining a first relative pose of the first point cloud and the second point cloud; Based on the first relative posture, the first point cloud data and the second point cloud data are mapped into the same coordinate system; based on the mapped first point cloud data and the second point cloud data, the external points are filtered out according to the confidence level.

6. The method according to claim 5, characterized in that The first point cloud data includes a plurality of first points, and the second point cloud data includes a plurality of second points; and filtering out the external points according to the confidence level based on the mapped first point cloud data and the second point cloud data, comprises: For a plurality of first points in the mapped first point cloud data, determining the confidence of the first point based on a distance between the first point and a target second point, the target second point being a second point among the plurality of second points that matches the first point, the distance being inversely proportional to the confidence; The first points in the first point cloud data whose confidence level is lower than a first confidence threshold are filtered out.

7. The method according to claim 6, characterized in that The first point cloud data is point cloud data obtained by scanning by an automatic vehicle, and the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map. The method further includes: For any first point in the mapped first point cloud data, if there is no second point matching the first point in the plurality of second points, retaining the first point in the first point cloud data; After accurately registering the first point cloud data and the second point cloud data after filtering out the external points, and determining a final relative pose between the second point cloud and the first point cloud, determine a third pose corresponding to the first point cloud data based on the final relative pose and a second pose corresponding to the second point cloud data in the electronic map; The electronic map is updated based on the first point cloud data after filtering the external points and the third posture.

8. The method according to claim 7, characterized in that The first point cloud data includes a plurality of first points, and the second point cloud data includes a plurality of second points; and filtering out the external points according to the confidence level based on the mapped first point cloud data and the second point cloud data, comprises: For multiple second points in the mapped second point cloud data, the confidence of the second point is determined based on at least one of whether there is a first point matching the second point among the multiple first points, matching parameters between the first point matching the second point and the second point, and the number of first points whose distance from the second point does not exceed a first distance threshold; and second points in the second point cloud data whose confidence is lower than a second confidence threshold are filtered out.

9. The method according to claim 8, characterized in that The first point cloud data is point cloud data obtained by scanning by an automatic vehicle, and the second point cloud data is point cloud data corresponding to the positioning position of the automatic vehicle in an electronic map. The method further includes: The second point cloud data in the electronic map is replaced with the filtered second point cloud data.

10. The method according to claim 1, characterized in that The step of precisely registering the first point cloud data and the second point cloud data after filtering out the external points to determine the final relative position of the second point cloud and the first point cloud includes: Determine a matching point pair relationship between the first point cloud data and the second point cloud data after filtering out the external points; Based on the relationship between the matching point pairs, a covariance matrix is ​​constructed, the covariance matrix is ​​decomposed by SVD, and a rotation matrix and a translation matrix are output; Based on the rotation matrix and the translation matrix, the first point cloud data and the second point cloud data from which the external points are filtered out are mapped to the same coordinate system, and an error function value between the first point cloud data and the second point cloud data after mapping is determined; If the error function value is greater than the objective function value, the process of determining the matching point pair relationship and the error function value is iterated until the obtained error function value is no greater than the objective function value, and the relative pose corresponding to the current error function value is determined as the final relative pose of the first point cloud and the second point cloud.

Images