Point cloud matching method and device, electronic equipment and storage medium

Abstract

The invention provides a point cloud matching method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a laser point cloud currently collected by a laser radar; for each laser point in the laser point cloud, constructing a matching constraint when each laser point is matched with the map point cloud; calculating a weight for describing the matching quality of each laser point from a plurality of dimensions; based on the weight, weighting the matching constraint corresponding to each laser point to obtain a weighted matching constraint; and based on the weighted matching constraint, matching the laser point cloud and the map point cloud to obtain a matching result. According to the method, the weight can be determined from multiple dimensions, the matching constraints of the laser points with different matching qualities are dynamically adjusted based on the weight, and the accuracy and robustness of the matching algorithm are improved.

Description

Technical Field

[0001] This application relates to the field of simultaneous localization and mapping (SLAM), and more specifically, to a method, apparatus, electronic device, and storage medium for point cloud matching in the field of SLAM. Background Technology

[0002] The LiDAR SLAM (Simultaneous Localization and Mapping) algorithm relies on point cloud data acquired by LiDAR sensors. By matching the current LiDAR point cloud with existing map point clouds, it solves the pose of the robot or vehicle, and then builds or updates the map.

[0003] In existing technologies, mainstream SLAM algorithms, such as those based on Iterative Closest Point (ICP), typically search for map points near the current laser point as matching points and construct point-to-point, point-to-line, or point-to-surface geometric constraints to solve for pose. However, ICP-based algorithms have relatively low matching accuracy and robustness. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for point cloud matching, which can improve the matching accuracy and robustness of the matching algorithm.

[0005] Firstly, a point cloud matching method is provided, which includes: acquiring the laser point cloud currently collected by the lidar; for each laser point in the laser point cloud, constructing matching constraints when matching each laser point with the map point cloud; calculating weights from multiple dimensions to describe the matching quality of each laser point; weighting the matching constraints corresponding to each laser point based on the weights to obtain weighted matching constraints; and matching the laser point cloud and the map point cloud based on the weighted matching constraints to obtain a matching result.

[0006] In the above technical solution, a matching constraint is constructed for each laser point in the laser point cloud when matching it with the map point cloud. The weights used to describe the matching quality of each laser point in the laser point cloud are calculated from multiple dimensions. The matching constraints of each laser point are weighted, and the laser point cloud and the map point cloud are matched based on the weighted matching constraints. This ensures that the matching constraints of laser points with high matching quality contribute more to the final matching result, thereby improving the overall matching accuracy and reducing the impact of laser points with low matching quality on the final matching result. This realizes an adaptive weighted elastic matching algorithm, which improves the accuracy and robustness of the matching algorithm.

[0007] In conjunction with the first aspect, in some possible implementations, weights for describing the matching quality of each laser point are calculated from multiple dimensions, including: for each laser point in the laser point cloud, determining the N closest map points to the laser point from the map point cloud, and constructing a local map patch corresponding to the laser point based on the N map points; where N is greater than 1; and calculating weights for describing the matching quality of each laser point from multiple dimensions based on the local map patch.

[0008] In the above technical solution, local map tiles are constructed by selecting the N map points closest to the laser point cloud. This creates a point-to-patch geometric model, which is compatible with matching 2D and 3D LiDAR point clouds with map point clouds, unifying the matching process for 2D and 3D point clouds. The point-to-patch geometric model also facilitates the subsequent process of determining the matching residual based on the distance between the laser point and the local map tile, achieving compatibility of the matching algorithm with both 2D and 3D LiDAR.

[0009] Combining the first aspect and the above implementation methods, in some possible implementation methods, the weights include at least one of the following: structural weights, depth weights, probability weights, and residual weights. Based on local map tiles, weights used to describe the matching quality of each laser point are calculated from multiple dimensions, including at least one of the following: determining the structural fitting error of the local map tile corresponding to the laser point, and determining the structural weight corresponding to the laser point based on the structural fitting error and the map resolution corresponding to the map point cloud; determining the depth weight corresponding to the laser point based on the depth of the laser point and the maximum measurement distance of the lidar; determining the probability weight corresponding to the laser point based on the average observation probability of the local map tile corresponding to the laser point; determining the matching residual corresponding to the laser point based on the laser point and the local map tile corresponding to the laser point, and determining the residual weight corresponding to the laser point based on the matching residual corresponding to the laser point.

[0010] In the above technical solution, by calculating weights across multiple dimensions to weight the matching constraints, each matching constraint can be weighted across multiple dimensions, improving the strength of high-quality matching constraints and reducing the impact of low-quality matching constraints. Specifically, by calculating structural weights to adjust matching constraints, the matching weight of planar environmental regions can be effectively increased, causing the laser point cloud to align as closely as possible with the map plane, reducing the impact of non-planar elements such as cluttered objects on the matching results and improving matching accuracy. By calculating depth weights to adjust matching constraints, the matching constraints of nearby laser points can be emphasized. Closer laser points typically have higher accuracy and reliability, and prioritizing these points can improve matching accuracy. By calculating probability weights using the average observation probability, the weight of areas with low observation probabilities can be reduced in localization and mapping, thereby minimizing the impact of these areas on overall localization and mapping. This allows the matching results of the laser point cloud to align with static, high-probability areas in the map, reducing the impact of unstable observations on the matching results. The distance from a laser point to the local map tile corresponding to that laser point is the matching residual of that laser point. Adjusting the matching constraints based on the residual weight determined by the matching residual can make the laser point cloud matching rely as much as possible on laser points with smaller errors, thereby reducing the matching error.

[0011] Combining the first aspect and the above implementation methods, in some possible implementation methods, determining the structure fitting error of the local map patch corresponding to the laser point includes: calculating the first distance from N map points to the local map patch respectively; calculating the average value of the first distance from N map points to the local map patch, and determining the average value as the structure fitting error of the local map patch.

[0012] In the above technical solution, the structure fitting error is calculated by constructing the first distance from N map points to the local map patch. This takes into account the error of the matching residual caused by the fact that not all N map points are on the fitted local map patch during the fitting process. This allows for the accurate determination of the structure fitting error of the local map patch.

[0013] Combining the first aspect and the above implementation methods, in some possible implementation methods, the residual weight corresponding to the laser point is determined based on the matching residual corresponding to the laser point, including: when the number of matching constraints corresponding to the laser point cloud is greater than a preset threshold, sorting the matching constraints corresponding to each laser point in ascending order of matching residuals to obtain a sorting result; filtering out a preset proportion of matching constraints at the end of the sorting result, and assigning corresponding residual weights to the unfiltered matching constraints according to the sorting result; wherein, the matching constraints closer to the head of the sorting result are assigned larger residual weights.

[0014] In the above technical solution, when the number of matching constraints exceeds the minimum threshold for matching constraints, the matching constraints are first sorted in ascending order based on the size of the matching residuals. During this sorting, the earlier the matching constraint appears, the smaller its weighted matching residual. A smaller residual indicates a higher matching quality for the current laser point. Eliminating a predetermined proportion of the last-ranked matching constraints further refines the matching constraints of the laser point cloud, removing potential low-quality and erroneous matches, thus improving the robustness and accuracy of the matching algorithm. For unfiltered matching constraints, their residual weights are decreased according to the sorting order to increase the weight of matching constraints with small residuals, highlighting the matching constraints corresponding to high-quality laser points, further improving the robustness and accuracy of the matching algorithm.

[0015] Combining the first aspect and the above implementation methods, in some possible implementation methods, after matching the laser point cloud and the map point cloud based on the weighted matching constraints to obtain the matching result, the method further includes: transforming each laser point into a map coordinate system based on the matching result; determining a second distance between each laser point and the nearest map point in the map coordinate system; identifying laser points whose second distance is greater than a first threshold and less than the second threshold as target laser points; wherein the second threshold is greater than the first threshold; accumulating the second distances that are greater than the first threshold and less than the third threshold to obtain the total distance; wherein the third threshold is greater than the second threshold; dividing the total distance by the number of target laser points to obtain the average matching error used to evaluate the matching quality.

[0016] In the above technical solution, by setting reasonable three-level distance thresholds to classify laser points, only the distance between the first and third thresholds is accumulated to obtain the total distance. This avoids the influence of noise points in non-overlapping areas on the matching quality assessment. Ignoring the second distance less than the first threshold reduces the discretization distance caused by point cloud resolution, allowing for a more accurate average matching error to be obtained based on the total distance and the total number of target laser points, thus achieving accurate matching quality assessment.

[0017] Combining the first aspect and the above implementation methods, in some possible implementation methods, after dividing the total distance by the number of target laser points to obtain the average matching error, the method further includes: dividing the number of target laser points by the total number of laser points in the laser point cloud to obtain the point cloud overlap; performing a scaling transformation on the point cloud overlap and then a cubic transformation to obtain the map overlap weight; and weighting the average matching error based on the map overlap weight to obtain the weighted matching error used to evaluate the matching quality.

[0018] In the above technical solution, after determining the average matching error, the map overlap is calculated, and the overlap weight is calculated based on the map overlap to weight the average matching error, so as to obtain the weighted matching error to evaluate the matching quality. This can avoid situations where some laser points match the map point cloud in certain erroneous matching cases, and the distance between the closest matching points is very small, while the matching distance of other points is greater than the maximum matching distance threshold. In such cases, it is also possible to calculate a smaller average matching error.

[0019] Combining the first aspect and the above implementation methods, in some possible implementation methods, before determining the N map points closest to the laser point in the map point cloud for each laser point in the laser point cloud, the method further includes: if the lidar is determined to be a 2D lidar, expanding the laser point cloud and map point cloud from two dimensions to three dimensions based on a preset value of the third dimension to obtain a three-dimensional laser point cloud and map point cloud.

[0020] In summary, this application unifies the matching process of 2D and 3D point clouds and the calculation method of matching residuals by establishing a geometric model from points to local map tiles. By calculating weights across multiple dimensions to weight matching constraints, it is possible to weight each matching constraint in multiple dimensions, thereby improving the strength of high-quality matching constraints, reducing the impact of low-quality matching constraints, and enhancing the robustness and accuracy of the matching algorithm. Setting reasonable three-level distance thresholds can avoid the influence of noise points in non-overlapping regions on the matching quality assessment, reduce the discretization distance caused by point cloud resolution, and achieve accurate matching quality assessment. Further weighting using nonlinear point cloud overlap yields accurate and robust weighted matching errors, achieving a more accurate assessment of matching quality.

[0021] Secondly, a point cloud matching device is provided, comprising: an acquisition module for acquiring a laser point cloud currently collected by a lidar; a construction module for constructing matching constraints for each laser point in the laser point cloud when matching it with a map point cloud; a calculation module for calculating weights from multiple dimensions to describe the matching quality of each laser point; a weighting module for weighting the matching constraints corresponding to each laser point based on the weights to obtain weighted matching constraints; and a matching module for matching the laser point cloud and the map point cloud based on the weighted matching constraints to obtain a matching result.

[0022] In conjunction with the second aspect, in some possible implementations, the calculation module is specifically used to: for each laser point in the laser point cloud, determine the N map points closest to the laser point from the map point cloud, and construct a local map patch corresponding to the laser point based on the N map points; where N is greater than 1; and calculate the weights used to describe the matching quality of each laser point from multiple dimensions based on the local map patch.

[0023] Combining the second aspect and the above implementation methods, in some possible implementation methods, the weights include at least one of the following: structural weights, depth weights, probability weights, and residual weights. The calculation module is specifically used to: determine the structural fitting error of the local map patch corresponding to the laser point, and determine the structural weight corresponding to the laser point based on the structural fitting error and the map resolution corresponding to the map point cloud; determine the depth weight corresponding to the laser point based on the depth of the laser point and the maximum measurement distance of the lidar; determine the probability weight corresponding to the laser point based on the average observation probability of the local map patch corresponding to the laser point; determine the matching residual corresponding to the laser point based on the laser point and the local map patch corresponding to the laser point, and determine the residual weight corresponding to the laser point based on the matching residual corresponding to the laser point.

[0024] Combining the second aspect and the above implementation methods, in some possible implementation methods, the calculation module is specifically used to calculate the first distance from N map points to the local map patch respectively; calculate the average value of the first distance from N map points to the local map patch, and determine the average value as the structure fitting error of the local map patch.

[0025] Combining the second aspect and the above implementation methods, in some possible implementation methods, the weighting module is specifically used to sort the matching constraints corresponding to each laser point in ascending order of matching residuals when the number of matching constraints corresponding to the laser point cloud is greater than a preset threshold, and obtain a sorting result; filter out the matching constraints at the end of the sorting result with a preset proportion, and assign corresponding residual weights to the unfiltered matching constraints according to the sorting result; wherein, the matching constraints closer to the head of the sorting result are assigned larger residual weights.

[0026] In conjunction with the second aspect and the above implementation methods, in some possible implementation methods, the device further includes: an evaluation module, configured to transform each laser point into a map coordinate system based on the matching results; determine a second distance between each laser point and the nearest map point in the map coordinate system; identify laser points whose second distance is greater than a first threshold and less than the second threshold as target laser points; wherein the second threshold is greater than the first threshold; accumulate the second distances that are greater than the first threshold and less than a third threshold to obtain a total distance; wherein the third threshold is greater than the second threshold; divide the total distance by the number of target laser points to obtain the average matching error used to evaluate the matching quality.

[0027] Combining the second aspect and the above implementation methods, in some possible implementation methods, the weighting module is also used to divide the number of target laser points by the total number of laser points in the laser point cloud to obtain the point cloud overlap; after scaling the point cloud overlap, perform a cubic transformation again to obtain the map overlap weight; and weight the average matching error based on the map overlap weight to obtain the weighted matching error used to evaluate the matching quality.

[0028] In combination with the second aspect and the above implementation, in some possible implementations, the device further includes: an expansion module, used to expand the laser point cloud and map point cloud from two dimensions to three dimensions based on a preset value of the third dimension before determining the N map points closest to the laser point in the map point cloud for each laser point in the laser point cloud, provided that the lidar is determined to be a 2D lidar.

[0029] Thirdly, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the electronic device to perform the methods of the first aspect or any possible implementation thereof.

[0030] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0031] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0032] Figure 1 This is a schematic flowchart of a point cloud matching method provided in an embodiment of this application.

[0033] Figure 2 This is a schematic diagram illustrating the distance between a three-dimensional laser point and its corresponding local map tile, provided in an embodiment of this application.

[0034] Figure 3 This is a schematic diagram of the distance from a point to a plane provided in an embodiment of this application.

[0035] Figure 4 This is a schematic diagram illustrating the distance between a two-dimensional laser point and its corresponding local map tile, provided in an embodiment of this application.

[0036] Figure 5 This is a schematic diagram of the structure of a point cloud matching device provided in an embodiment of this application.

[0037] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0039] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0040] The LiDAR SLAM algorithm relies on point cloud data acquired by LiDAR sensors. By matching the current LiDAR point cloud with existing map point clouds, it solves the pose of the robot or vehicle, and then builds or updates the map.

[0041] In existing technologies, SLAM algorithms all adopt the idea of ​​iterating the nearest point. Using the predicted pose, they search for the nearest map points in the vicinity of the current laser point as the matching points, and then solve the pose based on the matching constraints.

[0042] The matching algorithms used in the existing technology do not take into account the matching quality of different laser points, and treat the matching constraints of all laser points equally, resulting in low matching accuracy and robustness of the matching algorithms.

[0043] Based on this, this application proposes a point cloud matching method. By calculating the weight used to describe the matching quality of each laser point in the laser point cloud, and adjusting the matching constraints based on the weight, the laser point cloud and the map point cloud are matched, realizing an adaptive weight elastic matching algorithm, which improves the matching accuracy and robustness of the matching algorithm.

[0044] Figure 1 This is a schematic flowchart illustrating a point cloud matching method provided in an embodiment of this application. The method is applied to an electronic device, which may specifically be a robot or vehicle, equipped with a LiDAR. The electronic device can be in a mobile state, and the LiDAR collects real-time data about the surrounding environment of the electronic device during its movement. Based on changes in the surrounding environment, the LiDAR collects different LiDAR point clouds at different times.

[0045] Laser point clouds are generally used for simultaneous localization and mapping (SLAM). For example, an electronic device can use a lidar system to collect laser point clouds of its surroundings in real time while moving, and build a map based on these real-time collected laser point clouds, or add newly collected laser point clouds to the map to update the existing map. Alternatively, the electronic device can match the laser point clouds currently collected by the lidar with stored map point clouds to determine its current location. This embodiment uses a robot as an example for illustration.

[0046] For example, such as Figure 1 As shown, the method 100 includes:

[0047] Step 101: Obtain the laser point cloud currently collected by the lidar;

[0048] Step 102: For each laser point in the laser point cloud, construct matching constraints when matching each laser point with the map point cloud;

[0049] Step 103: Calculate the weights used to describe the matching quality of each laser point from multiple dimensions;

[0050] Step 104: Based on the weights, the matching constraints corresponding to each laser point are weighted to obtain the weighted matching constraints;

[0051] Step 105: Based on the weighted matching constraints, match the laser point cloud and the map point cloud to obtain the matching result.

[0052] exist Figure 1 In the illustrated embodiment, a matching constraint is constructed for each laser point in the laser point cloud when matching it with the map point cloud. Weights describing the matching quality of each laser point in the laser point cloud are calculated from multiple dimensions. The matching constraints of each laser point are weighted, and the laser point cloud and map point cloud are matched based on these weighted constraints. This ensures that the matching constraints of high-quality laser points contribute more to the final matching result, thereby improving the overall matching accuracy and reducing the impact of low-quality laser points on the final matching result. This implements an adaptive weighted elastic matching algorithm, improving the accuracy and robustness of the matching algorithm.

[0053] The following is about Figure 1 The specific implementation methods of each step in the illustrated embodiments will be explained below:

[0054] In step 101, the lidar is used to scan the surrounding environment with laser pulses. During the scanning process, the lidar generates a series of points, namely laser points. Each laser point represents information about the location of an object surface detected by the lidar. The dataset composed of all these laser points is called a laser point cloud.

[0055] The robot acquires a frame of laser point cloud collected by the lidar at the current moment, and obtains the robot's latest surrounding environment data, namely the aforementioned laser point cloud.

[0056] In step 102, the matching constraint can be understood as the restriction conditions generated when matching the currently observed laser point with points in the map point cloud. These restrictions can be one or more, such as distance constraints, angle constraints, etc. One or more restrictions generated when matching a laser point with a point in the map point cloud can be considered as a single matching constraint. Each laser point in the laser point cloud generates a matching constraint when matched with points in the map point cloud; that is, each laser point corresponds to one matching constraint.

[0057] A map point cloud is pre-built map data stored in the robot. Alternatively, the robot can store map data locally and retrieve the map point cloud from the storage location.

[0058] After obtaining the laser point cloud and the map point cloud, a geometric model from point to local map tile can be constructed for each point in the laser point cloud. Based on the geometric model from point to local tile, matching constraints can be constructed when matching each laser point with the map point cloud.

[0059] In step 103, the weight is used to describe the matching quality when matching laser points and map points. The higher the matching quality of a laser point, the greater its weight, and the greater its weight, the greater its influence in the matching process.

[0060] It is understandable that the existence of data in multiple dimensions affects the quality of the matching constraints of laser points. For each laser point, multiple dimensions of data that constrain that laser point can be identified. Based on the data in multiple dimensions, the weights used to describe the matching quality of each laser point can be calculated.

[0061] In one possible implementation, weights for describing the matching quality of each laser point are calculated from multiple dimensions, including: for each laser point in the laser point cloud, determining the N closest map points in the map point cloud, and constructing a local map patch corresponding to the laser point based on the N map points; where N is greater than 1; and calculating weights for describing the matching quality of each laser point from multiple dimensions based on the local map patch.

[0062] The map point cloud includes multiple map points. For each laser point, the N closest map points to the laser point can be determined from the map point cloud, and a local map block corresponding to the laser point can be constructed based on the N map points.

[0063] When the lidar is a 3D lidar, both the collected lidar point cloud and the pre-stored map point cloud are three-dimensional data. The constructed local map patch can be regarded as a three-dimensional plane. Then, the least squares method can be used to fit the N map points closest to the lidar point to obtain a fitting plane and obtain the local map patch corresponding to the lidar point.

[0064] The equations for a three-dimensional plane are generally as shown in the following formula (1):

[0065] Ax + By + Cz + D = 0 (1)

[0066] Moving the constant D in formula (1) to the right side of the equation, we get formula (2):

[0067] Ax + By + Cz = -D (2)

[0068] Dividing both sides of equation (2) by parameter D yields equation (3):

[0069] n x x+n y y+n z z = np = -1 (3)

[0070] Where, n = (n x ,n y ,n z ) = (A / D, B / D, C / D), where n is the normal vector of the constructed 3D plane, containing 3 variables, and p is a 3D point on the plane.

[0071] Assuming that for each laser point, the five closest map points are determined from the map point cloud, and a local map patch corresponding to the laser point is constructed by fitting a plane based on these five map points. For example, for a laser point p0 in the laser point cloud, the five closest map points to p0 are determined as follows: p1(x1, y1, z1), p2(x2, y2, z2), p3(x3, y3, z3), p4(x4, y4, z4), and p5(x5, y5, z5). The specific steps for solving the plane equation corresponding to the laser point p0 using the least squares method to fit these five map points include:

[0072] Step 1, construct a system of linear overdetermined equations, as shown in the following formula (4):

[0073]

[0074] Step 2, simplify formula (4) to formula (5):

[0075] An = b (5)

[0076] In formula (5), A is a 5*3 matrix, with each row representing the coordinates of a map point. n is a 3*1 normal vector, and b is a 5*1 column vector with each element being -1.

[0077] Step 3: Use the QR decomposition method to efficiently solve for n in formula (5), thereby obtaining the plane equation in formula (3), that is, obtaining the local map patch corresponding to the laser point p0.

[0078] For example, the solution yields: n x =2, n y =2, n z =1, then we can determine n = [n] in formula (5). x n y n z ] T =

[221] T Substituting the obtained n into formula (3), we can determine the equation of plane 1 (i.e. local map block 1) corresponding to laser point p0 as: 2x+2y+1z=-1.

[0079] After obtaining the local map patch corresponding to each laser point, matching constraints for each laser point can be constructed based on the local map patch. Then, based on the local map patch, weights used to describe the matching quality of each laser point are calculated from multiple dimensions.

[0080] In some embodiments, after obtaining a local map tile, the distance from the laser point to the local map tile can be calculated to obtain the matching residual of the laser point. The matching residual refers to the difference between each laser point and its corresponding point in the map point cloud during the point cloud matching process. This difference is usually represented by Euclidean distance.

[0081] like Figure 2 As shown, for three-dimensional data, the distance between a laser point and the local map tile corresponding to that laser point is the distance between the laser point and the plane corresponding to that laser point.

[0082] Figure 3 This is a schematic diagram illustrating the distance from a point to a plane according to an embodiment of this application. The following embodiments are combined with... Figure 3 The derivation of the formula for the distance from a point to a plane is explained:

[0083] For example, such as Figure 3 As shown, the equation of plane A is given by formula (1), the coordinates of a point p1 in the plane are (x1, y1, z1), and the coordinates of a point p0 outside the plane are (x0, y0, z0). The formula for calculating the distance d from point p0 to plane A is shown in formula (6):

[0084]

[0085] That is, the distance d is equal to the projection of the distance between points p0 and p1 onto the normal vector n. Here, α is the angle between the vectors of points p0 and p1 and the normal vector. Based on geometric relationships, we can obtain formula (7):

[0086]

[0087] Substituting formula (7) into formula (6), we get formula (8):

[0088]

[0089] As in the above embodiment, the normal vector Substituting the normal vector and points p0 and p1 into the numerator of formula (8), we can obtain formula (9):

[0090]

[0091] Since point p1(x1, y1, z1) lies on the plane, we can obtain formula (10) based on formula (3):

[0092] n x x1+n y y1+n z z1=-1 (10)

[0093] Substituting formula (10) into formula (9) yields formula (11):

[0094]

[0095] Substituting formula (11) into formula (8), we obtain the general form of the distance from point p0(x0, y0, z0) to plane A, as shown in formula (12):

[0096]

[0097] Let m be the magnitude of the normal vector. We can obtain the formula for the distance from a point to a plane, as shown in the following formula (13):

[0098]

[0099] After calculating the plane corresponding to the laser point, the coordinates of the laser point and the normal vector of the plane corresponding to the laser point are substituted into formula (13) to obtain the distance from the laser point to the local map tile.

[0100] For example, the coordinates of laser point p0 are (3, 6, 3). As in the above embodiment, the normal vector of plane 1 corresponding to laser point p0 is (2, 2, 1). The magnitude m of the normal vector is calculated based on the following formula (14):

[0101]

[0102] Substituting the normal vector (2, 2, 1) into formula (14), we can calculate m = 3. Substituting the normal vector (2, 2, 1), p0 (3, 4, 3), and m = 3 into formula (13), we can obtain the distance d = (2*3 + 2*4 + 1*3 + 1) / 3 = 6, that is, the matching residual corresponding to the laser point p0 is calculated to be 6.

[0103] It is understandable that the above steps can be performed to calculate the corresponding matching residual for each laser point.

[0104] In the above method, local map tiles are constructed by selecting the N map points closest to the laser point cloud. This creates a point-to-patch geometric model, which is compatible with matching 2D and 3D LiDAR point clouds with map point clouds, unifying the matching process for 2D and 3D point clouds. The point-to-patch geometric model also facilitates the subsequent process of determining the matching residual based on the distance between the laser point and the local map tile, achieving compatibility of the matching algorithm with both 2D and 3D LiDAR.

[0105] In some embodiments, the lidar may be a 2D lidar, in which case both the laser point cloud collected by the lidar and the map point cloud stored in the robot are two-dimensional data. The following embodiments will describe in detail the calculation of the matching residual in this case:

[0106] In one possible implementation, before determining the N closest map points to each laser point in the laser point cloud from the map point cloud, the method further includes: if the lidar is determined to be a 2D lidar, expanding the laser point cloud and map point cloud from two dimensions to three dimensions based on a preset value of the third dimension to obtain a three-dimensional laser point cloud and map point cloud.

[0107] When the LiDAR is a 2D radar, the collected laser point cloud and map point cloud are both two-dimensional data, represented as (x, y). The three-dimensional data is usually represented as (x, y, z), with the third dimension being the z-dimensional dimension. Assuming the preset value of the third dimension is 0, the z-dimensional data of both the two-dimensional laser point cloud and map point cloud can be set to 0, thus expanding the two-dimensional point cloud data to three dimensions.

[0108] After expanding the data from two dimensions to three dimensions, the expanded three-dimensional data (x, y, 0) can be substituted into the three-dimensional plane equation, i.e., the above formula (3), to obtain formula (15):

[0109] n x x+n y y+0=n x x+n y y = np = -1 (15)

[0110] Based on formula (15), when the dimension of z is 0, the plane equation degenerates into the equation of a straight line on the plane z = 0, and the normal vector also degenerates into a vector in the plane z = 0, which is the normal vector of the straight line in the plane.

[0111] like Figure 4 As shown, for two-dimensional data, the distance between a laser point and the local map tile corresponding to that laser point is the distance between the laser point and the line corresponding to that laser point. Specifically, the laser point p0(x0, y0, 0) extended to three dimensions can be substituted into the above formula for the distance from a point to a plane (13) to obtain the formula for the distance from a point to a line, as shown in formula (16):

[0112]

[0113] Based on formula (16), it can be seen that for 3D point clouds and 2D point clouds, this application can calculate the distance from the point to the local map tile in exactly the same way, that is, the matching residual of the laser point can be determined based on the same formula.

[0114] In some embodiments, the matching residual of laser points, the structural features of local map tiles, the observation probability of local map tiles, and the depth of each laser point will all have a certain impact on the matching constraints, that is, on the matching quality of laser points. Therefore, weights can be determined from multiple dimensions based on the above-mentioned multiple data for weighting.

[0115] Among them, the weights determined based on the structural features of local map tiles can be called structural weights, the weights determined based on the observation probabilities of local map tiles can be called probabilistic weights, the weights determined based on the depth of laser points can be called depth weights, and the weights determined based on the matching residuals of laser points can be called residual weights. The following example provides a detailed explanation of the process for determining the above four types of weights:

[0116] In one possible implementation, the weights include at least one of structural weights, depth weights, probability weights, and residual weights. Weights used to describe the matching quality of each laser point are calculated from multiple dimensions, including at least one of the following S11 to S14:

[0117] S11, determine the structure fitting error of the local map patch corresponding to the laser point, and determine the structure weight corresponding to the laser point based on the structure fitting error and the map resolution corresponding to the map point cloud.

[0118] Among them, the structural fitting error is used to describe the error generated when fitting local map tiles. The smaller the structural fitting error, the smaller the error generated when fitting the local map tile, and the smaller the error of the matching residual of the laser point determined based on the local map tile.

[0119] Map resolution represents the quality of a map. A higher resolution map contains more details, and the higher the quality of the map, the higher the accuracy of the fit. The structure fitting error can be divided by the map resolution to normalize the weights, so that the probability weights under different map resolutions are based on the same metric.

[0120] Specifically, a large structure fitting error indicates poor quality of the laser point in the structural feature dimension, and the corresponding structure weight should be small. At higher map resolutions, the impact of a large structure fitting error on the probability weights decreases. Conversely, at lower map resolutions, the impact of a large structure fitting error on the probability weights increases.

[0121] It is understandable that since local map tiles are fitted based on N map points, the structural fitting error of the local map tiles can be determined based on the error between the N map points and the fitted local map tiles. The following example illustrates this situation:

[0122] In one possible implementation, determining the structure fitting error of the local map patch corresponding to the laser point includes: calculating the first distance from N map points to the local map patch respectively; calculating the average value of the first distances from the N map points to the local map patch, and determining the average value as the structure fitting error of the local map patch.

[0123] In the above embodiments, for each laser point, the N nearest map points are searched, and a local map patch is obtained based on the N map points. The fitted local map patch is a plane or straight line representing the shared location of the N map points. However, only when all the map points are located in a planar environment such as a wall or the ground will all N map points be on the fitted local map patch. When all N map points are on the fitted local map patch, the structural fitting error of the local map patch is minimized.

[0124] When map points are located on objects of varying elevations, the fitted local map patch is the plane or line that minimizes the sum of the squares of the vertical distances from all map points to that local map patch. In other words, not all N map points lie on the fitted plane or line. In this case, the distances between each of the N map points and the fitted map patch can be calculated, and the structural fitting error can be determined based on these distances.

[0125] Specifically, you can substitute N map points into the above formula (13) and substitute the normal vector of the plane or line obtained by fitting the N map points into the above formula (13) to calculate the distance between the N map points and the fitted map block.

[0126] For example, the five map points are p1(1, -1, 1), p2(1, 0, 1), p3(1, -1, 2), p4(-1, 1, 1), and p5(1, -2, 1). The normal vector of plane 1 is (2, 2, 1). Substituting the normal vector and the five map points into the above formula (13), we can obtain the first distance D1 between p1 and plane 1, the first distance D2 between p2 and plane 1, the first distance D3 between p3 and plane 1, the first distance D4 between p4 and plane 1, and the first distance D5 between p5 and plane 1.

[0127] D1=(2*1+2*(-1)+1*1+1) / 3=2 / 3;

[0128] D2=(2*1+2*0+1*1+1) / 3=4 / 3;

[0129] D3=(2*1+2*(-1)+1*2+1) / 3=1;

[0130] D4=(2*(-1)+2*1+1*1+1) / 3=1;

[0131] D5=(2*1+2*(-2)+1*1+1) / 3=0.

[0132] After obtaining the N first distances from N map points to the local map patch, the N distances can be added together, and the sum can be divided by N to obtain the average value. The average value is determined as the structure fitting error of the local map patch (i.e., plane 1).

[0133] For example, the five first distances are: D1 = 2 / 3, D2 = 4 / 3, D3 = 1, D4 = 1, and D5 = 0. The sum of the five distances is 4, and the average value of 4 divided by 5 is 0.8. Therefore, the structure fitting error of the local map patch (i.e., plane 1) is 0.8.

[0134] In the above method, the structure fitting error is calculated by constructing the first distance from N map points to the local map patch. This takes into account the error of the matching residual caused by the fact that not all N map points are on the fitted local map patch during the fitting process. This method can accurately determine the structure fitting error of the local map patch.

[0135] After obtaining the structure fitting error, the map resolution corresponding to the map point cloud stored by the robot can be obtained. Substituting the structure fitting error and map resolution into the formula for calculating the structure weights, the structure weights corresponding to the local map patches are calculated. The formula for calculating the structure weights is shown in the following formula (17):

[0136]

[0137] In formula (17), R represents the structural fitting error. map For map resolution, G s For the structural weight gain, ω s (p i ) represents the structure weights, and a1 is a fixed gradient coefficient of the structure weights obtained based on a large number of experiments. For example, a1 can be 0.5. The structure weight gain can be a preset fixed value, or it can be obtained by image processing algorithms or other feature extraction methods such as edge detection and texture analysis.

[0138] Based on formula (17), it can be determined that when the structural fitting error... The larger the value, the greater the structural weight ω. s (p i The smaller the value, the more the structure weight will drop to half of its initial value when the structure fitting error is twice the map resolution.

[0139] For example, the calculated structure fitting error for plane 1 is 0.8. Assuming the map resolution of the acquired point cloud is 0.5 m / pixel, and the structure weight gain G... s If the value is 1, then the structural weight ω corresponding to p0 can be calculated based on formula (17). s (p i It is approximately equal to 0.555.

[0140] S12, based on the depth of the laser point and the maximum measurement distance of the lidar, determine the depth weight corresponding to the laser point.

[0141] The depth of the laser point, which is the distance between the laser point and the lidar, can be calculated based on the time difference between the time the laser is emitted and the time difference between the time the reflected signal from the laser point is received.

[0142] The maximum measurement distance refers to the ability of a lidar to reliably measure the farthest target. This distance varies depending on the sensor model and the environment, typically ranging from tens to hundreds of meters. In this embodiment, the maximum measurement distance of the lidar can be determined based on its model.

[0143] Understandably, due to factors such as robot vibration, environmental texture, and LiDAR measurement errors, the measurement error of distant LiDAR points will be larger than that of nearby points. Furthermore, due to the limited resolution of the LiDAR, the point cloud is denser at close range and sparser at far range, resulting in a sparser map at far range. A sparse point cloud provides lower accuracy in fitting local structures. Therefore, the depth of the LiDAR point also has a certain impact on the matching residual calculated from that point. A larger depth indicates lower matching quality, and the corresponding depth weight should be smaller.

[0144] Furthermore, since different lidars have different maximum measurement distances, the depth of the laser point can be divided by the maximum measurement distance of the lidar to normalize the weights, so that the depth weights of laser points measured by lidars with different maximum measurement distances are based on the same metric.

[0145] Based on the depth of the laser point and the maximum measurement distance of the lidar, the formula for calculating the depth weight is shown in the following formula (18):

[0146]

[0147] In formula (18), G r For depth weight gain, r i R is the current depth of the laser point. max denoted as , where is the maximum measurement range of the lidar, and 'a2' is a fixed gradient coefficient for the depth weights obtained from numerous experiments; 'a2' can be, for example, 2.5. The depth weight gain can be a preset fixed value.

[0148] Based on formula (18), it can be seen that the greater the distance of the laser point, the smaller the depth weight. When the distance of the laser point is 0.4 times the maximum measurement distance of the lidar, the depth weight will drop to half of the initial value.

[0149] For example, the depth r of laser point p0 is calculated. i The maximum measurement distance R of the lidar is determined based on its model, which is 6m. max For a depth of 50m, the depth weight gain G r If the value is 1, then the depth weight ω corresponding to the laser point p0 can be calculated. r (p i It is approximately equal to 0.769.

[0150] S13. Determine the probability weight corresponding to the laser point based on the average observation probability of the local map patch corresponding to the laser point.

[0151] The average observation probability is the average probability of all laser points within a local map tile being observed. It is an indicator for evaluating the reliability of a local map tile, reflecting whether the area is frequently visible or whether there is occlusion.

[0152] Understandably, the constructed map point cloud may contain map points representing dynamic targets such as pedestrians and vehicles, noise, or unstable regions such as object edges. These map points are not fixed and change over time, thus altering the probability of their observation. In other words, the LiDAR cannot capture these LiDAR points every time it acquires data. Furthermore, local map tiles are constructed based on map points. When the map points used to construct a local map tile include the aforementioned map points, the matching residual determined based on that local map tile will contain a certain degree of error.

[0153] Based on historical observation data from N map points, the average observation probability of a local map patch can be evaluated. Specifically, this involves: collecting historical observation data; for each map point, accumulating the number of times it appears in historical observations; for each map point, dividing its accumulated observation count by the total number of scans to obtain its observation probability; and then calculating the average of the observation probabilities of the N map points used to construct the local map patch to obtain the average observation probability of the local map patch.

[0154] For example, in the historical observation data, map point p1 was observed 3 times, map point p2 was observed 4 times, map point p3 was observed 4 times, map point p4 was observed 5 times, and map point p5 was observed 4 times, with a total of 5 scans. The observation probability of p1 is calculated to be 3 / 5, the observation probability of p2 is 4 / 5, the observation probability of p3 is 4 / 5, the observation probability of p4 is 1, and the observation probability of p5 is 4 / 5. Therefore, the average observation probability of local map patch 1 is calculated to be 4 / 5.

[0155] In some embodiments, for map points, there is also an initial observation probability. The probability weight of local map tiles, i.e., the probability weight of laser points, can be calculated based on the average observation probability and the initial observation probability.

[0156] The initial observation probability can be understood as the probability that a laser point will be observed for the first time. Generally, when starting to build a map, since the long-term observation probability of the laser point is unknown, it is usually assumed that the observation probability of each point is the same. The initial observation probability is usually set to a small value, such as 0.5 or 0.1.

[0157] Dividing the average observation probability by the initial observation probability normalizes the weights, ensuring that the probability weights of laser points with different initial observation probabilities are based on the same metric.

[0158] The formula for calculating the probability weights based on the average observation probability and the initial observation probability is shown in the following formula (19):

[0159]

[0160] In formula (19), P represents the average observation probability of the currently matched map patch block. init Let a be the initial observation probability, and a3 be a fixed gradient coefficient of the probability weights obtained from a large number of experiments. For example, a3 can be 0.75. Based on formula (19), it can be seen that the greater the average observation probability of a local map patch, the greater the probability weight.

[0161] For example, if the average observation probability of a local map patch is 4 / 5 and the initial observation probability is 0.5, then the probability weight corresponding to the local map patch can be calculated based on formula (19) as 1.2, that is, the probability weight corresponding to the laser point p0 is 1.2.

[0162] Understandably, map points in unstable regions such as dynamic targets, noise points, or object edges, as well as newly expanded map edge regions, are observed less frequently. Therefore, their observation probability is near the initial probability or decreases to a low level, resulting in a lower probability weight. Weighting the matching constraints based on this lower probability weight reduces the strength of the matching constraints corresponding to laser points with low probability weights. Conversely, map points in static and stable regions of the environment are repeatedly observed, leading to a continuously increasing observation probability and a higher probability weight. Weighting the matching constraints based on this higher probability weight strengthens the matching constraints corresponding to laser points with high probability weights. Ultimately, this causes the matching results of the laser point cloud to gravitate towards static, high-probability regions in the map, reducing the impact of unstable observations on the matching results.

[0163] S14. Based on the laser point and the local map patch corresponding to the laser point, determine the matching residual corresponding to the laser point, and determine the residual weight corresponding to the laser point based on the matching residual corresponding to the laser point.

[0164] The distance from a laser point to its corresponding local map tile is called the matching residual. The matching residual can be used as data to describe the matching quality of a laser point, and the matching weight of each laser point can be determined based on the matching residual. Generally, the larger the matching residual, the lower the matching quality of the laser point. Therefore, the rule for determining the residual weight describing the matching quality of a laser point based on the matching residual could be: the larger the matching residual, the smaller the residual weight.

[0165] It is understandable that the distance between each laser point and its corresponding local map tile may be different; therefore, the matching residual for each laser point may also be different. The matching residual for each laser point can be calculated based on the distance formula from each laser point to the local map tile calculated in the above embodiment. After obtaining the matching residual for each laser point, the residual weight corresponding to each laser point is determined based on the magnitude of the matching residual.

[0166] In the above method, by calculating weights across multiple dimensions to weight the matching constraints, each matching constraint can be weighted across multiple dimensions, improving the strength of high-quality matching constraints and reducing the impact of low-quality matching constraints. Specifically, adjusting matching constraints by calculating structure weights can effectively increase the matching weight of planar environmental regions, making the laser point cloud as close as possible to the map plane, reducing the influence of non-planar elements such as cluttered objects in the environment on the matching results, and improving matching accuracy. Adjusting matching constraints by calculating depth weights can highlight the matching constraints of nearby laser points, as closer laser points usually have higher accuracy and reliability, and prioritizing these points can improve matching accuracy. Calculating probability weights by average observation probability can reduce the weight of areas with low observation probability in localization and mapping, thereby reducing the impact of these areas on overall localization and mapping. This can make the matching results of the laser point cloud closer to static high-probability areas in the map, reducing the impact of unstable observations on the matching results. The distance from a laser point to the local map tile corresponding to that laser point is the matching residual of that laser point. Adjusting the matching constraints based on the residual weight determined by the matching residual can make the laser point cloud matching rely as much as possible on laser points with smaller errors, thereby reducing the matching error and further improving the robustness and accuracy of the matching algorithm.

[0167] In one possible implementation, the residual weights corresponding to laser points are determined based on the matching residuals corresponding to the laser points. This includes: when the number of matching constraints corresponding to the laser point cloud is greater than a preset threshold, sorting the matching constraints corresponding to each laser point in ascending order of matching variance to obtain a sorting result; filtering out a preset proportion of matching constraints at the end of the sorting result, and assigning corresponding residual weights to the unfiltered matching constraints according to the sorting result; wherein, the matching constraints closer to the beginning of the sorting result are assigned larger residual weights.

[0168] The preset threshold is a pre-defined numerical standard used to determine when to begin sorting and filtering. When the number of matching constraints corresponding to the laser point cloud exceeds the preset threshold, it indicates that there are too many matching constraints. In this case, some constraints can be filtered out before further matching, and sorting and filtering can begin.

[0169] Table 1

[0170]

[0171] For example, as shown in Table 1, the laser point cloud includes: laser point 1, laser point 2, laser point 3, laser point 4, laser point 5, laser point 6, laser point 7, laser point 8, laser point 9, and laser point 10. The corresponding matching constraints include: matching constraint 1, matching constraint 2, matching constraint 3, matching constraint 4, matching constraint 5, matching constraint 6, matching constraint 7, matching constraint 8, matching constraint 9, and matching constraint 10. The number of matching constraints corresponding to the laser point cloud is 10. Assuming a preset threshold of 6, the number of matching constraints is greater than the preset threshold, indicating a large number of matching constraints. Therefore, some constraints can be filtered out before further matching.

[0172] After obtaining the matching residual (i.e., the distance from the laser point to the local map tile) for each laser point, the matching constraints corresponding to each laser point are sorted in ascending order of their matching residuals. In this sorting, the earlier the matching constraint appears, the smaller its matching residual.

[0173] Table 2

[0174]

[0175] For example, as shown in Table 2, the matching residuals corresponding to laser point 1 are 3.1, 2.9, 3.5, 4.2, 5, 6.3, 7, 8, 9, and 10, respectively. The order of the matching residuals from smallest to largest is: laser point 6: 2.5, laser point 2: 2.9, laser point 1: 3.1, laser point 7: 3.3, laser point 3: 3.5, laser point 8: 3.8, laser point 4: 4.2, laser point 9: 4.5, laser point 10: 6, and laser point 5: 6.3. Based on the order of matching residuals from smallest to largest, the matching constraints corresponding to each laser point are sorted, and the sorting results are as follows: matching constraint 6, matching constraint 2, matching constraint 1, matching constraint 7, matching constraint 3, matching constraint 8, matching constraint 4, matching constraint 9, matching constraint 10, and matching constraint 5.

[0176] It's understandable that a smaller matching residual indicates a higher matching quality for the current laser point, while a larger matching residual indicates a lower matching quality. Setting a preset ratio will filter out the matching constraints at the end of the sorting, that is, filter out the laser point with the lowest matching quality and its corresponding matching constraints, thus obtaining laser points with higher matching quality and their corresponding matching constraints.

[0177] For example, if the preset ratio is 10%, then it can be determined that the last matching constraint in the order of the 10 matching constraints, namely matching constraint 5, needs to be selected, resulting in the filtered matching constraints: matching constraint 6, matching constraint 2, matching constraint 1, matching constraint 7, matching constraint 3, matching constraint 8, matching constraint 4, matching constraint 9, and matching constraint 10.

[0178] In some embodiments, after filtering, the unfiltered matching constraints can be assigned weights in descending order of their ranking, meaning that matching constraints closer to the top of the ranking have larger residual weights. For example, matching constraint 6 can be assigned the largest weight, matching constraint 2 the second largest, and so on. Based on this, the residual weight of the matching constraint corresponding to each laser point is obtained. These weights can increase the weight of matching constraints with smaller matching residuals, highlight the matching constraints corresponding to laser points with high matching quality, and improve the robustness and accuracy of the algorithm.

[0179] Table 3

[0180]

[0181] For example, as shown in Table 3, a target weight of 8 is assigned to matching constraint 6, a target weight of 7.4 is assigned to matching constraint 2, a target weight of 6.8 is assigned to matching constraint 1, a target weight of 6 is assigned to matching constraint 7, a target weight of 5.6 is assigned to matching constraint 3, a target weight of 5 is assigned to matching constraint 8, a target weight of 4.2 is assigned to matching constraint 4, a target weight of 3.6 is assigned to matching constraint 9, and a target weight of 2.8 is assigned to matching constraint 10, thus determining the residual weight of each matching constraint. Assuming that laser point p0 is laser point 10, the matching residual corresponding to laser point p0 can be determined to be 2.8.

[0182] In the above method, when the number of matching constraints exceeds the minimum threshold for matching constraints, the matching constraints are sorted in ascending order based on their matching residuals. In this sorting, the earlier a matching constraint appears, the smaller its matching residual; a smaller residual indicates a higher matching quality for the current laser point. Eliminating a predetermined proportion of the last-ranked matching constraints further refines the matching constraints of the laser point cloud, removing potential low-quality and erroneous matches, thus improving the robustness and accuracy of the matching algorithm. For unfiltered matching constraints, their residual weights are decreased according to the sorting order to increase the weight of matching constraints with small residuals, highlighting the matching constraints corresponding to high-quality laser points, further improving the robustness and accuracy of the matching algorithm.

[0183] In step 104, the above four weights can be calculated for each laser point. The matching constraints corresponding to each laser point can be weighted based on the weights corresponding to each laser point to obtain the weighted matching constraints.

[0184] Specifically, the four weights corresponding to each laser point can be added together to obtain the target weight. Each matching constraint can be multiplied by the corresponding target weight and weighted to obtain the weighted matching constraint.

[0185] For example, the matching constraint corresponding to laser point p0 is: matching constraint 10, and the corresponding structural weight ω s (p i The value is approximately equal to 0.555, corresponding to the depth weight ω. r (p i The value is approximately 0.769, corresponding to a probability weight of 1.2 and a residual weight of 2.8. The four weights can be added together to obtain the target weight of 0.555 + 0.769 + 1.2 + 2.8 = 5.324. The matching constraint 10 is multiplied by the target weight obtained by the sum to obtain the weighted matching constraint 10.

[0186] Specifically, the matching constraints corresponding to each laser point can include distance constraints and angle constraints. The distance and angle constraints included in each matching constraint can be multiplied by their respective target weights to perform a weighted calculation, resulting in a weighted matching constraint.

[0187] Table 4

[0188] Matching constraints Target weight Distance constraints, angle constraints Weighted matching constraints Matching constraint 6 12 3、30° 36、360° Matching constraint 2 10.7 5、40° 53.5、428° Matching constraint 1 9.2 6.2、33° 57.04、303.6° Matching constraint 7 8 5.8、42° 46.4、336° Matching constraint 3 7.2 7、10° 50.4、72° Matching constraint 8 6 5、12° 30、72° Matching constraint 4 5.2 6、13° 31.2、67.6° Matching constraint 9 4.1 5、21° 20.5、86.1° Matching constraint 10 5.324 3、15° 15.927、79.86°

[0189] For example, the matching constraints corresponding to each laser point include: distance constraints and angle constraints. As shown in Table 4, matching constraint 6 includes distance and angle constraints of 3 and 30° respectively, with a corresponding target weight of 12; matching constraint 2 includes distance and angle constraints of 5 and 40° respectively, with a corresponding target weight of 10.7; matching constraint 1 includes distance and angle constraints of 6.2 and 33° respectively, with a corresponding target weight of 9.2; matching constraint 7 includes distance and angle constraints of 5.8 and 42° respectively, with a corresponding target weight of 8; matching constraint 3 includes distance and angle constraints of 7 and 10° respectively, with a corresponding target weight of 7.2; matching constraint 8 includes distance and angle constraints of 5 and 12° respectively, with a corresponding target weight of 6; matching constraint 4 includes distance and angle constraints of 6 and 13° respectively, with a corresponding target weight of 5.2; matching constraint 9 includes distance and angle constraints of 5 and 21° respectively, with a corresponding target weight of 4.1; and matching constraint 10 includes distance and angle constraints of 3 and 15° respectively, with a corresponding target weight of 5.324. The distance and angle constraints included in the matching constraints are multiplied by their respective target weights and then weighted to obtain the following weighted matching constraints: 6: 36, 360°; 2: 53.5, 428°; 1: 57.04, 303.6°; 7: 46.4, 336°; 3: 50.4, 72°; 8: 30, 72°; 4: 31.2, 67.6°; 9: 20.5, 86.1°; and 10: 15.927, 79.86°.

[0190] In step 105, based on the weighted matching constraints corresponding to each laser point, the matching relationship between the acquired laser point cloud and the map point cloud can be obtained. The laser point cloud and the map point cloud are then matched based on this matching relationship to obtain the matching result. Specifically, the matching includes determining the map point that matches each laser point in the laser point cloud frame from the map point cloud based on the matching relationship, thus obtaining a matching pair, which is the matching result.

[0191] Understandably, for a frame of laser point cloud, the matching relationship between the laser point cloud and the map point cloud can be determined based on the matching constraints of each laser point in the laser point cloud. Based on the obtained matching relationship, the laser point cloud and the map point cloud are matched, and the map points matching each laser point in the laser point cloud are determined from the map point cloud, resulting in multiple matching pairs, which is the matching result.

[0192] Specifically, the weighted matching constraints corresponding to each laser point can be added together, and the sum can be divided by the total weight of the target weights corresponding to each laser point to obtain the matching relationship. When the matching constraints include constraint distance and constraint angle, the obtained matching relationship includes: final constraint distance and final constraint angle.

[0193] The final distance constraint is (36+53.5+57.04+46.4+50.4+30+31.2+20.5+15.927) / (12+10.7+9.2+8+7.2+6+5.2+4.1+5.324) = 340.967 / 67.724, which is approximately 5.035.

[0194] Final angle constraint = (360°+428°+303.6°+336°+72°+72°+67.6°+86.1°+79.86°) / (12+10.7+9.2+8+7.2+6+5.2+4.1+5.324) = 1805.16° / 67.724 = 26.64°.

[0195] After obtaining the matching relationship, based on the matching relationship: final distance constraint 5.035, final angle constraint 26.64°, the map point matching each laser point is determined from the map point cloud to obtain the matching pair, that is, the matching result.

[0196] In one possible implementation, after matching the laser point cloud and the map point cloud based on the adjusted matching constraints to obtain the matching result, the method further includes: transforming each laser point into a map coordinate system based on the matching result; determining a second distance between each laser point and its nearest map point in the map coordinate system; identifying laser points whose second distance is greater than a first threshold and less than the second threshold as target laser points; wherein the second threshold is greater than the first threshold; accumulating the second distances that are greater than the first threshold and less than a third threshold to obtain the total distance; wherein the third threshold is greater than the second threshold; and dividing the total distance by the number of target laser points to obtain the average matching error used to evaluate the matching quality.

[0197] The matching result obtained by matching the laser point cloud and the map point cloud is a matching pair. The pose, i.e. the robot's pose relative to the map, can be obtained based on the matching pair. The obtained pose is usually a rotation matrix and a translation matrix. Based on the rotation matrix and translation matrix, the laser points in the laser point cloud can be transformed into the map coordinate system where the map point cloud is located, so as to unify the laser point cloud and the map point cloud into the same coordinate system.

[0198] Once the coordinate system is determined, the map point closest to each laser point can be identified in the map coordinate system. Based on the distance formula between two points, the distance between each laser point and the nearest map point can be calculated and denoted as the second distance.

[0199] The robot can set three-level distance thresholds based on map resolution, and classify laser points with various secondary distances based on these three-level distance thresholds. Let the map resolution be `map_resol`, then the specific rules for setting the three-level distance thresholds are as follows:

[0200] Set the hit distance threshold `zero_dis` to 0.5 times the map resolution: `zero_dis = 0.5 * map_resol`. Set the inlier distance threshold `inlier_dis` to 1.5 times the map resolution: `inlier_dis = 1.5 * map_resol`. Set the maximum match distance threshold `max_match_dis` to 15 times the map resolution: `max_match_dis = 15.0 * map_resol`. Here, `zero_dis` is the first threshold mentioned above, `inlier_dis` is the second threshold mentioned above, and `max_match_dis` is the third threshold mentioned above.

[0201] For example, if the map resolution is 0.5m / pixel, then the first threshold zero_dis = 0.5 * 0.5 = 0.25, the second threshold inlier_dis = 1.5 * 0.5 = 0.75, and the third threshold max_match_dis = 15.0 * 0.5 = 7.5.

[0202] Since point clouds have a certain resolution, the probability of two matching points completely overlapping is very low. Therefore, when the second distance is less than the first threshold `zero_dis`, the laser point can be identified as the hit point, and the second distance corresponding to the hit point is set to 0 to reduce the discretization distance caused by the point cloud resolution. When the second distance is greater than the first threshold `zero_dis` and less than the second threshold `inlier_dis`, the laser point can be identified as a matching interior point, i.e., the aforementioned target laser point. The number of matching interior points can be accumulated to obtain the total number of matching interior points, i.e., the total number of the aforementioned target laser points.

[0203] When the second distance is greater than the second threshold `inlier_dis` and less than the third threshold `max_match_dis`, the laser point can be identified as an outside matching point. The second distances corresponding to the inside matching points and the outside matching points are summed to obtain the total distance. Dividing the total distance by the number of target laser points yields the average matching error used to evaluate matching quality. A smaller average matching error indicates higher matching quality.

[0204] Table 5

[0205]

[0206] For example, as shown in Table 5, the second distance corresponding to laser point 1 is 0.35, the second distance corresponding to laser point 2 is 0.15, the second distance corresponding to laser point 3 is 0.48, the second distance corresponding to laser point 4 is 1.25, the second distance corresponding to laser point 5 is 8.1, the second distance corresponding to laser point 6 is 0.13, the second distance corresponding to laser point 7 is 0.52, the second distance corresponding to laser point 8 is 3.8, the second distance corresponding to laser point 9 is 0.67, and the second distance corresponding to laser point 10 is 4.8. Among them, the second distances of laser points 2 and 6, 0.15 and 0.13 respectively, are less than the first threshold of 0.25. Therefore, the second distances of laser points 2 and 6 are set to 0. The second distances corresponding to laser points 1, 3, 7, and 9 are 0.35, 0.48, 0.52, and 0.67, respectively. These distances are greater than the first threshold of 0.25 and less than the second threshold of 0.75. Therefore, laser points 1, 3, 7, and 9 are identified as target laser points, and the number of target laser points is 4. The second distances corresponding to laser points 4, 8, and 10 are 1.25, 3.8, and 4.8, respectively. These distances are greater than the second threshold of 0.75 and less than the third threshold of 7.5. Therefore, laser points 4, 8, and 10 are identified as matching outliers. The total distance = 0.35 + 0.48 + 0.52 + 0.67 + 1.25 + 3.8 + 4.8 = 11.07. The average matching error = 11.07 / 4 = 2.7675.

[0207] In some embodiments, when the second distance is greater than the third threshold max_match_dis, the region of the collected laser point is considered to be a non-overlapping region with the map point cloud, and the laser point is filtered out. For example, in the embodiment described above, the second distance 8.1 corresponding to laser point 5 is greater than the third threshold 7.5, so laser point 5 is filtered out.

[0208] In the above method, by setting reasonable three-level distance thresholds to classify laser points, only the distance between the first and third thresholds is accumulated to obtain the total distance. This avoids the influence of noise points in non-overlapping areas on the matching quality assessment. Ignoring the second distance less than the first threshold reduces the discretized distance caused by point cloud resolution. This allows for a more accurate average matching error to be obtained based on the total distance and the total number of target laser points, thus achieving accurate matching quality assessment.

[0209] In one possible implementation, after dividing the total distance by the number of target laser points to obtain the average matching error, the method further includes: dividing the number of target laser points by the total number of laser points in the laser point cloud to obtain the point cloud overlap; performing a scaling transformation on the point cloud overlap and then a cubic transformation to obtain the map overlap weight; and weighting the average matching error based on the map overlap weight to obtain the weighted matching error used to evaluate the matching quality.

[0210] Understandably, in some cases of incorrect matching, some laser points may match the map point cloud, and the nearest matching points are very close, while the matching distances of other points are greater than the maximum matching distance threshold. In such cases, a smaller average matching error may be calculated. Therefore, a map overlap weight can be calculated based on the number of target laser points (i.e., the points within the matching area mentioned above) to weight the average matching error, avoiding the erroneous judgment that the average matching error is small when only a few points overlap.

[0211] Specifically, the number of target laser points is divided by the total number of laser points in the laser point cloud to obtain the point cloud overlap ratio (overlap_ratio). The point cloud overlap ratio is multiplied by a preset coefficient to scale the point cloud overlap ratio, resulting in the scaled point cloud overlap ratio (overlap_trans). Then, the scaled point cloud overlap ratio (overlap_trans) is cubically transformed to obtain the map overlap weight (overlap_weight).

[0212] The preset coefficient for scaling transformation can be set such that the overlap after scaling transformation is greater than 1 when the overlap is greater than 0.72. For example, the preset coefficient can be 1.4. The formula for calculating the map overlap weight is shown in formula (20):

[0213]

[0214] When the overlap after scaling is greater than 1, performing a cubic transformation on overlap_trans greater than 1 will cause overlap_weight to increase dramatically with a cubic gradient, increasing the weight of map overlap when the overlap is high, with a maximum overlap_weight of 2.74. Conversely, if the map overlap is less than 0.72, overlap_weight will decrease dramatically as the overlap decreases; for example, when the map overlap is 0.3, overlap_weight is only 0.074.

[0215] For example, in the above embodiment, the laser point cloud includes a total of 10 laser points and 4 internal points. Then the point cloud overlap ratio is overlap_ratio = 0.4, overlap_trans = 1.4 * 0.4 = 0.56, and overlap_weight = 0.56 * 0.56 * 0.56 = 0.15488.

[0216] The greater the overlap weight, the higher the matching quality should be, that is, the smaller the matching error should be. Therefore, the average matching error can be divided by the overlap weight to obtain the weighted matching error. The formula for calculating the weighted matching error is shown in the following formula (21):

[0217] match_error=ave_match_dis / overlap_weight (21)

[0218] Here, match_error is the weighted matching error, and ave_match_dis is the average matching error.

[0219] For example, as in the above embodiment, the calculated average matching error ave_match_dis is 2.7675, and the overlap weight is 0.15488. Substituting the average matching error of 2.7675 and the overlap weight of 0.15488 into the above formula (21) for calculation, the weighted matching error match_error = 2.7675 / 0.15488 is approximately equal to 17.91. The weighted average matching error is relatively large, indicating that the matching quality is generally average.

[0220] In the above method, after determining the average matching error, the map overlap is calculated, and the overlap weight is calculated based on the map overlap to weight the average matching error, so as to obtain the weighted matching error to evaluate the matching quality. This can avoid situations where some laser points match the map point cloud and the nearest matching point is very close, while the matching distance of other points is greater than the maximum matching distance threshold. In such cases, a smaller average matching error may also be calculated.

[0221] In summary, this application unifies the matching process and the calculation method of matching residuals for 2D and 3D point clouds by establishing a geometric model from points to local map tiles. By calculating weights across multiple dimensions to weight matching constraints, it achieves multi-dimensional weighting of each matching constraint, enhancing the strength of high-quality matching constraints and reducing the impact of low-quality matching constraints, thereby improving the robustness and accuracy of the matching algorithm. Setting reasonable three-level distance thresholds can avoid the influence of noise points in non-overlapping regions on matching quality evaluation, reduce the discretization distance caused by point cloud resolution, and achieve accurate matching quality evaluation. Further weighting using nonlinear point cloud overlap yields accurate and robust weighted matching errors, achieving a more accurate evaluation of matching quality.

[0222] Figure 5 This is a schematic diagram of the structure of a point cloud matching device provided in an embodiment of this application.

[0223] For example, such as Figure 5 As shown, the device 500 includes:

[0224] The acquisition module 501 is used to acquire the laser point cloud currently collected by the lidar;

[0225] Module 502 is used to construct matching constraints for each laser point in the laser point cloud when matching each laser point with the map point cloud;

[0226] The calculation module 503 is used to calculate the weights used to describe the matching quality of each laser point from multiple dimensions;

[0227] The weighting module 504 is used to weight the matching constraints corresponding to each laser point based on the weights, so as to obtain the weighted matching constraints;

[0228] The matching module 505 is used to match laser point clouds and map point clouds based on weighted matching constraints to obtain matching results.

[0229] In one possible implementation, the calculation module 503 is specifically used to: for each laser point in the laser point cloud, determine the N map points closest to the laser point from the map point cloud, and construct a local map patch corresponding to the laser point based on the N map points; where N is greater than 1; and calculate the weights used to describe the matching quality of each laser point from multiple dimensions based on the local map patch.

[0230] In one possible implementation, the weights include at least one of structural weights, depth weights, probability weights, and residual weights. Specifically, the calculation module 503 is used to: determine the structural fitting error of the local map patch corresponding to the laser point; and determine the structural weight corresponding to the laser point based on the structural fitting error and the map resolution corresponding to the map point cloud; determine the depth weight corresponding to the laser point based on the depth of the laser point and the maximum measurement distance of the lidar; determine the probability weight corresponding to the laser point based on the average observation probability of the local map patch corresponding to the laser point; and determine the matching residual corresponding to the laser point based on the laser point and the local map patch corresponding to the laser point, and determine the residual weight corresponding to the laser point based on the matching residual.

[0231] In one possible implementation, the calculation module 503 is specifically used to calculate the first distance from N map points to the local map patch respectively; calculate the average value of the first distance from N map points to the local map patch, and determine the average value as the structure fitting error of the local map patch.

[0232] In one possible implementation, the weighting module 504 is specifically used to sort the matching constraints corresponding to each laser point in ascending order of matching residuals when the number of matching constraints corresponding to the laser point cloud is greater than a preset threshold, and obtain a sorting result; filter out the matching constraints at the end of the sorting result with a preset proportion, and assign corresponding residual weights to the unfiltered matching constraints according to the sorting result; wherein, the matching constraints closer to the head of the sorting result are assigned larger residual weights.

[0233] In one possible implementation, the device 500 further includes: an evaluation module, configured to transform each laser point into a map coordinate system based on the matching results; determine a second distance between each laser point and its nearest map point in the map coordinate system; identify laser points whose second distance is greater than a first threshold and less than the second threshold as target laser points; wherein the second threshold is greater than the first threshold; accumulate the second distances that are greater than the first threshold and less than a third threshold to obtain a total distance; wherein the third threshold is greater than the second threshold; and divide the total distance by the number of target laser points to obtain the average matching error used to evaluate the matching quality.

[0234] In one possible implementation, the weighting module 504 is further configured to divide the number of target laser points by the total number of laser points in the laser point cloud to obtain the point cloud overlap; perform a scaling transformation on the point cloud overlap and then perform a cubic transformation to obtain the map overlap weight; and weight the average matching error based on the map overlap weight to obtain the weighted matching error used to evaluate the matching quality.

[0235] In one possible implementation, the device 500 further includes an expansion module, which, before determining the N closest map points to the laser point from the map point cloud for each laser point in the laser point cloud, expands the laser point cloud and map point cloud from two dimensions to three dimensions based on a preset value of the third dimension, in the case that the lidar is determined to be a 2D lidar, to obtain a three-dimensional laser point cloud and map point cloud.

[0236] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0237] For example, such as Figure 6 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 stores executable program code 6011, and the processor 602 is used to call and execute the executable program code 6011 to perform a point cloud matching method.

[0238] Furthermore, embodiments of this application also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a point cloud matching method provided in embodiments of this application.

[0239] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0240] When each functional module is divided according to its corresponding function, the device may further include an acquisition module, a determination module, a calculation module, a weighting module, and an adjustment module. It should be noted that all relevant content regarding the steps involved in the above method embodiments can be referenced to the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0241] It should be understood that the apparatus provided in this embodiment is used to perform the point cloud matching method described above, and therefore can achieve the same effect as the above implementation method.

[0242] When using integrated units, the device may include a processing module and a storage module. When applied to an electronic device, the processing module can be used to control and manage the operation of the electronic device. The storage module can be used to support the execution of relevant program code by the electronic device.

[0243] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0244] In addition, the apparatus provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a point cloud matching method provided in the above embodiments.

[0245] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described related method steps to implement the point cloud matching method provided in the above embodiment.

[0246] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement a point cloud matching method provided in the above embodiment.

[0247] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0248] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0249] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0250] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for point cloud matching, characterized in that, The method includes: Acquire the laser point cloud currently collected by the lidar; For each laser point in the laser point cloud, construct matching constraints when matching each laser point with the map point cloud; Weights used to describe the matching quality of each laser point are calculated from multiple dimensions; Based on the weights, the matching constraints corresponding to each laser point are weighted to obtain the weighted matching constraints. Based on the weighted matching constraints, the laser point cloud and the map point cloud are matched to obtain the matching result.

2. The method according to claim 1, characterized in that, The calculation of weights from multiple dimensions to describe the matching quality of each laser point includes: For each laser point in the laser point cloud, determine the N map points closest to the laser point from the map point cloud, and construct a local map patch corresponding to the laser point based on the N map points; where N is greater than 1. Based on the local map tiles, weights for describing the matching quality of each laser point are calculated from multiple dimensions.

3. The method according to claim 2, characterized in that, The weights include at least one of structural weights, depth weights, probability weights, and residual weights. The weights used to describe the matching quality of each laser point, calculated from multiple dimensions based on the local map patch, include at least one of the following: Determine the structural fitting error of the local map patch corresponding to the laser point, and determine the structural weight corresponding to the laser point based on the structural fitting error and the map resolution corresponding to the map point cloud; The depth weight corresponding to the laser point is determined based on the depth of the laser point and the maximum measurement distance of the lidar. The probability weight corresponding to the laser point is determined based on the average observation probability of the local map patch corresponding to the laser point. Based on the laser point and the local map patch corresponding to the laser point, the matching residual corresponding to the laser point is determined, and the residual weight corresponding to the laser point is determined based on the matching residual corresponding to the laser point.

4. The method according to claim 3, characterized in that, The determination of the structural fitting error of the local map patch corresponding to the laser point includes: Calculate the first distance from each of the N map points to the local map tile; Calculate the average of the first distances from the N map points to the local map patch, and determine the average value as the structure fitting error of the local map patch.

5. The method according to claim 3 or 4, characterized in that, The step of determining the residual weight corresponding to the laser point based on the matching residual corresponding to the laser point includes: If the number of matching constraints corresponding to the laser point cloud is greater than a preset threshold, the matching constraints corresponding to each laser point are sorted in ascending order of the matching residuals to obtain a sorting result. The matching constraints at the end of the sorting results are filtered out by a predetermined proportion, and the unfiltered matching constraints are assigned corresponding residual weights according to the sorting results; wherein, the matching constraints closer to the head of the sorting results are assigned larger residual weights.

6. The method according to any one of claims 1 to 4, characterized in that, After matching the laser point cloud and the map point cloud based on the weighted matching constraints to obtain the matching result, the method further includes: Based on the matching results, each laser point is transformed into a map coordinate system; In the map coordinate system, a second distance is determined between each laser point and the nearest map point; Laser points whose second distance is greater than the first threshold and less than the second threshold are identified as target laser points; wherein, the second threshold is greater than the first threshold; The total distance is obtained by accumulating the second distances that are greater than the first threshold and less than the third threshold; wherein the third threshold is greater than the second threshold. Dividing the total distance by the number of target laser points yields the average matching error used to evaluate the matching quality.

7. The method according to claim 6, characterized in that, After dividing the total distance by the number of target laser points to obtain the average matching error, the method further includes: The point cloud overlap is obtained by dividing the number of target laser points by the total number of laser points in the laser point cloud. After scaling the point cloud overlap, a cubic transformation is performed again to obtain the map overlap weight. The average matching error is weighted based on the map overlap weight to obtain a weighted matching error used to evaluate the matching quality.

8. The method according to any one of claims 2 to 4, characterized in that, Before determining the N closest map points to each laser point in the laser point cloud from the map point cloud, the method further includes: If the lidar is determined to be a 2D lidar, the lidar point cloud and map point cloud are expanded from two dimensions to three dimensions based on a preset value of the third dimension, so as to obtain the three-dimensional lidar point cloud and map point cloud.

9. A point cloud matching device, characterized in that, The device includes: The acquisition module is used to acquire the laser point cloud currently collected by the lidar; The construction module is used to construct matching constraints for each laser point in the laser point cloud when matching each laser point with the map point cloud; A calculation module is used to calculate weights from multiple dimensions to describe the matching quality of each laser point; The weighting module is used to weight the matching constraints corresponding to each laser point based on the weights, so as to obtain the weighted matching constraints; The matching module is used to match the laser point cloud and the map point cloud based on the weighted matching constraints to obtain the matching result.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the electronic device to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 8.

Images