Semantic feature based relative pose calculation method and device, equipment and medium

Abstract

The application relates to the technical field of automatic driving, and discloses a relative pose calculation method and device based on semantic features, equipment and a medium. The application extracts features based on semantic segmentation results of multiple frames of laser point cloud data to obtain an edge feature point list and a surface feature point list, and the surface feature point list comprises a ground feature point sub-list and a non-ground feature point sub-list. Then, point cloud registration is performed based on the edge feature point list and the non-ground feature point sub-list to obtain a first relative pose in a two-dimensional Euclidean space. Point cloud registration is performed on the ground feature point sub-list based on the first relative pose to obtain a target relative pose in a three-dimensional Euclidean space. Thus, edge features and surface features are extracted through semantic feature information of laser point cloud data, manual feature extraction is not needed, the calculation efficiency is improved, and the target relative pose of any frame is calculated by using the extracted edge features and surface features, so that the target of laser odometry is achieved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and specifically to a method, apparatus, device, and medium for calculating relative pose based on semantic features. Background Technology

[0002] For autonomous driving perception modules, the static and dynamic states of target obstacles can provide crucial clues for downstream control modules. Since autonomous vehicles are in motion most of the time, accurately compensating for their own state is particularly important. LiDAR odometry utilizes lidar for recursive position and pose estimation. Using lidar odometry data to compensate for the vehicle's motion trend effectively avoids the time synchronization and spatial parameter calibration operations required with other sensors such as vision and wheel speed sensors. Furthermore, lidar odometry offers advantages in accuracy compared to other sensors.

[0003] Currently, most classic laser odometry systems are based on algorithms that use LiDAR for real-time mapping (LidarOdometry and Mapping, LOAM). When processing point clouds, it is necessary to manually extract geometric features such as edges and planes, and then construct an optimized residual function based on the changes in geometric features to calculate the final pose information. Manual feature extraction currently relies on curvature smoothness to extract surface features and depth discontinuities to extract edge features. This method not only fails to perceive the category information of the point cloud being processed, but is also extremely time-consuming and inefficient. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device and medium for relative pose calculation based on semantic features, in order to solve the problems that related laser odometry methods cannot perceive point cloud categories and are inefficient.

[0005] In a first aspect, the present invention provides a relative pose calculation method based on semantic features, the method comprising:

[0006] Acquire multi-frame laser point cloud data of the vehicle's surrounding environment, and obtain the semantic segmentation result of each frame of laser point cloud data based on the multi-frame laser point cloud data;

[0007] Based on the semantic segmentation results, feature extraction is performed on multiple frames of laser point cloud data to obtain the edge feature point list and the surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes the ground feature point sub-list and the non-ground feature point list.

[0008] Point cloud registration is performed based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0009] Point cloud registration is performed on the list of ground feature points based on the first relative pose to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space.

[0010] This method acquires multi-frame laser point cloud data of the vehicle's surrounding environment, obtaining semantic segmentation results for each frame. Next, feature extraction is performed based on the semantic segmentation results, yielding lists of edge feature points and area feature points. The area feature point list includes lists of ground feature points and non-ground feature points, eliminating the need for manual geometric feature extraction and improving efficiency and accuracy. Then, point cloud registration is performed based on the edge and non-ground feature point lists to obtain the first relative pose in Euclidean two-dimensional space. Point cloud registration is then performed on the ground feature point list based on the first relative pose to obtain the target's relative pose in Euclidean three-dimensional space. Thus, by using the semantic feature information from the acquired multi-frame laser point cloud data, edge and area features are extracted, eliminating the need for manual feature extraction and improving computational efficiency. The extracted edge and area features are then used to calculate the target's relative pose in any frame, completing the laser odometry objective.

[0011] In one optional implementation, point cloud registration is performed on the list of ground feature points based on the first relative pose to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space, including:

[0012] Based on the first relative pose, fixed parameters in the target relative pose of the current frame relative to the starting frame in Euclidean 3D space are obtained.

[0013] By performing point cloud registration on a list of ground feature points, non-fixed parameters in the target's relative pose are determined.

[0014] The relative pose of the target in the current frame with respect to the starting frame is determined based on fixed and non-fixed parameters.

[0015] Based on the first relative pose obtained in Euclidean two-dimensional space, the fixed parameters in the target relative pose are determined. Then, by performing point cloud registration on the list of ground feature points, the non-fixed parameters in the target relative pose are determined. By combining the semantic features of the laser point cloud data, the target relative pose is calculated, thus improving the calculation accuracy.

[0016] In one optional implementation, the edge feature point list includes multiple edge feature points, and the non-ground feature point sub-list includes multiple non-ground feature points; point cloud registration is performed based on the edge feature point list and non-ground feature point list corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space, including:

[0017] The coordinate transformation of the laser point cloud data of the current frame and the previous frame is performed by a preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame.

[0018] Perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and calculate the edge feature loss value based on the feature point coordinates;

[0019] The non-ground feature points of the current frame are registered with the non-ground feature points of the previous frame to obtain feature surfaces, and the surface feature loss value is calculated based on the feature surfaces and feature point coordinates.

[0020] Based on edge feature loss and surface feature loss, the second relative pose is iteratively updated to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0021] By combining the semantic feature information of laser point cloud data, edge feature loss values ​​and surface feature loss values ​​are calculated in different ways. Based on the edge feature loss values ​​and surface feature loss values, the value of the second relative pose is iteratively optimized to improve the calculation accuracy of the relative pose.

[0022] In one optional implementation, the surface feature loss value is calculated based on the feature surface and the coordinates of the feature points, including:

[0023] Based on the coordinates of the feature points, calculate the normal vector of the feature surface and the intercept of the feature points in the current frame from the feature surface;

[0024] Calculate the product between the feature point coordinates and the normal vector of the current frame, and obtain the surface feature loss value based on the sum of the product and the intercept.

[0025] Therefore, the degree of matching of point cloud registration is measured by calculating the surface feature loss value based on the feature surface and feature point coordinates.

[0026] In one optional implementation, the second relative pose is iteratively updated based on edge feature loss values ​​and surface feature loss values ​​to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space, including:

[0027] The registration loss value is obtained by summing the edge feature loss value and the surface feature loss value.

[0028] Determine whether the registration loss value is lower than the preset loss threshold;

[0029] If the registration loss value is lower than the preset loss threshold, the current second relative pose is taken as the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0030] If the registration loss value is not lower than the preset loss threshold, the second relative pose is iteratively updated along the direction of decreasing registration loss value, and the process of performing coordinate transformation on the laser point cloud data of the current frame and the previous frame using the preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame is repeated until the registration loss value is lower than the preset loss threshold.

[0031] By judging whether the sum of the edge feature loss value and the surface feature loss value is lower than a preset threshold, the value of the second relative pose is iteratively optimized until the first relative pose that meets the conditions is obtained, thereby improving the calculation accuracy of the relative pose.

[0032] In one optional implementation, multiple frames of laser point cloud data of the vehicle's surrounding environment are acquired, and based on the multiple frames of laser point cloud data, a semantic segmentation result for each frame of laser point cloud data is obtained, including:

[0033] Multi-frame laser point cloud data of the vehicle's surrounding environment is acquired using lidar.

[0034] Multiple frames of laser point cloud data are input into a pre-trained semantic segmentation network to obtain the semantic segmentation result of each frame of laser point cloud data.

[0035] Therefore, semantic segmentation results of laser point cloud data are obtained through semantic segmentation network, so as to use the semantic segmentation results for feature extraction of point cloud data.

[0036] In one optional implementation, the semantic segmentation result includes at least one of tree trunk point cloud data, pole point cloud data, guardrail point cloud data, building point cloud data, traffic sign point cloud data, and ground point cloud data;

[0037] Feature extraction is performed on multiple frames of laser point cloud data based on semantic segmentation results, resulting in a list of edge feature points and a list of surface feature points for each frame of laser point cloud data, including:

[0038] Edge features are extracted from tree trunk point cloud data and / or pole point cloud data and / or guardrail point cloud data in multiple frames of laser point cloud data to obtain a list of edge feature points corresponding to each frame of laser point cloud data.

[0039] Surface features are extracted from building point cloud data and / or traffic sign point cloud data and / or ground point cloud data in multiple frames of laser point cloud data to obtain a list of surface feature points corresponding to each frame of laser point cloud data.

[0040] Therefore, feature extraction is performed based on the semantic segmentation results of laser point cloud data, eliminating the need for manual extraction of geometric features and improving efficiency and accuracy.

[0041] Secondly, the present invention provides a relative pose calculation device based on semantic features, the device comprising:

[0042] The first processing module is used to acquire multi-frame laser point cloud data of the vehicle's surrounding environment and, based on the multi-frame laser point cloud data, obtain the semantic segmentation result of each frame of laser point cloud data.

[0043] The second processing module is used to extract features from multiple frames of laser point cloud data based on semantic segmentation results, and obtain the edge feature point list and the surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes a ground feature point sub-list and a non-ground feature point list.

[0044] The third processing module is used to perform point cloud registration based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data, so as to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0045] The fourth processing module is used to perform point cloud registration on the list of ground feature points based on the first relative pose, so as to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space.

[0046] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the relative pose calculation method based on semantic features as described in the first aspect or any corresponding embodiment thereof.

[0047] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the semantic feature-based relative pose calculation method of the first aspect or any corresponding embodiment thereof.

[0048] The beneficial effects of this invention are:

[0049] By acquiring multiple frames of laser point cloud data of the vehicle's surrounding environment, semantic segmentation results are obtained for each frame. Next, feature extraction is performed based on the semantic segmentation results, yielding lists of edge feature points and surface feature points. The surface feature point list includes lists of ground feature points and non-ground feature points, eliminating the need for manual geometric feature extraction and improving efficiency and accuracy. Then, point cloud registration is performed based on the edge and non-ground feature point lists to obtain the first relative pose in Euclidean two-dimensional space. Point cloud registration is then performed on the ground feature point list based on the first relative pose to obtain the target's relative pose in Euclidean three-dimensional space. Thus, edge and surface features are extracted using the semantic feature information from the acquired multiple frames of laser point cloud data, eliminating the need for manual feature extraction and improving computational efficiency. The extracted edge and surface features are then used to calculate the target's relative pose in any frame, completing the laser odometry objective. Attached Figure Description

[0050] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating a relative pose calculation method based on semantic features according to an embodiment of the present invention.

[0052] Figure 2 This is a flowchart illustrating another relative pose calculation method based on semantic features according to an embodiment of the present invention.

[0053] Figure 3 This is a flowchart illustrating another relative pose calculation method based on semantic features according to an embodiment of the present invention;

[0054] Figure 4 This is a structural block diagram of a relative pose calculation device based on semantic features according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Automotive LiDAR is one of the most important sensors in the field of autonomous driving, primarily used to provide real-time vehicle location and depth information of the surrounding environment. In recent years, more and more OEMs have begun to equip mass-produced vehicles with LiDAR. The reason for LiDAR's popularity lies in its unique advantages: it can obtain three-dimensional point information, has high measurement accuracy, and is independent of external lighting conditions. LiDAR is increasingly becoming a standard sensor in mass-produced vehicles, which places more stringent demands on its perception algorithms. Meanwhile, for autonomous driving perception modules, accurately detecting the static or dynamic state of target obstacles is particularly important, as it provides crucial clues for downstream control modules to react correctly. Since autonomous vehicles are in motion most of the time, accurately compensating for the vehicle's state becomes especially critical.

[0058] Laser odometry, by comparing and calculating multiple continuous point clouds, can obtain the attitude and position information of an object at various moments, thereby achieving the perception and tracking of the object's motion state. Compared to traditional wheeled odometry, laser odometry has higher accuracy and reliability, and can adapt to more complex environments and scenarios. Using the results of laser odometry to compensate for the trend of the vehicle's motion during laser sensing and tracking can effectively avoid the time (time synchronization) and spatial (calibration parameter) information required when using odometry results from other sensors. At the same time, laser odometry also has an advantage in accuracy compared to other sensors (vision, speedometers, etc.). However, at present, laser odometry also suffers from large height offset and slow processing speed.

[0059] According to an embodiment of the present invention, a relative pose calculation method based on semantic features is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0060] This embodiment provides a relative pose calculation method based on semantic features, which can be used in computer devices or electronic devices that perform relative pose calculations, such as in-vehicle computers, mobile phones, tablets, etc. Figure 1 This is a flowchart of a relative pose calculation method based on semantic features according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0061] Step S101: Acquire multi-frame laser point cloud data of the vehicle's surrounding environment, and obtain the semantic segmentation result of each frame of laser point cloud data based on the multi-frame laser point cloud data.

[0062] Specifically, multi-frame laser point cloud data of the vehicle's surrounding environment is acquired using an onboard LiDAR. This multi-frame laser point cloud data is then input into a pre-trained semantic segmentation network to obtain the semantic segmentation result for each frame of laser point cloud data. It should be noted that a deep learning-based semantic segmentation network can be used for semantic segmentation, which can assign a category label to each point in the laser point cloud data, such as vehicle, pedestrian, road, building, bicycle, etc.

[0063] Therefore, semantic segmentation results of laser point cloud data are obtained through semantic segmentation network, so as to use the semantic segmentation results for feature extraction of point cloud data.

[0064] Specifically, the semantic segmentation results may include at least one of the following: tree trunk point cloud data, pole point cloud data, guardrail point cloud data, building point cloud data, traffic sign point cloud data, and ground point cloud data.

[0065] In this way, after the lidar point cloud data is processed by a deep learning semantic segmentation network, the semantic information of each point in the lidar point cloud data can be obtained. This includes not only "edge" features such as poles, tree trunks, and guardrails, but also "surface" features such as the ground, buildings, and traffic signs. By using this semantic information, accurate geometric features can be obtained, while consuming almost no computing power, thus improving computational efficiency.

[0066] Step S102: Based on the semantic segmentation results, feature extraction is performed on multiple frames of laser point cloud data to obtain the edge feature point list and the surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes the ground feature point sub-list and the non-ground feature point list.

[0067] Specifically, edge features are extracted from tree trunk point cloud data and / or pole-shaped object point cloud data and / or guardrail point cloud data in multiple frames of laser point cloud data to obtain a list of edge feature points corresponding to each frame of laser point cloud data; surface features are extracted from building point cloud data and / or traffic sign point cloud data and / or ground point cloud data in multiple frames of laser point cloud data to obtain a list of surface feature points corresponding to each frame of laser point cloud data.

[0068] Therefore, feature extraction is performed based on the semantic segmentation results of laser point cloud data, eliminating the need for manual extraction of geometric features and improving efficiency and accuracy. Furthermore, the geometric features extracted using semantic information are unaffected by dynamic feature points, and the segmented ground information can also be used to further enhance the accuracy of laser odometry.

[0069] Step S103: Perform point cloud registration based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0070] Specifically, for feature points in the edge feature point list and non-ground feature point sublist, the constraints provided in the yaw, x, and y directions are more sufficient. Therefore, the special Euclidean group SE(2) is first used to solve the vehicle's pose change. The special Euclidean group SE(2) refers to the special Euclidean group in Euclidean two-dimensional space. It is a two-dimensional matrix group composed of two-dimensional rotation matrices and translation matrices, which can be used to describe the motion state of an object in a two-dimensional plane, including position and orientation. The special Euclidean group SE(2) can be expressed as:

[0071]

[0072] in, It is a two-dimensional rotation matrix. It is a two-dimensional translation vector.

[0073] It should be noted that the form of the first relative pose is consistent with the special Euclidean group SE(2), which includes the relative position and relative attitude of the current frame relative to the starting frame, that is, the pose change of the vehicle relative to the starting point.

[0074] Step S104: Based on the first relative pose, perform point cloud registration on the list of ground feature points to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space.

[0075] Specifically, after obtaining the vehicle's first relative pose in Euclidean two-dimensional space, the value of the first relative pose is used to determine some parameters in the target relative pose. Point cloud registration is then performed using feature points from the ground feature point sub-list to obtain the final target relative pose. In this way, the relative pose of each frame relative to the starting frame can be obtained, achieving the goal of laser odometry: acquiring the vehicle's pose data in each laser scanning frame, thus providing data guidance and reference for downstream planning modules.

[0076] Specifically, the relative pose of the target can be obtained by solving the special Euclidean group SE(3). The form of the relative pose of the target is consistent with that of the special Euclidean group SE(3). The special Euclidean group SE(3) refers to the special Euclidean group in Euclidean three-dimensional space, which can be expressed as:

[0077]

[0078] in, Represents a three-dimensional rotation matrix. This represents a three-dimensional translation vector.

[0079] The semantic feature-based relative pose calculation method provided in this embodiment obtains semantic segmentation results for each frame of laser point cloud data by acquiring multi-frame laser point cloud data of the vehicle's surrounding environment. Next, feature extraction is performed based on the semantic segmentation results to obtain an edge feature point list and a surface feature point list. The surface feature point list includes a ground feature point sub-list and a non-ground feature point list, eliminating the need for manual extraction of geometric features and improving efficiency and accuracy. Then, point cloud registration is performed based on the edge feature point list and the non-ground feature point list to obtain the first relative pose in Euclidean two-dimensional space. Finally, point cloud registration is performed on the ground feature point list based on the first relative pose to obtain the target relative pose in Euclidean three-dimensional space.

[0080] By using the semantic feature information of multi-frame laser point cloud data of the vehicle's surrounding environment, edge and surface features can be extracted without manual feature extraction, thus improving computational efficiency. The extracted edge and surface features are then used to calculate the target's relative pose in any frame, thus completing the laser odometry target.

[0081] This embodiment provides a relative pose calculation method based on semantic features, which can be used in computer devices or electronic devices that perform relative pose calculations, such as in-vehicle computers, mobile phones, tablets, etc. Figure 2 This is a flowchart of a relative pose calculation method based on semantic features according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0082] Step S201: Acquire multi-frame laser point cloud data of the vehicle's surrounding environment, and based on this multi-frame data, obtain the semantic segmentation result for each frame. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0083] Step S202: Based on the semantic segmentation results, feature extraction is performed on multiple frames of laser point cloud data to obtain a list of edge feature points and a list of surface feature points corresponding to each frame of laser point cloud data; wherein, the list of surface feature points includes a list of ground feature points and a list of non-ground feature points. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0084] Step S203: Perform point cloud registration based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0085] Specifically, step S203 includes:

[0086] Step S2031: Perform coordinate transformation on the laser point cloud data of the current frame and the previous frame using the preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame.

[0087] Specifically, by transforming the point cloud coordinates to the map coordinate system, the feature point coordinates of each feature point are obtained. The feature point coordinates can be calculated using the following formula:

[0088]

[0089] Among them, P cur Represents the current point cloud coordinates. Represents the rotation matrix in the SE(2) space. P′ represents the translation vector in the SE(2) space. cur The coordinates of the feature points after the SE(2) spatial transformation are represented.

[0090] It should be noted that since laser odometry is a process of recursive pose estimation, the preset second relative pose can be obtained from the Euclidean two-dimensional space relative to the previous frame, that is, the relative position and relative attitude of the previous frame relative to the starting frame.

[0091] Step S2032: Perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and calculate the edge feature loss value based on the feature point coordinates.

[0092] Specifically, point cloud registration involves finding a feature point in the previous frame that matches the feature point in the current frame. If the feature point cur in the current frame is an edge feature point cur, find the nearest feature point A of feature point cur in the previous frame's point cloud, and find the next nearest point B around feature point A (within a range of several lines above and below). The feature line formed by (A, B) is called the correspondence of feature point cur in the previous frame's point cloud.

[0093] It should be noted that KD-trees (K-Dimensional Trees) can be used to find the correspondence of feature points in the current frame in the point cloud of the previous frame. KD-trees are mainly used to solve search problems in multi-dimensional space, such as nearest neighbor search and range search. For detailed implementation process, please refer to the description of relevant technologies, which will not be repeated here.

[0094] In some alternative implementations, the formula for calculating the edge feature loss value can be as follows:

[0095]

[0096] Among them, P edgeA The map coordinates representing feature point A. The map coordinates of feature point B, Cedge This represents the edge feature loss value.

[0097] Step S2033: Perform point cloud registration between the non-ground feature points of the current frame and the non-ground feature points of the previous frame to obtain feature surfaces, and calculate the surface feature loss value based on the feature surfaces and feature point coordinates.

[0098] Specifically, if the currently registered feature point cur belongs to the surface feature point cur, find the nearest feature point J of feature point cur in the previous frame point cloud, find the next nearest point L around feature point J, and then find M around L. The feature surface formed by (J, L, M) is called the corresponding feature point cur in the previous frame point cloud.

[0099] In some optional implementations, based on the feature point coordinates, the normal vector of the feature surface and the intercept of the feature point from the feature surface in the current frame are calculated. The product between the feature point coordinates and the normal vector in the current frame is then calculated. The surface feature loss value is obtained by summing the product and the intercept, i.e.:

[0100]

[0101] Where Norm represents the normal vector of the feature surface. Represents the intercept, C surf Representative surface feature loss value.

[0102] Therefore, the degree of matching of point cloud registration is measured by calculating the surface feature loss value based on the feature surface and feature point coordinates.

[0103] Step S2034: Based on the edge feature loss value and the surface feature loss value, the second relative pose is iteratively updated to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0104] Specifically, step S2034 above includes:

[0105] Step a1: Calculate the sum of the edge feature loss value and the surface feature loss value to obtain the registration loss value.

[0106] Step a2: Determine whether the registration loss value is lower than the preset loss threshold.

[0107] Step a3: If the registration loss value is lower than the preset loss threshold, then the current second relative pose is taken as the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0108] Step a4: If the registration loss value is not lower than the preset loss threshold, the second relative pose is iteratively updated along the direction of decreasing registration loss value, and the process returns to step S2031 until the registration loss value is lower than the preset loss threshold.

[0109] In some optional implementations, the optimal value is iteratively solved using the Levenberg-Marquardt (LM) optimization method based on edge feature loss values ​​and surface feature loss values. The LM algorithm is a commonly used trust-region-based nonlinear optimization algorithm. While the traditional Gauss-Newton method solves the problem of high computational complexity or even incomputability of the Hessian matrix in Newton iterations, it still suffers from issues such as non-positive definite matrix calculations and a large number of iteration steps. The Gauss-Newton method uses the following formula to calculate the updated data at each iteration step.

[0110]

[0111] Where J represents the Jacobian matrix, represents the update step size, and e represents the registration loss value of the current solution.

[0112] The LM method adds a damping term to the Gauss-Newton method, and updates it using the following formula:

[0113]

[0114] By adjusting the damping term μ, the positive definiteness of the optimization matrix can be guaranteed, thus preventing unsolvable problems. When μ is large, it approximates gradient descent, reducing the number of iterations. When μ is small, it approximates Gauss-Newton, avoiding incorrect optimal solutions.

[0115] By combining the semantic feature information of laser point cloud data, edge feature loss value and surface feature loss value are calculated in different ways. By judging whether the sum of edge feature loss value and surface feature loss value is lower than a preset threshold, the value of the second relative pose is iteratively optimized until the first relative pose that meets the conditions is obtained, thereby improving the calculation accuracy of relative pose.

[0116] Step S204: Based on the first relative pose, perform point cloud registration on the list of ground feature points to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space.

[0117] Specifically, step S204 includes:

[0118] Step S2041: Based on the first relative pose, obtain the fixed parameters in the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space.

[0119] Specifically, the target relative pose includes a three-dimensional rotation matrix and a three-dimensional translation vector, while the first relative pose includes a two-dimensional rotation matrix and a two-dimensional translation vector. For each two-dimensional rotation and translation transformation, it can be extended into Euclidean three-dimensional space to form a three-dimensional rotation and translation transformation. Therefore, fixed parameters in the target relative pose can be determined based on the value of the first relative pose. For details, please refer to the description of relevant technologies, which will not be elaborated here.

[0120] Step S2042: By performing point cloud registration on the list of ground feature points, the non-fixed parameters in the target's relative pose are determined.

[0121] Specifically, the non-fixed parameters in the target's relative pose can be determined by iteratively solving the problem by referring to the relevant descriptions of steps S2031 to S2034, which will not be elaborated here.

[0122] Step S2043: Determine the target relative pose of the current frame to the starting frame in Euclidean three-dimensional space based on fixed and non-fixed parameters.

[0123] Specifically, the target relative pose in Euclidean three-dimensional space is composed of fixed parameters and determined non-fixed parameters, which are the current frame relative to the starting frame.

[0124] Based on the first relative pose obtained in Euclidean two-dimensional space, the fixed parameters in the target relative pose are determined. Then, by performing point cloud registration on the list of ground feature points, the non-fixed parameters in the target relative pose are determined. By combining the semantic features of the laser point cloud data, the target relative pose is calculated, thus improving the calculation accuracy.

[0125] The semantic feature-based relative pose calculation method provided in this embodiment obtains semantic segmentation results for each frame of laser point cloud data by acquiring multi-frame laser point cloud data of the vehicle's surrounding environment. Next, feature extraction is performed based on the semantic segmentation results to obtain an edge feature point list and a surface feature point list. The surface feature point list includes a ground feature point sub-list and a non-ground feature point list, eliminating the need for manual geometric feature extraction and improving efficiency and accuracy. Then, point cloud registration is performed based on the edge feature point list and the non-ground feature point list to obtain the first relative pose in Euclidean two-dimensional space. Finally, fixed parameters in the target relative pose are obtained based on the first relative pose. By performing point cloud registration on the ground feature point list, non-fixed parameters in the target relative pose are determined, thereby obtaining the target relative pose.

[0126] By using the semantic feature information of multi-frame laser point cloud data of the vehicle's surrounding environment, edge and surface features can be extracted without manual feature extraction, thus improving computational efficiency. The extracted edge and surface features are then used to calculate the target's relative pose in any frame, thus completing the laser odometry target.

[0127] The relative pose calculation method based on semantic features of this invention will be further explained below with a specific application example, such as... Figure 3 As shown, this application example includes the following steps:

[0128] Step 1: Activate the vehicle-mounted LiDAR to collect laser point cloud data of the surrounding environment.

[0129] Step 2: Input the laser point cloud data into the semantic segmentation model to obtain the semantic segmentation result.

[0130] Step 3: Use the semantic segmentation results to extract the corresponding geometric features. Traverse the semantic segmentation results, extract the point cloud data belonging to tree trunks, poles, and guardrails, and store them in the edge feature point list. Extract the point cloud data belonging to buildings, traffic signs, and the ground, and store them in the surface feature point list, which includes a ground feature point sublist and a non-ground feature point sublist.

[0131] Step 4: Use the feature points in the non-ground feature point sublist (e.g., feature points corresponding to tree trunks, buildings, or traffic signs) to construct a residual function in the SE(2) space to solve for the relative pose and perform optimization.

[0132] Step 4.1: Perform coordinate transformation on the laser point cloud data of the current frame and the previous frame using a preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame. The point cloud coordinates are then transformed to the map coordinate system to obtain the feature point coordinates of each feature point. The feature point coordinates can be calculated using the following formula:

[0133]

[0134] Among them, P cur Represents the current point cloud coordinates. Represents the rotation matrix in the SE(2) space. P′ represents the translation vector in the SE(2) space. cur The coordinates of the feature points after the SE(2) spatial transformation are represented.

[0135] Step 4.2: Perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and calculate the edge feature loss value according to the following formula:

[0136]

[0137] Among them, P edgeA The map coordinates representing feature point A. The map coordinates of feature point B, C edge This represents the edge feature loss value.

[0138] Step 4.3: Perform point cloud registration between the non-ground feature points of the current frame and the non-ground feature points of the previous frame to obtain feature surfaces, and calculate the surface feature loss value according to the following formula:

[0139]

[0140] Where Norm represents the normal vector of the feature surface. Represents the intercept, C surf Representative surface feature loss value.

[0141] Step 4.4: Based on the edge feature loss value and the surface feature loss value, use the LM optimization method to iteratively solve for the optimal value and obtain the first relative pose.

[0142] Step 5: Fix the results obtained in Step 4, use the list of ground feature points, and continue to optimize in SE(3) space to obtain the final target relative pose, i.e., the laser odometry result.

[0143] Step 6: Update the feature map based on the laser odometry results to provide data guidance and reference for the downstream path planning module of the vehicle.

[0144] The relative pose calculation method based on semantic features of this invention obtains semantic information for each point in the laser point cloud by inputting the laser point cloud data acquired by LiDAR into a semantic segmentation module. This information includes not only "edge" features such as poles, tree trunks, and guardrails, but also "surface" features such as the ground, buildings, and traffic signs. Directly using this information not only yields accurate geometric features but also consumes almost no computational power. Furthermore, the geometric features extracted using semantic information are unaffected by dynamic feature points, and the segmented ground information can be used to further improve the accuracy of laser odometry. By replacing manually extracted geometric features with semantic features to construct a geometric residual function for optimization, accurate laser odometry results can be obtained.

[0145] This embodiment also provides a relative pose calculation device based on semantic features, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0146] This embodiment provides a relative pose calculation device based on semantic features, such as... Figure 4 As shown, it includes:

[0147] The first processing module 401 is used to acquire multi-frame laser point cloud data of the vehicle's surrounding environment and obtain the semantic segmentation result of each frame of laser point cloud data based on the multi-frame laser point cloud data.

[0148] The second processing module 402 is used to extract features from multiple frames of laser point cloud data based on semantic segmentation results, and obtain an edge feature point list and a surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes a ground feature point sublist and a non-ground feature point list.

[0149] The third processing module 403 is used to perform point cloud registration based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data, so as to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0150] The fourth processing module 404 is used to perform point cloud registration on the list of ground feature points based on the first relative pose to obtain the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space.

[0151] In some alternative implementations, the first processing module 401 includes:

[0152] The first processing unit is used to acquire multi-frame laser point cloud data of the vehicle's surrounding environment through lidar.

[0153] The second processing unit is used to input multiple frames of laser point cloud data into a pre-trained semantic segmentation network to obtain the semantic segmentation result of each frame of laser point cloud data.

[0154] In some optional implementations, the semantic segmentation result includes at least one of tree trunk point cloud data, pole point cloud data, guardrail point cloud data, building point cloud data, traffic sign point cloud data, and ground point cloud data; the second processing module 402 includes:

[0155] The third processing unit is used to extract edge features from tree trunk point cloud data and / or pole point cloud data and / or guardrail point cloud data in multiple frames of laser point cloud data to obtain a list of edge feature points corresponding to each frame of laser point cloud data.

[0156] The fourth processing unit is used to extract surface features from building point cloud data and / or traffic sign point cloud data and / or ground point cloud data in multiple frames of laser point cloud data to obtain a list of surface feature points corresponding to each frame of laser point cloud data.

[0157] In some optional implementations, the edge feature point list includes multiple edge feature points, and the non-ground feature point sublist includes multiple non-ground feature points; the third processing module 403 includes:

[0158] The fifth processing unit is used to perform coordinate transformation on the laser point cloud data of the current frame and the previous frame through a preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame.

[0159] The sixth processing unit is used to perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and to calculate the edge feature loss value based on the feature point coordinates.

[0160] The seventh processing unit is used to perform point cloud registration between the non-ground feature points of the current frame and the non-ground feature points of the previous frame to obtain feature surfaces, and to calculate the surface feature loss value based on the feature surfaces and feature point coordinates.

[0161] The eighth processing unit is used to iteratively update the second relative pose based on the edge feature loss value and the surface feature loss value to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space.

[0162] In some optional implementations, the seventh processing unit includes:

[0163] The first processing subunit is used to calculate the normal vector of the feature surface and the intercept of the feature point from the feature surface in the current frame based on the feature point coordinates.

[0164] The second processing subunit is used to calculate the product between the feature point coordinates and the normal vector of the current frame, and obtain the surface feature loss value based on the sum of the product and the intercept.

[0165] In some optional implementations, the eighth processing unit includes:

[0166] The third processing subunit is used to calculate the sum of the edge feature loss value and the surface feature loss value to obtain the registration loss value;

[0167] The fourth processing subunit is used to determine whether the registration loss value is lower than a preset loss threshold;

[0168] The fifth processing subunit is used to take the current second relative pose as the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space if the registration loss value is lower than the preset loss threshold.

[0169] The sixth processing subunit is used to iteratively update the second relative pose along the direction of decreasing registration loss value if the registration loss value is not lower than the preset loss threshold, and return to the step of performing coordinate transformation on the laser point cloud data of the current frame and the previous frame through the preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame, until the registration loss value is lower than the preset loss threshold.

[0170] In some alternative implementations, the fourth processing module 404 includes:

[0171] The ninth processing unit is used to obtain fixed parameters in the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space based on the first relative pose.

[0172] The tenth processing unit is used to determine the non-fixed parameters in the relative pose of the target by performing point cloud registration on the list of ground feature points;

[0173] The eleventh processing unit is used to determine the target relative pose of the current frame to the starting frame in Euclidean three-dimensional space based on fixed and non-fixed parameters.

[0174] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0175] In this embodiment, the relative pose calculation device based on semantic features is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0176] This invention also provides a computer device having the above-described features. Figure 4 The device shown is a relative pose calculation device based on semantic features.

[0177] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0178] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0179] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0180] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0181] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0182] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0183] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0184] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A relative pose calculation method based on semantic features, characterized in that, The method includes: Acquire multiple frames of laser point cloud data of the vehicle's surrounding environment, and obtain the semantic segmentation result of each frame of laser point cloud data based on the multiple frames of laser point cloud data; Based on the semantic segmentation results, feature extraction is performed on multiple frames of the laser point cloud data to obtain an edge feature point list and a surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes a ground feature point sublist and a non-ground feature point list. Point cloud registration is performed based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space. Based on the first relative pose, point cloud registration is performed on the list of ground feature points to obtain the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space. The edge feature point list includes multiple edge feature points, and the non-ground feature point sub-list includes multiple non-ground feature points; the point cloud registration based on the edge feature point list and non-ground feature point list corresponding to each frame of laser point cloud data to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space includes: The coordinate transformation of the laser point cloud data of the current frame and the previous frame is performed by a preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame. Perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and calculate the edge feature loss value based on the coordinates of the feature points; The non-ground feature points of the current frame are registered with the non-ground feature points of the previous frame to obtain feature surfaces, and the surface feature loss value is calculated based on the feature surfaces and the coordinates of the feature points. Based on the edge feature loss value and the surface feature loss value, the second relative pose is iteratively updated to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space. The step of performing point cloud registration on the list of ground feature points based on the first relative pose to obtain the target relative pose of the current frame relative to the starting frame in Euclidean 3D space includes: Based on the first relative pose, fixed parameters in the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space are obtained. By performing point cloud registration on the list of ground feature points, the non-fixed parameters in the relative pose of the target are determined; The target relative pose in the current frame relative to the starting frame is determined based on the fixed parameters and the non-fixed parameters in Euclidean three-dimensional space.

2. The method according to claim 1, characterized in that, The calculation of the surface feature loss value based on the feature surface and the coordinates of the feature points includes: Based on the coordinates of the feature points, calculate the normal vector of the feature surface and the intercept of the feature points in the current frame from the feature surface; Calculate the product between the feature point coordinates of the current frame and the normal vector, and obtain the surface feature loss value based on the sum of the product and the intercept.

3. The method according to claim 1, characterized in that, The step of iteratively updating the second relative pose based on the edge feature loss value and the surface feature loss value to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space includes: The registration loss value is obtained by summing the edge feature loss value and the surface feature loss value. Determine whether the registration loss value is lower than a preset loss threshold; If the registration loss value is lower than the preset loss threshold, then the current second relative pose is taken as the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space. If the registration loss value is not lower than the preset loss threshold, the second relative pose is iteratively updated along the direction of decreasing registration loss value, and the process of performing coordinate transformation on the laser point cloud data of the current frame and the previous frame using the preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame is repeated until the registration loss value is lower than the preset loss threshold.

4. The method according to claim 1, characterized in that, The acquisition of multiple frames of laser point cloud data of the vehicle's surrounding environment, and the obtaining of semantic segmentation results for each frame of laser point cloud data based on these multiple frames, includes: Multi-frame laser point cloud data of the vehicle's surrounding environment is acquired using lidar. Multiple frames of the laser point cloud data are input into a pre-trained semantic segmentation network to obtain the semantic segmentation result of each frame of laser point cloud data.

5. The method according to claim 4, characterized in that, The semantic segmentation results include at least one of the following: tree trunk point cloud data, pole-shaped object point cloud data, guardrail point cloud data, building point cloud data, traffic sign point cloud data, and ground point cloud data; The step involves extracting features from multiple frames of laser point cloud data based on the semantic segmentation results, obtaining a list of edge feature points and a list of surface feature points corresponding to each frame of laser point cloud data, including: Edge features are extracted from the tree trunk point cloud data and / or pole-shaped object point cloud data and / or guardrail point cloud data in multiple frames of laser point cloud data to obtain a list of edge feature points corresponding to each frame of laser point cloud data. Surface features are extracted from the building point cloud data and / or traffic sign point cloud data and / or ground point cloud data in multiple frames of the laser point cloud data to obtain a list of surface feature points corresponding to each frame of laser point cloud data.

6. A relative pose calculation device based on semantic features, characterized in that, The device includes: The first processing module is used to acquire multiple frames of laser point cloud data of the vehicle's surrounding environment, and based on the multiple frames of laser point cloud data, to obtain the semantic segmentation result of each frame of laser point cloud data. The second processing module is used to extract features from multiple frames of laser point cloud data based on the semantic segmentation results, and obtain an edge feature point list and a surface feature point list corresponding to each frame of laser point cloud data; wherein, the surface feature point list includes a ground feature point sublist and a non-ground feature point list. The third processing module is used to perform point cloud registration based on the list of edge feature points and the list of non-ground feature points corresponding to each frame of laser point cloud data, so as to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space. The fourth processing module is used to perform point cloud registration on the list of ground feature points based on the first relative pose to obtain the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space. The edge feature point list includes multiple edge feature points, and the non-ground feature point sub-list includes multiple non-ground feature points; the third processing module is further configured to: The coordinate transformation of the laser point cloud data of the current frame and the previous frame is performed by a preset second relative pose to obtain the feature point coordinates of each feature point in the current frame and the previous frame. Perform point cloud registration between the edge feature points of the current frame and the edge feature points of the previous frame, and calculate the edge feature loss value based on the coordinates of the feature points; The non-ground feature points of the current frame are registered with the non-ground feature points of the previous frame to obtain feature surfaces, and the surface feature loss value is calculated based on the feature surfaces and the coordinates of the feature points. Based on the edge feature loss value and the surface feature loss value, the second relative pose is iteratively updated to obtain the first relative pose of the current frame relative to the starting frame in Euclidean two-dimensional space. The fourth processing module is also used for: Based on the first relative pose, fixed parameters in the target relative pose of the current frame relative to the starting frame in Euclidean three-dimensional space are obtained. By performing point cloud registration on the list of ground feature points, the non-fixed parameters in the relative pose of the target are determined; The target relative pose in the current frame relative to the starting frame is determined based on the fixed parameters and the non-fixed parameters in Euclidean three-dimensional space.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the relative pose calculation method based on semantic features as described in any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the relative pose calculation method based on semantic features as described in any one of claims 1 to 5.

Images