Methods, apparatus and storage media for extracting rotation-invariant features of 3D point clouds

Abstract

This application provides a method, apparatus, and storage medium for extracting rotation-invariant features of a 3D point cloud. The extraction method includes: acquiring multiple sampling points in a target point cloud; for each sampling point, determining the nearest neighbor point and the centroid of the nearest neighbor point; for each centroid corresponding to a sampling point, determining a rotation matrix corresponding to the sampling point based on the position coordinates of the centroid; the rotation matrix is ​​used to fix the rotation direction of the centroid; and determining the rotation-invariant features of the target point cloud based on the rotation matrices corresponding to the multiple sampling points in the target point cloud. By determining the rotation-invariant 3D features of the target point cloud through the rotation matrices corresponding to each sampling point, the problem of feature changes caused by rotation to the target point cloud is avoided, improving the success rate and accuracy of retrieving rotational features of 3D point clouds.

Description

Technical Field

[0001] This application relates to the field of point cloud processing technology, and in particular to a method, apparatus and storage medium for extracting rotation-invariant features of three-dimensional point clouds. Background Technology

[0002] Feature extraction from point clouds is a fundamental step in 3D vision tasks. In most application scenarios, the pose of objects is uncertain, making it crucial to ensure the rotation invariance of features. However, existing scene recognition and retrieval methods based on 3D point clouds neglect the rotation problem of 3D point clouds in real-world applications. For example, 3D point cloud retrieval networks based on the PointNet architecture only extract features from the initial point cloud, losing their original retrieval capabilities after rotation. However, for real-world scene recognition tasks, it's difficult to guarantee that the dataset and the collected data are in the same direction, thus rendering large-scale 3D point cloud scene recognition work inadequate due to a lack of consideration for rotation invariance. Therefore, how to achieve feature extraction from rotated 3D point clouds and improve their retrieval capabilities has become a significant technical challenge. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, apparatus and storage medium for extracting rotation-invariant features of three-dimensional point clouds. By determining the rotation matrix through the centroid point corresponding to each sampling point, the rotation direction of the centroid point is fixed. The rotation matrix is ​​used to rotate the nearest neighbor points of the corresponding sampling points, thereby determining the rotation-invariant three-dimensional features of the target point cloud. This avoids the feature changes caused by rotation to the target point cloud and improves the success rate and accuracy of searching for rotation of three-dimensional point clouds.

[0004] This application provides a method for extracting rotation-invariant features of a three-dimensional point cloud, characterized in that the extraction method includes:

[0005] Obtain multiple sampling points from the target point cloud;

[0006] For each sampling point, determine the nearest neighbor points of the sampling point.

[0007] For each sampling point, determine the centroid of the nearest neighbor point;

[0008] For each sampling point, a rotation matrix corresponding to the sampling point is determined based on the position coordinates of the centroid; wherein, the rotation matrix is ​​used to fix the rotation direction of the centroid.

[0009] Based on the rotation matrices corresponding to multiple sampling points in the target point cloud, the rotation invariance characteristics of the target point cloud are determined.

[0010] In one possible implementation, acquiring multiple sampling points in the target point cloud includes:

[0011] Based on the farthest distance point sampling algorithm, the multiple sampling points are obtained by uniformly sampling from the target point cloud;

[0012] Alternatively, based on the bounding box grid division algorithm, the point closest to the center point of each grid in the target point cloud is found as the sampling point to obtain the multiple sampling points;

[0013] Alternatively, the target point cloud can be arranged into a one-dimensional spatial order, and the multiple sampling points can be obtained by sampling according to the one-dimensional spatial order.

[0014] In one possible implementation, determining the nearest neighbors of each sampling point includes:

[0015] For each sampling point, in the target point cloud, a preset number of points that are closest to the sampling point are determined as the nearest neighbors of the sampling point;

[0016] In one possible implementation, determining the centroid of the nearest neighbor point for each sampling point includes:

[0017] Geometric calculations are performed on the nearest neighbor points to determine their centroids.

[0018] In one possible implementation, determining the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid includes:

[0019] Based on the position information of the centroid on the first coordinate axis and the position information of the corresponding sampling point on the first coordinate axis, a first distance is determined;

[0020] Based on the position information of the centroid on the second coordinate axis and the position information of the corresponding sampling point on the second coordinate axis, the second distance is determined;

[0021] Based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined.

[0022] In one possible implementation, determining the rotation invariance characteristics of the target point cloud based on the rotation matrices corresponding to multiple sampling points in the target point cloud includes:

[0023] Based on the target point cloud, extract the features corresponding to each sampling point;

[0024] Based on the features and rotation matrix corresponding to each sampling point, the rotation invariance features of the target point cloud are determined.

[0025] This application embodiment also provides a device for extracting rotation-invariant features of three-dimensional point clouds, the extraction device comprising:

[0026] The sampling point acquisition module is used to acquire multiple sampling points in the target point cloud;

[0027] The nearest neighbor determination module is used to determine the nearest neighbors of each sampling point.

[0028] The centroid determination module is used to determine the centroid of the nearest neighbor points for each sampling point;

[0029] A rotation matrix determination module is used to determine the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid point; wherein, the rotation matrix is ​​used to fix the rotation direction of the centroid point;

[0030] The feature determination module is used to determine the rotation invariance features of the target point cloud based on the rotation matrix corresponding to multiple sampling points in the target point cloud.

[0031] In one possible implementation, when determining the rotation matrix corresponding to the sampling point, the rotation matrix determination module is specifically used for:

[0032] Based on the position information of the centroid on the first coordinate axis and the position information of the corresponding sampling point on the first coordinate axis, a first distance is determined;

[0033] Based on the position information of the centroid on the second coordinate axis and the position information of the corresponding sampling point on the second coordinate axis, the second distance is determined;

[0034] Based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined.

[0035] This application embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the three-dimensional point cloud rotation invariance feature extraction method as described above are performed.

[0036] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the method for extracting rotation-invariant features of three-dimensional point clouds as described above.

[0037] This application provides a method, apparatus, and storage medium for extracting rotation-invariant features of 3D point clouds. The extraction method includes: acquiring multiple sampling points in a target point cloud; determining the nearest neighbor points for each sampling point; determining the centroid of each nearest neighbor point for each sampling point; determining a rotation matrix corresponding to each sampling point based on the position coordinates of the centroid; the rotation matrix is ​​used to fix the rotation direction of the centroid; and determining the rotation-invariant 3D features of the target point cloud based on the rotation matrices corresponding to the multiple sampling points in the target point cloud. By using the rotation matrices corresponding to each sampling point and rotating the nearest neighbor points of the corresponding sampling points, the rotation-invariant 3D features of the target point cloud are determined, avoiding the feature changes caused by rotation to the target point cloud, and improving the success rate and accuracy of searching for rotational conditions of 3D point clouds.

[0038] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a method for extracting rotation-invariant features of a three-dimensional point cloud, provided in an embodiment of this application;

[0041] Figure 2 A schematic diagram showing the rotation of the centroid and nearest neighbor points corresponding to a sampling point provided in an embodiment of this application;

[0042] Figure 3 A schematic diagram of the structure of the device for extracting rotation-invariant features of three-dimensional point clouds provided in the embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0045] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0046] To enable those skilled in the art to use the content of this application and, in conjunction with the specific application scenario of "extracting rotation invariant features of 3D point clouds", the following implementation method is provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0047] First, the applicable application scenarios of this application will be introduced. This application can be applied to the field of point cloud processing technology.

[0048] Feature extraction from point clouds is a fundamental step in 3D vision tasks. In most application scenarios, the pose of objects is uncertain, making it crucial to ensure the rotation invariance of features. However, existing scene recognition and retrieval methods based on 3D point clouds neglect the rotation problem of 3D point clouds in real-world applications. For example, 3D point cloud retrieval networks based on the PointNet architecture only extract features from the initial point cloud, losing their original retrieval capabilities when the point cloud rotates. However, for real-world scene recognition tasks, datasets acquired by different devices at different times are unlikely to be aligned in the same direction. Therefore, large-scale 3D point cloud scene recognition work that does not consider rotation invariance is flawed. Thus, how to achieve feature extraction from 3D point clouds under rotation and improve the retrieval capability of 3D point clouds under rotation has become a significant technical challenge.

[0049] Based on this, the embodiments of this application provide a method for extracting rotation-invariant features of three-dimensional point clouds. By using the rotation matrix corresponding to each sampling point, the nearest points of the corresponding sampling points are rotated using the rotation matrix, thereby determining the rotation-invariant three-dimensional features of the target point cloud. This avoids the feature changes caused by rotation to the target point cloud and improves the success rate and accuracy of searching for rotation of three-dimensional point clouds.

[0050] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for extracting rotation-invariant features of a 3D point cloud, as provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the extraction method includes:

[0051] S101: Obtain multiple sampling points in the target point cloud.

[0052] In this step, multiple sampling points are obtained from the target point cloud dataset.

[0053] Here, the target point cloud is downsampled to obtain multiple sampling points, thereby obtaining point cloud feature information with reduced dimensionality.

[0054] In one possible implementation, acquiring multiple sampling points in the target point cloud includes:

[0055] A: Based on the farthest distance point sampling algorithm, the multiple sampling points are obtained from the target point cloud.

[0056] Here, N points are sampled from the target point cloud using the farthest distance point sampling method.

[0057] Here, N is a positive integer.

[0058] For example, 24 sampling points are selected from 100 target point clouds. The first point in the target point cloud is taken as the first sampling point; the distances between the first sampling point and the other 99 points in the target point cloud are calculated, and the point with the largest distance is selected as the second sampling point; the sum of the distances between the first sampling point and the other 98 points in the target point cloud and the second sampling point is calculated as the distances from the other 98 points to the sampling point, and the point with the largest distance is selected as the third sampling point; and so on, until the 24th sampling point is selected.

[0059] B: Based on the bounding box grid division algorithm, the point closest to the center point of each grid in the target point cloud is found as the sampling point to obtain the multiple sampling points.

[0060] Here, bounding boxes are used to divide the grid and sample N points from the target point cloud as sampling points.

[0061] Here, N is a positive integer.

[0062] For example, 24 sampling points are selected from 100 target point clouds. For a 3D point cloud, the bounding box of the target point cloud is divided into 2x3x4 or 2x2x6 grids in each of the three coordinate directions, for a total of 24 grids. From the 100 points in the target point cloud, the point closest to the center of the first grid is selected as the first sampling point; from the other 99 points in the target point cloud, the point closest to the center of the second grid is selected as the second sampling point; from the other 98 points in the target point cloud, the point closest to the center of the third grid is selected as the third sampling point; and so on, until the 24th sampling point is selected.

[0063] C: Arrange the target point cloud into a one-dimensional spatial order, and obtain the multiple sampling points based on the one-dimensional spatial order.

[0064] Here, one-dimensional spatial order sampling is used to sample N points from the target point cloud as sampling points.

[0065] Here, N is a positive integer.

[0066] For example, 24 sampling points are selected from 100 target point clouds. For 3D point clouds, the target point clouds are arranged into a one-dimensional spatial order. This can be done according to the input order or according to a set rule, such as sorting by Morton code or Hilbert code. Using the one-dimensional spatial order sampling, the first point of the one-dimensional spatial order is taken as the first sampling point; the fifth point of the one-dimensional spatial order is taken as the second sampling point; the ninth point of the one-dimensional spatial order is taken as the third sampling point; and so on, until the 24th sampling point is selected.

[0067] S102: For each sampling point, determine the nearest neighbor point of the sampling point.

[0068] In this step, for each sampling point, one or more nearest neighbors of each sampling point are determined.

[0069] Here, this step obtains the nearest neighbor information of the sampled point obtained after sampling. The nearest neighbor is the surrounding point of the sampled point in the target point cloud, excluding the sampled point itself.

[0070] Each sampling point has at least one nearest neighbor.

[0071] In one possible implementation, determining the nearest neighbors of each sampling point includes:

[0072] For each sampling point, in the target point cloud, the distance between the sampling point and other points besides the sampling point is calculated, and the point with the smallest distance among a preset number of points is determined as the nearest neighbor of the sampling point.

[0073] Here, the distance can be a weighted sum of the P-th powers of the differences between the two points in the N coordinate components; or the distance can be the maximum value of the differences between the two points in the N coordinate components.

[0074] Here, the step of obtaining the nearest neighbor information of the sampled points can be achieved using the K-nearest neighbor algorithm.

[0075] In one possible implementation, for each sampling point, calculating the distances between the sampling point and other points in the target point cloud (excluding the sampling point itself), and determining the points with the smallest distances among a preset number of points as the nearest neighbors of the sampling point, includes:

[0076] a: For each sampling point, obtain other points in the target point cloud besides the sampling point, and determine the distance between each other point and the sampling point;

[0077] Here, the Euclidean distance formula can be used to determine the Euclidean distance from the sampling point to other points in the target point cloud besides the sampling point.

[0078] Here, in addition to using the Euclidean distance formula, other distance calculation formulas can also be used to determine the other distances between the target point cloud and the sampling point, excluding the sampling point.

[0079] b: Sort the multiple distances corresponding to the sampling points, and select a preset number of points corresponding to the shortest distance from the sorted multiple distances;

[0080] Here, multiple distances are sorted in ascending or descending order, and a preset number of points corresponding to the shortest distance are selected from the sorted distances.

[0081] Here, the preset number is at least 1.

[0082] c: The points corresponding to the shortest distance of the preset number are determined as the nearest neighbors of the sampling points.

[0083] Here, the points corresponding to the shortest distance of a preset number are determined as nearest neighbors.

[0084] Among them, the points corresponding to the shortest Euclidean distance of a preset number are determined as the nearest neighbors of the sampling points.

[0085] In one possible implementation, for each sampling point, calculating the distances between the sampling point and other points in the target point cloud (excluding the sampling point itself), and determining the points with the smallest distances among a preset number of points as the nearest neighbors of the sampling point, includes:

[0086] (1): For each sampling point, obtain a preset number of other points in the target point cloud other than the sampling point, determine the distance between each of the preset number of other points and the sampling point, and sort them according to the distance;

[0087] Here, the Euclidean distance formula can be used to determine the Euclidean distance from the sampling point to a predetermined number of other points in the target point cloud, excluding the sampling point.

[0088] Here, in addition to using the Euclidean distance formula, other distance calculation formulas can also be used to determine the other distances between the target point cloud, a preset number of other points besides the sampling point, and the sampling point.

[0089] Here, the preset number is at least 1.

[0090] Here, sorting by distance can be either ascending or descending.

[0091] (2): Continue to acquire other points in the target point cloud besides the sampling point, calculate the distance between the other point and the sampling point, compare the distance with the largest sorted distance, if the distance is less than the largest sorted distance, remove the point with the largest sorted distance and add the other point, and then update the sorting according to the distance; and so on, until all other points in the target point cloud besides the sampling point are acquired.

[0092] Here, for the points in the target point cloud that are acquired other than the sampling point, the distance between the other point and the sampling point is calculated. First, this distance is compared with the largest sorted distance. If the distance is less than the largest sorted distance, the point with the largest sorted distance is removed, and the other point is added. Then, the sorting is updated according to distance. If the distance is greater than the largest sorted distance, no sorting update is needed. Then, this process is repeated for all other points in the target point cloud except the sampling point, performing the distance calculation, distance comparison, and sorting update as described above, and so on, until all other points in the target point cloud except the sampling point have been acquired.

[0093] (3): The preset number of points in the sorting are determined as the nearest neighbors of the sampling points.

[0094] Here, the preset number of points in the sorting are the nearest neighbors that are closest to the sampling point.

[0095] S103: For each sampling point, determine the centroid of the nearest neighbor point.

[0096] In this step, for each sampling point, the centroid of the nearest neighbor point of each sampling point is determined.

[0097] Here, the centroid is the center of its nearest neighbor.

[0098] Using nearest neighbor information, the centroid of the nearest neighbors of the sampled point is calculated. The centroid reflects the distribution of the nearest neighbors of the sampled point.

[0099] Here, for each sampling point's nearest neighbor, the centroid of the nearest neighbor is determined.

[0100] In one possible implementation, determining the centroid of the nearest neighbor point for each sampling point includes:

[0101] Geometric calculations are performed on the nearest neighbor points to determine the centroid among them.

[0102] Here, when there is only one nearest neighbor, that nearest neighbor is the centroid.

[0103] Here, when there are multiple nearest neighbor points, geometric calculations are performed on the multiple nearest neighbor points to determine the centroid of the multiple nearest neighbor points.

[0104] For example, if the two-dimensional coordinates of the sampling point are (0,0), the three-dimensional coordinates of the first nearest neighbor are (1,0), and the three-dimensional coordinates of the second nearest neighbor are (1,1), the centroid of the nearest neighbor is determined by calculating the geometric mean of the geometric coordinates of multiple nearest neighbors. The two-dimensional position coordinates of the centroid of the nearest neighbor are then (1,0.5). If the three-dimensional coordinates of the sampling point are (0,0,0), the three-dimensional coordinates of the first nearest neighbor are (1,0,0), the three-dimensional coordinates of the second nearest neighbor are (0,1,1), and the three-dimensional coordinates of the third nearest neighbor are (1,1,1), the centroid of the nearest neighbor is determined by calculating the geometric mean of the geometric coordinates of multiple nearest neighbors. The three-dimensional position coordinates of the centroid of the nearest neighbor are then (2 / 3,2 / 3,2 / 3). The method for determining the centroid of other numbers of nearest neighbors is consistent with the above description and will not be repeated here.

[0105] S104: For each sampling point, based on the position coordinates of the centroid, determine the rotation matrix corresponding to the sampling point; wherein, the rotation matrix is ​​used to fix the rotation direction of the centroid.

[0106] In this step, for each sampling point, the rotation matrix corresponding to the centroid is determined based on the position coordinates of the centroid.

[0107] The rotation matrix is ​​used to fix the rotation direction of the centroid, thereby rotating the nearest neighbor points of the corresponding sampling point.

[0108] Here, the purpose of using the rotation matrix is ​​to fix the rotation direction of the centroid and thus regulate the rotation direction of the nearest neighbor points, so that the 3D point cloud can be adjusted to a consistent direction even if it rotates, thereby achieving rotation invariance.

[0109] In one possible implementation, determining the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid includes:

[0110] S1041: Based on the position information of the centroid on the first coordinate axis to the position information of the corresponding sampling point on the first coordinate axis, determine the first distance.

[0111] Here, the first distance is determined based on the difference between the coordinate position information of the centroid on the first coordinate axis and the coordinate position information of the corresponding sampling point on the first coordinate axis.

[0112] S1042: Determine the second distance based on the position information of the centroid on the second coordinate axis and the position information of the corresponding sampling point on the second coordinate axis.

[0113] Here, the second distance is determined based on the difference between the coordinate position information of the centroid on the second coordinate axis and the coordinate position information of the corresponding sampling point on the second coordinate axis.

[0114] S1043: Based on the first distance and the second distance, determine the rotation matrix corresponding to the sampling point.

[0115] Here, the rotation matrix corresponding to the sampling point is determined based on the first distance and the second distance.

[0116] In one possible implementation, determining the rotation matrix corresponding to the sampling point based on the first distance and the second distance includes:

[0117] S10431: Sum the square of the first distance and the square of the second distance to determine the first value.

[0118] Here, the square of the first distance is added to the square of the second distance to determine the first value.

[0119] S10432: Take the square root of the first value to determine the second value.

[0120] Here, the second value is determined by taking the square root of the first value.

[0121] S10433: Divide the first distance by the second value to determine the third value.

[0122] Here, the first distance is divided by the second value to determine the third value.

[0123] S10434: Divide the second distance by the second value to determine the fourth value.

[0124] Here, the second distance is divided by the second value to determine the fourth value.

[0125] S10435: Determine the rotation matrix corresponding to the sampling point based on the third value and the fourth value.

[0126] Here, the rotation matrix corresponding to the sampling point is determined based on the third and fourth values.

[0127] Here, the rotation matrix is ​​a rotation matrix about the third coordinate axis, which can be represented as:

[0128]

[0129] In one possible embodiment, where the rotation matrix is ​​a rotation matrix around the Z-axis, determining the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid of the centroid includes:

[0130] Based on the position information of the centroid on the X-axis to the position information of the corresponding sampling point on the X-axis, a first distance is determined; based on the position information of the centroid on the Y-axis to the position information of the corresponding sampling point on the Y-axis, a second distance is determined; based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined.

[0131] Here, the rotation matrix is ​​a rotation matrix around the Z-axis, and the rotation matrix can be specifically represented as:

[0132]

[0133] Among them, M F Let be a rotation matrix.

[0134]

[0135]

[0136] in, The first distance, m1 is the second distance, m2 is the third value, and m3 is the fourth value.

[0137] In one possible embodiment, where the rotation matrix is ​​a rotation matrix around the Y-axis, determining the rotation matrix of the centroid point based on the position coordinates of the centroid point corresponding to each sampling point further includes:

[0138] Based on the position information of the centroid on the Y-axis to the position information of the corresponding sampling point on the Y-axis, a first distance is determined; based on the position information of the centroid on the Z-axis to the position information of the corresponding sampling point on the Z-axis, a second distance is determined; based on the first distance and the second distance, the sine and cosine values ​​of the centroid are determined; based on the sine and cosine values, the rotation matrix of the centroid is determined.

[0139] Here, the rotation matrix is ​​a rotation matrix around the X-axis, and the rotation matrix can be specifically represented as:

[0140]

[0141] Among them, M F Let be a rotation matrix.

[0142]

[0143]

[0144] in, The first distance, m1 is the second distance, m2 is the third value, and m2 is the fourth value.

[0145] In one possible embodiment, where the rotation matrix is ​​a rotation matrix around the X-axis, determining the rotation matrix of the centroid point based on the position coordinates of the centroid point corresponding to each sampling point further includes:

[0146] Based on the position information of the centroid on the X-axis to the position information of the corresponding sampling point on the X-axis, a first distance is determined; based on the position information of the centroid on the Z-axis to the position information of the corresponding sampling point on the Z-axis, a second distance is determined; based on the first distance and the second distance, the sine and cosine values ​​of the centroid are determined; based on the sine and cosine values, the rotation matrix of the centroid is determined.

[0147] Here, the rotation matrix is ​​a rotation matrix around the X-axis, and the rotation matrix can be specifically represented as:

[0148]

[0149] Among them, M F Let be a rotation matrix.

[0150]

[0151]

[0152] in, The first distance, m1 is the second distance, m2 is the third value, and m2 is the fourth value.

[0153] S105: Determine the rotation invariance characteristics of the target point cloud based on the rotation matrix corresponding to multiple sampling points in the target point cloud.

[0154] In this step, the rotation-invariant three-dimensional features of the target point cloud are determined based on the rotation matrices corresponding to multiple sampling points in the target point cloud.

[0155] In one possible implementation, determining the rotation invariance characteristics of the target point cloud based on the rotation matrices corresponding to multiple sampling points in the target point cloud includes:

[0156] S1051: Based on the target point cloud, extract the features corresponding to each sampling point.

[0157] Here, the spatial relationship between each sampling point and its nearest neighbors can be used as its feature;

[0158] In addition to utilizing spatial relationships, other feature extraction methods can also be used to determine the features of each sampling point.

[0159] S1052: Based on the features and rotation matrix corresponding to each sampling point, determine the rotation invariance features of the target point cloud.

[0160] Here, for each sampling point, the corresponding feature is multiplied by the corresponding rotation matrix to obtain a new rotated feature, which serves as the rotation-invariant feature of the target point cloud. The new feature has rotation-invariant characteristics.

[0161] For example, if the 3D coordinates of sampling point Pi are (1,2,3), and the coordinates of its three nearest neighbors Pi0, Pi1, and Pi2 are (2,2,3), (1,3,4), and (2,3,4) respectively, and the spatial relationship between the sampling point and its nearest neighbors is used as its feature, such as the feature extraction method Fi = Pij - Pi, where j = 0, 1, 2, the corresponding feature information Fi is ((1,0,0), (0,1,1), (1,1,1)), and the 3D position coordinates of the centroid point Pic obtained from its nearest neighbors are (5 / 3, 8 / 3, 11 / 3). The 3D position coordinates of the centroid point Pic relative to the sampling point Pi are Pici = Pic - Pi = (2 / 3, 2 / 3, 2 / 3). Its rotation matrix is ​​a rotation matrix around the Z-axis with the sampling point as the origin. The first distance Second distance Third value Fourth value Rotation matrix After rotating using the rotation matrix corresponding to the sampling point, the coordinates of the centroid relative to the sampling point become... The corresponding rotation-invariant three-dimensional feature is

[0162] Here, the rotation that the target point cloud may undergo is processed so that the features extracted by the subsequent network are not affected by the rotation of the target point cloud.

[0163] Here, after determining the rotation invariance features of the target point cloud, point cloud retrieval can be performed using these features to improve retrieval accuracy.

[0164] Here, the nearest neighbor points of the corresponding sampling points in the target point cloud are rotated according to the rotation matrix corresponding to multiple sampling points. This ensures that even if the nearest neighbor points of the sampling points are rotated, the distribution direction of the nearest neighbor points relative to the sampling points can be kept consistent through the rotation matrix. This avoids the problem of point cloud feature changes caused by rotation to a certain extent, keeps the point cloud features unchanged under rotation, and thus obtains the rotation invariance feature of the target point cloud.

[0165] Here, a rotation matrix obtained by fixing the rotation direction of the centroid is used to make the nearest neighbor points of the sampling point rotate according to the fixed direction of the corresponding centroid, thereby avoiding the problem of point cloud feature changes caused by rotation to a certain extent.

[0166] Existing methods do not consider the rotation of 3D point cloud models acquired in large-scale scene recognition tasks in practical applications. The present invention samples the target point cloud to obtain multiple sampling points. For each sampling point, its nearest neighbors are determined. The position information of the centroid can be obtained from the information of the nearest neighbors. A rotation matrix is ​​then determined based on the position information of the centroid, and the corresponding nearest neighbors of the sampling point are rotated using the rotation matrix, thereby avoiding the problem of point cloud feature alteration caused by rotation to a certain extent. Based on geometric characteristics, this method can be flexibly applied to different large-scale 3D point cloud retrieval algorithms. It proposes an efficient solution to the problems faced in real-world application scenarios, providing a fundamental guarantee for the industrial application of related algorithms.

[0167] Figure 2 This is a schematic diagram showing the rotation of the centroid and nearest neighbor points corresponding to a sampling point provided in an embodiment of this application. For example... Figure 2 As shown, the sampling point Pi obtained after sampling has Pi0 and Pi1 as its nearest neighbors, and Pic is the centroid obtained from its nearest neighbors Pi0 and Pi1. Pic is rotated around Pi in a fixed direction to the position Pic2. Because it is only a rotation, the distance from Pic2 to Pi is equal to the distance between Pic and Pi. Even if the point cloud rotates, this process can always rotate Pic to Pic2 in a fixed direction, so that the 3D point cloud can be adjusted to a consistent direction even if it rotates, thus achieving rotation invariance.

[0168] This application provides a method for extracting rotation-invariant features of 3D point clouds. The method includes: acquiring multiple sampling points in a target point cloud; determining the nearest neighbor points for each sampling point; determining the centroid of each nearest neighbor point for each sampling point; determining a rotation matrix corresponding to each sampling point based on the position coordinates of the centroid; the rotation matrix is ​​used to fix the rotation direction of the centroid; and determining the rotation-invariant 3D features of the target point cloud based on the rotation matrices corresponding to the multiple sampling points in the target point cloud. By using the rotation matrices corresponding to each sampling point and rotating the nearest neighbor points of the corresponding sampling points, the rotation-invariant 3D features of the target point cloud are determined, avoiding the feature changes caused by rotation to the target point cloud and improving the success rate and accuracy of searching for rotation in 3D point clouds.

[0169] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of the device for extracting rotation-invariant features of a three-dimensional point cloud provided in an embodiment of this application. Figure 3 As shown, the extraction device 300 for the rotation invariance features of the three-dimensional point cloud includes:

[0170] The sampling point acquisition module 310 is used to acquire multiple sampling points in the target point cloud;

[0171] The nearest neighbor determination module 320 is used to determine the nearest neighbors of each sampling point.

[0172] The centroid determination module 330 is used to determine the centroid of the nearest neighbor point for each sampling point;

[0173] The rotation matrix determination module 340 is used to determine the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid point; wherein the rotation matrix is ​​used to fix the rotation direction of the centroid point.

[0174] The feature determination module 350 is used to determine the rotation invariance features of the target point cloud based on the rotation matrix corresponding to multiple sampling points in the target point cloud.

[0175] In one possible implementation, when the sampling point acquisition module 310 is used to acquire multiple sampling points in the target point cloud, the sampling point acquisition module 310 is specifically used for:

[0176] Based on the farthest distance point sampling algorithm, the multiple sampling points are obtained by sampling from the target point cloud;

[0177] Alternatively, based on the bounding box grid division algorithm, the point closest to the center point of each grid in the target point cloud is found as the sampling point to obtain the multiple sampling points;

[0178] Alternatively, the target point cloud can be arranged into a one-dimensional spatial order, and the multiple sampling points can be obtained by sampling according to the one-dimensional spatial order.

[0179] In one possible implementation, when determining the nearest neighbor point determination module 320 for each sampling point, the nearest neighbor point determination module 320 is specifically used for:

[0180] For each sampling point, in the target point cloud, a preset number of points that are closest to the sampling point are determined as the nearest neighbors of the sampling point.

[0181] In one possible implementation, when the centroid determination module 330 is used to determine the centroid of the nearest neighbor point for each sampling point, the centroid determination module 330 is specifically used for:

[0182] Geometric calculations are performed on the nearest neighbor points to determine their centroids.

[0183] In one possible implementation, when the rotation matrix determination module 340 determines the rotation matrix corresponding to each sampling point based on the position coordinates of the centroid point, the rotation matrix determination module 340 is specifically used for:

[0184] Based on the position information of the centroid on the first coordinate axis and the position information of the corresponding sampling point on the first coordinate axis, a first distance is determined;

[0185] Based on the position information of the centroid on the second coordinate axis and the position information of the corresponding sampling point on the second coordinate axis, the second distance is determined;

[0186] Based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined.

[0187] In one possible implementation, when the feature determination module 350 determines the rotation invariance features of the target point cloud based on the rotation matrices corresponding to multiple sampling points in the target point cloud, the feature determination module 350 is specifically used for:

[0188] Based on the target point cloud, extract the features corresponding to each sampling point;

[0189] Based on the features and rotation matrix corresponding to each sampling point, the rotation invariance features of the target point cloud are determined.

[0190] This application provides an apparatus for extracting rotation-invariant features of a 3D point cloud. The apparatus includes: a sampling point acquisition module for acquiring multiple sampling points in a target point cloud; a nearest neighbor determination module for determining the nearest neighbors of each sampling point; a centroid determination module for determining the centroid of each nearest neighbor for each sampling point; a rotation matrix determination module for determining a rotation matrix corresponding to each sampling point based on the position coordinates of the centroid; wherein the rotation matrix is ​​used to fix the rotation direction of the centroid; and a feature determination module for determining the rotation-invariant features of the target point cloud based on the rotation matrices corresponding to the multiple sampling points in the target point cloud. By using the rotation matrices corresponding to each sampling point, the features of the corresponding sampling points are rotated using the rotation matrices to determine the rotation-invariant 3D features of the target point cloud. This avoids the feature changes caused by rotation to the target point cloud, improving the success rate and accuracy of retrieving rotational features of 3D point clouds.

[0191] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0192] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the method for extracting rotation-invariant features of 3D point clouds in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0193] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the method for extracting rotation-invariant features of 3D point clouds in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0194] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0195] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0198] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for extracting a three-dimensional point cloud rotation invariance feature, characterized in that, The extraction method comprises: obtaining a plurality of sampling points in a target point cloud; for each sampling point, determining a nearest neighbor point of the sampling point; for each sampling point, determining a centroid point of the nearest neighbor point; for each sampling point, determining a rotation matrix corresponding to the sampling point based on the position coordinates of the centroid point; wherein the rotation matrix is used to fix the rotation direction of the centroid point; based on the rotation matrix corresponding to the plurality of sampling points in the target point cloud, determining the rotation invariance feature of the target point cloud; for the rotation matrix being a rotation matrix around the Z axis, the determination of the rotation matrix corresponding to the centroid point of each sampling point based on the position coordinates of the centroid point comprises: based on the position information of the centroid point on the X axis to the corresponding position information of the sampling point on the X axis, a first distance is determined; based on the position information of the centroid point on the Y axis to the corresponding position information of the sampling point on the Y axis, a second distance is determined; based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined; wherein, for the rotation matrix being a rotation matrix around the Z axis, the rotation matrix is represented as: ； ； wherein is a rotation matrix, is a first distance, is a second distance, is a third value, is a fourth value.

2. The extraction method according to claim 1, characterized in that, the obtaining of the plurality of sampling points in the target point cloud comprises: based on the farthest distance point sampling algorithm, sampling from the target point cloud to obtain the plurality of sampling points; or, based on the bounding box division grid algorithm, finding the nearest point to each grid center point in the target point cloud as a sampling point to obtain the plurality of sampling points; or, arranging the target point cloud into a one-dimensional space sequence, sampling according to the one-dimensional space sequence to obtain the plurality of sampling points.

3. The extraction method of claim 1, wherein, the determination of the nearest neighbor point of each sampling point comprises: for each sampling point, in the target point cloud, a preset number of points closest to the sampling point are determined as the nearest neighbor points of the sampling point.

4. The extraction method of claim 1, wherein, the determination of the centroid point of the nearest neighbor point of each sampling point comprises: geometric calculation is performed on the nearest neighbor points to determine the centroid point of the nearest neighbor points.

5. The extraction method of claim 1, wherein, the determination of the rotation matrix corresponding to the sampling point of each sampling point based on the position coordinates of the centroid point comprises: based on the position information of the centroid point on the first coordinate axis to the corresponding position information of the sampling point on the first coordinate axis, a first distance is determined; based on the position information of the centroid point on the second coordinate axis to the corresponding position information of the sampling point on the second coordinate axis, a second distance is determined; based on the first distance and the second distance, the rotation matrix corresponding to the sampling point is determined.

6. The extraction method of claim 1, wherein, the determination of the rotation invariance feature of the target point cloud based on the rotation matrix corresponding to the plurality of sampling points in the target point cloud comprises: based on the target point cloud, extracting the feature corresponding to each sampling point; based on the feature corresponding to each sampling point and the rotation matrix, determining the rotation invariance feature of the target point cloud.

7. A device for extracting rotation-invariant features of three-dimensional point clouds, characterized in that, the extraction device comprises: a sampling point acquisition module for obtaining a plurality of sampling points in a target point cloud; a neighbor point determination module configured to determine, for each sampling point, a neighbor point of the sampling point; a centroid point determination module configured to determine, for each sampling point, a centroid point of the neighbor point; a rotation matrix determination module configured to determine, for each sampling point, a rotation matrix corresponding to the sampling point based on a position coordinate of the centroid point, wherein the rotation matrix is used to fix a rotation direction of the centroid point; a feature determination module configured to determine a rotation invariance feature of the target point cloud based on the rotation matrix corresponding to each sampling point in the target point cloud; the rotation matrix determination module is configured to, for a rotation matrix being a rotation matrix around a Z axis, determine, for each sampling point, a rotation matrix corresponding to the sampling point based on a position coordinate of the centroid point, including: determining a first distance based on position information of the centroid point on an X axis to position information of the corresponding sampling point on the X axis; determining a second distance based on position information of the centroid point on a Y axis to position information of the corresponding sampling point on the Y axis; determining the rotation matrix corresponding to the sampling point based on the first distance and the second distance; wherein, for a rotation matrix being a rotation matrix around a Z axis, the rotation matrix is represented as: ； ； wherein is a rotation matrix, is a first distance, is a second distance, is a third value, is a fourth value.

8. The extraction device of claim 7, wherein, when determining the rotation matrix corresponding to the sampling point, the rotation matrix determination module is specifically configured to: determine a first distance based on position information of the centroid point on a first coordinate axis to position information of the corresponding sampling point on the first coordinate axis; determine a second distance based on position information of the centroid point on a second coordinate axis to position information of the corresponding sampling point on the second coordinate axis; determine the rotation matrix corresponding to the sampling point based on the first distance and the second distance.

9. An electronic device, comprising: including: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the three-dimensional point cloud rotation invariance feature extraction method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, the computer readable storage medium stores a computer program, the computer program is executed by the processor to perform the steps of the three-dimensional point cloud rotation invariance feature extraction method according to any one of claims 1 to 6.

Images