A tensor-based point cloud enhancement method

Abstract

The present invention relates to a tensor-based point cloud enhancement method, which belongs to the field of multimedia information processing and comprises the following steps: S1: resampling a reference point cloud and passing the reference point cloud through a high-pass image filter to obtain high-frequency key points on its geometric structure; S2: using a k-nearest neighbor classification algorithm, constructing a local domain centered on the high-frequency key points in the reference point cloud and the distorted point cloud, denoted as #imgabs0# and #imgabs1# respectively; S3: expressing #imgabs2# and #imgabs3# in the form of tensors, denoted as T respectively. r and T d , for T r and T d Perform Tucker decomposition to obtain the first principal component C of the local domain of the reference point cloud and the distorted point cloud r and C d ; S4: Perform weighted aggregation on the first principal components of all local domains formed with the geometric skeleton key points as the center to obtain an enhanced point cloud model.

Description

Technical Field

[0001] The present invention belongs to the field of multimedia information processing and relates to a tensor-based point cloud enhancement method. Background Art

[0002] Point clouds, as an emerging multimedia format, have garnered widespread attention in fields such as intelligent transportation, autonomous driving, and virtual reality. A typical point cloud consists of millions of unstructured, non-uniformly distributed 3D points, each containing at least position and color information. Compared to two-dimensional images, point clouds have a higher dimensionality, which can lead to a certain degree of information redundancy and complicate subsequent data processing.

[0003] The applicant found that the existing point cloud processing process directly evaluates the point cloud, ignoring the high dimensionality and high information redundancy of the point cloud, resulting in inefficiency in subsequent processing steps. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a tensor-based point cloud enhancement method. This method takes into account the high-dimensional features of the point cloud. First, the geometric key points are extracted by resampling the reference point cloud to form a geometric skeleton. Then, local domains are constructed in the reference point cloud and the distorted point cloud with these key points as the center, and then the local domains are represented as tensor forms. Tucker decomposition is performed on the local domains in the tensor form, and the first principal component is obtained as the most important dimensionality reduction local domain to enter the subsequent processing link. When applying this method, without the need to use special hardware such as GPU, the tensorization and Tucker decomposition of the point cloud can be achieved with high efficiency, resulting in a low-dimensional point cloud and achieving good results.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A tensor-based point cloud enhancement method comprises the following steps:

[0007] S1: Resample the reference point cloud and pass it through a high-pass image filter to obtain high-frequency key points on its geometric structure;

[0008] S2: Using the k-nearest neighbor classification algorithm, a local domain centered on high-frequency key points is constructed in the reference point cloud and the distorted point cloud, which are denoted as and

[0009] S3: and Expressed in the form of tensors, respectively denoted as T r and T d , for T r and T dPerform Tucker decomposition to obtain the first principal component C of the local domain of the reference point cloud and the distorted point cloud r and C d ;

[0010] S4: For all C r and C d Perform weighted aggregation to obtain an enhanced point cloud model.

[0011] Furthermore, step S1 includes the following steps:

[0012] S11: The reference point cloud consists of N points, each of which has K attributes, recorded as:

[0013]

[0014] in, Represents the 1st to Nth points, Represents a space of dimension N×K;

[0015] S12: Using geometric distance To calculate the local domain midpoint and The connection weight between them is defined as follows:

[0016]

[0017] Among them, σ 2 represents the variance between points, and τ represents the Euclidean distance threshold for clustering adjacent points into the same local domain.

[0018] S13: Use the diagonal matrix D to measure the edge density of each vertex, point Connection density The edge density D is defined as follows:

[0019]

[0020] S14: The high-pass filter through which the reference point cloud passes is expressed as:

[0021]

[0022] Where L is the filter length, h l is the lth filter coefficient; is a local domain shift operator, defined as:

[0023] A=D -1 W

[0024] S15: Based on h(A), select a Haar-like local domain filter h HH (A) Implement high-pass filtering, defined as follows:

[0025]

[0026] Among them λ1~λ N is the eigenvalue of A, V is the eigenvector of A;

[0027] S16: Every point that passes through the filter The frequency response is:

[0028]

[0029] in Indicates a point A set of neighboring points, Indicates a point and The characteristic covariance matrix of

[0030] Assume the filter sampling frequency is f s , the total number of points in the point cloud after sampling Resampled reference point cloud Each point in is a geometric key point.

[0031] Furthermore, in step S2, a Euclidean distance threshold γ is given, and the distance between the reference point cloud and the distorted point cloud is Geometric key points in The local domain is constructed using the k-nearest neighbor search algorithm for the center, which is defined as follows:

[0032]

[0033]

[0034] in represents the local domain centered on high-frequency key points constructed in the reference point cloud, represents a point in the reference point cloud, represents the resampled reference point cloud, represents the local domain centered on high-frequency key points constructed in the distorted point cloud, represents a point in the distorted point cloud, Represents the distorted point cloud after resampling;

[0035] The points in the local domain are arranged from small to large according to the distance from the geometric key points.

[0036] Further, step S3 will and Expressed in the form of a tensor, as follows:

[0037] The local domain is a part of the point cloud. The local domain contains β points, and each point is represented by in Yes The geometric information is expressed in coordinates; Yes The three-dimensional color attributes are represented by RGB values; Yes Normal value of

[0038] Use the Tensor Toolbox to and Convert to tensor form and T r and T d There are three dimensions, the first two dimensions represent the geometric color normal properties, and the third dimension is the spatial dimension.

[0039] Further, in step S3, r and T d Perform Tucker decomposition to obtain the first principal component of the local domain of the reference point cloud and the distorted point cloud and The following steps are involved:

[0040] S31: Tensor form T for local domain r and T d Perform Tucker decomposition and get:

[0041]

[0042]

[0043] in It's T r The core tensor of It's T d The core tensor of R, the content of the core tensor describes the level of interaction between different factor matrices; r1 ×R r2 ×R r3 express The spatial dimension, R d1 ×R d2 ×R d3 express spatial dimension; and is a factor matrix of geometric dimension, and is the factor matrix of the color dimension, and is the factor matrix of spatial dimensions;

[0044] S32: For tensor T r and Td , by multiplying the core tensor and the factor matrix along each module in the geometric color normal dimension to obtain the Tucker decomposition component of the spatial dimension, that is, the first principal component and The formula is as follows:

[0045]

[0046]

[0047] The beneficial effect of the present invention is that an enhanced point cloud model is proposed, which not only retains the main features of the point cloud but also takes into account the high-dimensional characteristics of the point cloud. Based on the theory of Tucker decomposition, the dimension of the point cloud is reduced.

[0048] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0050] Figure 1 A flow chart of the system used in the present invention;

[0051] Figure 2 Schematic diagram of point cloud resampling in step S1 of the method of the present invention;

[0052] Figure 3 This is a schematic diagram of clustering based on high-frequency key points in step S2 of the method of the present invention;

[0053] Figure 4 The Tucker decomposition components and their energy histograms in step S3 of the method of the present invention. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0055] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0056] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0057] like Figure 1 As shown, the present invention provides a tensor-based point cloud enhancement method, comprising the following steps:

[0058] S1: If Figure 2 As shown in Figure 3, the point cloud is resampled through a high-pass filter to obtain geometric key points.

[0059] The point cloud consists of N points, each with K attributes. This point cloud is denoted as

[0060]

[0061] Use geometric distance To calculate the local domain and The connection weight between them is defined as follows:

[0062]

[0063] Use the diagonal matrix D to measure the edge density of each vertex. Connection density The edge density D is defined as follows:

[0064]

[0065] The high-pass filter passed by the reference point cloud is expressed as

[0066] Where L is the filter length, h l is the lth filter coefficient. is a local domain shift operator, defined as:

[0067] A=D -1 W

[0068] Based on h(A), a Haar-like local domain filter h is selected HH (A) Implement high-pass filtering, defined as follows:

[0069]

[0070] where λ i is the eigenvalue of A, and V is the eigenvector of A.

[0071] Every point that passes through the filter The frequency response is

[0072]

[0073] Assume the filter sampling frequency is f s , the total number of points in the point cloud after sampling Resampled point cloud Each point in is a geometric key point.

[0074] Furthermore, in step S2, a Euclidean distance threshold γ is given, and the reference point cloud and the distorted point cloud are separated by Geometric key points in The k-nearest neighbor search algorithm is used to construct Figure 3 The local domain shown is defined as follows;

[0075]

[0076]

[0077] The points in the local domain are arranged from small to large according to the distance from the geometric key points.

[0078] Furthermore, the tensor expression steps of the local domain Z in step S3 are as follows:

[0079] The local domain Z is part of the point cloud. The local domain contains β points, each of which is represented by in is the geometric information of the point, expressed in coordinates; is the three-dimensional color attribute of the point, expressed as RGB value. Therefore, use the tensor toolbox to convert Z into tensor form T has three dimensions, the first two dimensions represent geometric color properties, and the third dimension is the spatial dimension.

[0080] To obtain the final first principal component and The specific solution process includes the following steps:

[0081] S31: Perform Tucker decomposition on the local domain Z and obtain:

[0082]

[0083] in is the core tensor whose content describes the level of interaction between different factor matrices. is a factor matrix of geometric dimension, is the factor matrix of the color dimension, is the factor matrix of spatial dimensions.

[0084] S32: For the tensor T, the Tucker decomposition component of the spatial dimension, that is, the first principal component C, can be obtained by multiplying the core tensor and the factor matrix along each module in the geometric color dimension. The formula is as follows:

[0085]

[0086] S4: According to the steps of S3, the first principal component C of the local domain of the reference point cloud and the distorted point cloud is obtained respectively. r and C d . Figure 4 The energy histogram distribution of different decomposition components is shown. Clearly, texture information and energy are primarily concentrated in the first principal component. Therefore, it is feasible to use the first principal component of the local domain obtained through dimensionality reduction to replace the high-dimensional local domain of the point cloud. The first principal component is the local point cloud after dimensionality reduction, which solves the information redundancy problem of high-dimensional point clouds and can be directly applied to subsequent point cloud processing.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A tensor-based point cloud enhancement method, characterized by: The following steps are involved: S1: Resample the reference point cloud and pass it through a high-pass image filter to obtain high-frequency key points on its geometric structure; S2: Using the k-nearest neighbor classification algorithm, a local domain centered on high-frequency key points is constructed in the reference point cloud and the distorted point cloud, which are denoted as and S3: and Expressed in the form of tensors, respectively denoted as T r and T d , for T r and T d Perform Tucker decomposition to obtain the first principal component C of the local domain of the reference point cloud and the distorted point cloud r and C d ; S4: To C r and C d Quality assessment is performed separately, and all local domains formed around the geometric skeleton key points are weightedly aggregated to obtain an enhanced point cloud model.

2. The tensor-based point cloud enhancement method according to claim 1, characterized in that: Step S1 includes the following steps: S11: The reference point cloud consists of N points, each of which has K attributes, recorded as: in, Represents the 1st to Nth points, Represents a space of dimension N×K; S12: Using geometric distance To calculate the local domain midpoint and The connection weight between them is defined as follows: Among them, σ 2 represents the variance between points, τ represents the Euclidean distance threshold for clustering adjacent points into the same local domain; S13: Use the diagonal matrix D to measure the edge density of each vertex, point Connection density The edge density D is defined as follows: S14: The high-pass filter through which the reference point cloud passes is expressed as: Where L is the filter length, h l is the lth filter coefficient; is a local domain shift operator, defined as: A=D -1 W S15: Based on h(A), select a Haar-like local domain filter h HH (A) Implement high-pass filtering, defined as follows: Among them λ1~λ N is the eigenvalue of A, V is the eigenvector of A; S16: Every point that passes through the filter The frequency response is: in Indicates a point A set of neighboring points, Indicates a point and The characteristic covariance matrix of Assume the filter sampling frequency is f s , the total number of points in the point cloud after sampling Resampled reference point cloud Each point in is a geometric key point.

3. The tensor-based point cloud enhancement method according to claim 2, characterized in that: In step S2, a Euclidean distance threshold γ is given, and the reference point cloud and the distorted point cloud are separated by Geometric key points in The local domain is constructed using the k-nearest neighbor search algorithm for the center, which is defined as follows: where Z r represents the local domain centered on high-frequency key points constructed in the reference point cloud, represents a point in the reference point cloud, represents the resampled reference point cloud, represents the local domain centered on high-frequency key points constructed in the distorted point cloud, represents a point in the distorted point cloud, Represents the distorted point cloud after resampling; The points in the local domain are arranged from small to large according to the distance from the geometric key points.

4. The tensor-based point cloud enhancement method according to claim 3, characterized in that: Step S3 and Expressed in the form of a tensor, as follows: The local domain is a part of the point cloud. The local domain contains β points, and each point is represented by in Yes The geometric information is expressed in coordinates; Yes The three-dimensional color attributes are represented by RGB values; Yes Normal value of Use the Tensor Toolbox to and Convert to tensor form and T r and T d There are three dimensions, the first two dimensions represent the geometric color normal properties, and the third dimension is the spatial dimension.

5. The tensor-based point cloud enhancement method according to claim 4, characterized in that: In step S3, the r and T d Perform Tucker decomposition to obtain the first principal component C of the local domain of the reference point cloud and the distorted point cloud r and C d , including the following steps: S31: Tensor form T for local domain r and T d Perform Tucker decomposition and get: in It's T r The core tensor of It's T d The core tensor of R, the content of the core tensor describes the level of interaction between different factor matrices; r1 ×R r2 ×R r3 express The spatial dimension, R d1 ×R d2 ×R d3 express spatial dimension; and is a factor matrix of geometric dimension, and is the factor matrix of the color dimension, and is the factor matrix of spatial dimensions; S32: For tensor T r and T d , by multiplying the core tensor and the factor matrix along each module in the geometric color normal dimension to obtain the Tucker decomposition component of the spatial dimension, that is, the first principal component and The formula is as follows:

Images