Point cloud coding method based on multilevel ball octree and graph-driven attention entropy model

Through multi-level ball octree and graph-driven attention entropy model, combined with graph convolution and attention mechanism, the problems of high computational complexity and poor resolution robustness in point cloud data compression are solved, and efficient and accurate Point cloud data processing and compression.

CN119006620AActive Publication Date: 2024-11-22SUN YAT SEN UNIV

Patent Information

Application Number
CN202411151533.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-22
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

The prior art has high computational complexity, poor resolution robustness in point cloud data compression, and fails to fully utilize the redundant information of point clouds, resulting in low compression efficiency and quality.

Method used

Using a point cloud encoding method based on multi-level ball octree and graph-driven attention entropy model, an octree structure is constructed through spherical coordinate representation, and a graph convolution and attention mechanism are used to learn context information to predict the placeholder probability of nodes. , and compressed by arithmetic encoding.

Benefits of technology

It significantly improves the encoding efficiency and feature extraction accuracy of point cloud data, reduces the computational complexity, enhances the adaptability and compression effect to high-resolution point cloud data, and reduces quantization errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006620A_ABST
    Figure CN119006620A_ABST
Patent Text Reader

Abstract

The invention provides a point cloud coding method based on a multilevel spherical octree and a graph-driven attention entropy model. The method comprises the following steps: coding point cloud data by using an octree entropy model based on multilevel spherical coordinates; constructing an adjacent matrix of the parent graph and the distance graph; a graph convolutional network module is used for assisting context information embedding by utilizing an adjacent matrix; constructing a grouping graph attention module and a cross attention module to learn the correlation between a parent node context and a peer node context; predicting the probability of a placeholder symbol of each octree node; compressing the placeholder sequence into a binary floating-point number sequence and converting the binary floating-point number sequence into a bit stream; and converting the bit stream into a placeholder symbol sequence of the octree, reconstructing the octree and restoring the point cloud. According to the invention, through the combination of the multilevel spherical coordinate octree structure, the graph convolution, the grouping graph attention module and the cross attention module, quantization errors are reduced, the compression efficiency is improved, the calculation complexity and the compression effect are balanced, and the adaptability to high-resolution point cloud data is significantly enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data coding, and in particular to a point cloud coding method based on a multi-level ball octree and a graph-driven attention entropy model. Background Art

[0002] Point cloud data has been rapidly developed and widely used in fields such as autonomous driving, robotics, and 3D mapping due to its high precision and resolution. However, the huge amount of such high-precision point cloud data poses a huge challenge to storage and transmission. Faced with such a huge data processing task, how to reduce the computational complexity and improve the compression rate has become a key technical issue. In addition, due to the disorder and sparseness of point cloud data, traditional image and video compression methods are difficult to directly apply to point cloud data compression.

[0003] Existing technologies mainly compress point clouds through voxel-based, image-based, and tree-based algorithms. Among them, voxel-based methods grid the point cloud data and then compress the grid data; by organizing the point cloud into voxels and then using 3D convolution to predict the occupancy of each voxel. However, these methods are less robust to the resolution of the point cloud and can only obtain a limited receptive field of neighborhood information in a fixed-size voxel. In addition, the introduction of voxel encoding brings a lot of computation, especially for higher-resolution point clouds.

[0004] Secondly, the image-based method compresses the point cloud data by projecting it onto a depth map and then compressing the depth map. Although this method has a fast compression speed, it may cause information loss when processing non-planar point cloud data.

[0005] Furthermore, tree-based methods include KD trees, prediction trees, quadtrees, and octrees, among which the octree-based method stands out due to its high compression efficiency, fast compression speed, and adaptability to point cloud sparsity, and is widely used in point cloud data compression. However, it mainly constructs an octree structure based on Cartesian coordinates, which ignores the azimuth invariance brought by LiDAR rotation scanning and the multi-circular features in the point cloud, resulting in the model failing to fully utilize the redundant information of the point cloud, while increasing additional computational overhead and reducing the ability to learn local features.

[0006] Therefore, there is an urgent need to provide a method for fully mining geometric features and finding fine-grained prediction differences between nodes while saving computing resources, so as to obtain a more compact and higher-quality point cloud compression method. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a point cloud encoding method based on a multi-level spherical octree and a graph-driven attention entropy model. The present invention can improve the encoding efficiency and feature extraction accuracy of point cloud data, thereby realizing efficient and accurate point cloud data processing.

[0008] The technical solution of the present invention is: a point cloud encoding method based on a multi-level ball octree and a graph-driven attention entropy model, comprising:

[0009] S1), using an octree entropy model based on multi-level spherical coordinates to encode point cloud data;

[0010] S2), constructing the adjacency matrix of the parent graph and the distance graph using the octree structure and the spherical coordinate representation of the point cloud;

[0011] S3), construct an entropy coding model based on graph-driven attention;

[0012] S4), using the graph convolutional network module to utilize the adjacency matrix to assist in embedding context information;

[0013] S5), using the grouped graph attention module and the cross attention module to learn the relevance of the parent node context and the sibling node context;

[0014] S6), predicting the probability of the placeholder symbol of each octree node based on the entropy coding model;

[0015] S7), using arithmetic coding to compress the placeholder symbol sequence of the octree node into a binary floating point number sequence;

[0016] S8), converting the binary floating point number sequence into a bit stream;

[0017] S9), using arithmetic decoding to convert the bit stream into a placeholder symbol sequence of the octree, reconstruct it into the octree and restore the point cloud data.

[0018] Preferably, in step S1), constructing a multi-level spherical coordinate octree specifically includes the following steps:

[0019] S11), introduce the quantization step size of the spherical coordinate octree; the quantization step size q under the spherical coordinates (ρ, θ, φ) described in the spherical coordinates ρ ,q θ , Respectively expressed as:

[0020] q ρ =q;

[0021]

[0022] Where b represents the quantization step size q ρ The total number of bins for the ρ coordinate is expressed as ρ is the radial distance from the origin to the point in spherical coordinates; θ is the polar angle measured downward from the positive z-axis; φ is the azimuth measured along the projection of the xy plane starting from the positive x-axis; q is the octree quantization step size based on the Cartesian coordinate system; ρ max is the maximum value of radial distance;

[0023] S12), introduce the reconstruction error ε of the spherical coordinate octree s ;Right now:

[0024]

[0025] In the formula, p i is the original point cloud coordinate; is the center coordinate of the voxel;

[0026] S13), obtain the original point cloud data, and convert the Cartesian coordinate system (x, y, z) in the data into a spherical coordinate system Calculate the maximum radial distance ρ after coordinate transformation max ;

[0027] Calculate the quantization step size according to the formula in step S11), and normalize the converted point cloud data; calculate the normalized maximum radial distance r in the point cloud data again. max , using the maximum radial distance r max Construct an octree with the processed point cloud;

[0028] S14), according to r max Calculate the boundary range of the point cloud, and then calculate the center and maximum range of the bounding box that surrounds the entire point cloud; starting from the root node of the octree, decide whether to continue to divide the current octant node into child nodes or keep it as a leaf node based on the maximum depth of the preset tree; if the current octant node has not reached the maximum depth, recursively split it into eight child nodes based on the center point of the node; each child node covers a specific sub-area of ​​the parent node space;

[0029] For each octree node, add node features, including the center sphere coordinates Quadrant (octant), tree depth (depth), occupancy symbol (occupancy); recursively build until the maximum depth to obtain an octree containing node features.

[0030] Preferably, in step S11), the point cloud is divided into N levels, and a different number of additional quantization levels are allocated to each level; the number of additional quantization levels for each level is n, and the value range is {0, 1, ..., N-1}, and the ρ of each level is n In the interval [t n ρ max,t n+1 ρ max ), where t n is the critical value of the partition, satisfying 0≤t n <t n+1 <1,t0=0,t N =1; the additional quantization level is expressed as a reduction in the quantization step size, specifically q n =q / 2 n .

[0031] Preferably, in step S12), the reconstruction error ε s The upper limit is controlled to be less than or equal to the octree reconstruction error ε in the Cartesian coordinate system c The upper limit of the spherical coordinate octree with the multi-level concept is introduced, and its reconstruction error ε n It is expressed as:

[0032]

[0033] Preferably, in step S2), constructing the adjacency matrix of the parent graph and the distance graph specifically includes the following steps:

[0034] S21), obtaining the adjacency matrix of the parent graph according to the hierarchical relationship of the multi-level spherical coordinate octree constructed in step S1);

[0035] S22), constructing an adjacency matrix of the distance graph based on the geometric distance relationship of the point cloud.

[0036] Preferably, in step S21), a parent graph is constructed by traversing ancestor nodes of multiple levels of a multi-level spherical coordinate octree; and unit edge weights are assigned to context nodes that share the same ancestor.

[0037] As a preferred embodiment, in step S21), for node o in the same context i and j , the edge weight of the node in the parent graph p (o i ,o j ) is calculated as:

[0038]

[0039]

[0040] Where m represents the search node o i and j The ancestor node is the depth of the current octree; Respectively represent the current octree depth m, node o i and j ancestor node.

[0041] As a preferred embodiment, in step S22), for the node o of the spherical coordinate octree i and j , the distance between them is D(o i ,o j ) is expressed as:

[0042]

[0043] In the formula, Represents node o i and j The spherical coordinate radial length ρ; Represents node o i and j The polar angle θ of the spherical coordinates; Represents node o i and j The azimuth angle φ of the spherical coordinates.

[0044] As a preferred embodiment, in step S22), for the node o of the spherical coordinate octree i and j The unit edge weight between is expressed as:

[0045]

[0046] Where D max , D min Represents node o i and j The maximum and minimum distances between them; the above regularization operation can limit the final unit edge weight to between [0,1].

[0047] The entropy coding model based on graph-driven attention includes a graph convolutional network module, a grouped graph attention module and a cross attention module;

[0048] Preferably, in step S3), the grouped graph attention module is used to learn the context information of the parent node, which reduces the computational complexity by reducing the calculation of distant points, enhances the local context learning ability, and shares information between groups to capture global features;

[0049] The described criss-cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context.

[0050] Preferably, in step S4), the graph convolutional network module includes multiple linear layers and a multi-layer perceptron MLP.

[0051] Preferably, in step S4), a graph convolutional network module is used to embed contextual information into the adjacency matrix, specifically comprising: the graph convolutional network module maps the contextual information to be embedded to the target dimension through a linear layer Linear, and then multiplies it with the adjacency matrix; the obtained features are passed through another linear layer Linear and then multiplied with the adjacency matrix; and the obtained features are normalized by LayerNorm; finally, the convolution output is mapped to the target output dimension through a multi-layer perceptron MLP and a residual connection, so as to obtain the adjacency matrix embedded with the contextual information.

[0052] Preferably, in step S5), the grouped graph attention module is used to learn the context information of the parent node, which reduces the computational complexity by reducing the calculation of distant points, enhances the local context learning ability, and shares information between groups to capture global features;

[0053] The grouped graph attention module splits the long sequence context into groups according to the edge weights of the adjacency matrix embedded with the context information, specifically comprising the following steps:

[0054] S511), alternately input the adjacency matrix of the parent graph / distance graph, and transform the tensor input as (batch_size, N, in_feature) into (batch_size*n, N / n, in_feature) accordingly; where n is the number of groups, and in_feature is the feature dimension of the input tensor;

[0055] S512), mapping the features of the context information to the target dimension out_feature through a linear layer;

[0056] S513), then through a horizontal copy and splicing, the data dimension is expanded from (batch_size*n, N / n, out_feature) to (batch_size*n, N / n, N / n, out_feature);

[0057] S514), after another vertical copy and splicing, the data dimension is expanded from (batch_size*n, N / n, N / n, out_feature) to (batch_size*n, N / n, N / n, 2*out_feature); where N represents the length of the input sequence;

[0058] S515), then use the adjacency matrix of the corresponding part to perform mask processing; extract the attention weights between nodes;

[0059] S516), multiply the attention weight by the value vector previously mapped to the target dimension through linear transformation to obtain the attention output, and finally capture the inter-group information through the multi-layer perceptron MLP to achieve more refined feature learning, reshape the data dimension, merge and restore the groups, and obtain the final output result of the group graph attention module.

[0060] Preferably, in step S5), the cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context, as follows:

[0061] First, the parent node context is projected to the same feature dimension through two linear layers to generate key K and value V vectors;

[0062] Then, the neighbor node context is compressed into feature dimensions through a linear layer to generate the query Q vector;

[0063] Next, query the parent node context through the neighbor node context, that is, multiply the query Q vector of the neighbor node context with the key K vector of the parent node context; and normalize it through the softmax activation function to obtain the attention weight;

[0064] Finally, the attention weight is multiplied by the value V vector of the parent node context to get the final result.

[0065] Preferably, in step S6), the probability distribution of the placeholder symbol of each octree node is calculated by the entropy coding model. Make a prediction, that is:

[0066]

[0067] In the formula, Represents the probability value of the entropy coding model at a given point a; n a Indicates the occupancy of the coding node; f a 、f a ′ represents the embedding of the current feature and the reference feature respectively; ω is the context window length, and w is the weight of the entropy model.

[0068] Preferably, in step S6), the entropy coding model aims to minimize the estimated distribution The cross entropy between the real distribution Q(n) is used to improve the accuracy; therefore, the objective function of the entropy coding model is:

[0069] E n~Q [-log n Q(n)];

[0070] Where E represents the expected logarithmic loss of the predicted value of node symbol n by the entropy coding model prediction distribution Q when the node symbol n follows the distribution Q; Q(n) represents the true probability distribution of the node placeholder symbol n generated by the actual data; n represents the node placeholder symbol of the octree.

[0071] Preferably, in step S7), arithmetic coding is used to compress the placeholder symbol sequence of the octree node into a binary floating point number sequence, which specifically includes the following steps:

[0072] S71), establish frequency table;

[0073] S72) k (x i )=(b,v) form to create a series of nested intervals, where Φ k (x i ) represents the number axis mapping interval; b represents the starting point of the nested interval, v represents the length of the nested interval, x i The placeholder symbol for node i; the initial interval Φ0(x i ) is [0,1), and in each step, the interval is further divided; as shown in the following formula:

[0074] Φ0(x i )=(b0,v0)=(0,1);

[0075] Φ k (x i )=(b k ,v k )=(b k-1 +c(s k ) k-1 ,p(s k ) k-1 ); k = 1, 2, ..., 256;

[0076] In the formula, s k The placeholder x represents the node i i The kth format of p(s k ) indicates the format s k The probability of c(s k ) represents the cumulative probability; l k-1 is the length of the previous interval; b k-1 is the starting point of the previous interval;

[0077] S73), the placeholder symbol x of node i is i Map to a value Y on the real number interval [0,1), and use the binary expansion of Y as the compression encoding result; place the symbol x in the node to be encoded i The corresponding number axis mapping interval Φ k (xi ), select a floating point number to represent the placeholder symbol x of node i i .

[0078] Preferably, in step S8), converting the binary floating point number sequence into a bit stream specifically includes:

[0079] S81), selecting a finite floating point number within a specified range to represent a placeholder symbol of a node, and converting each floating point number into a binary form, and then converting it into a byte stream;

[0080] S82), according to the interval length v k , dynamically select the floating point number with the least number of binary digits to represent each node;

[0081] S83), storing the dynamic floating point binary bits of each node and converting them into a bit stream;

[0082] S84), using breadth-first search to connect the bit streams of all nodes in the octree into a bit stream with a certain length;

[0083] S85) A predicted probability frequency table of the node placeholder symbol is established through arithmetic decoding, and is mapped to the real number interval [0,1), and the sequence number mapped to the interval is converted into binary form, thereby obtaining the decoded node placeholder symbol.

[0084] Preferably, in step S9), arithmetic decoding is used to convert the bit stream into a placeholder symbol sequence of an octree, and the octree is reconstructed and the point cloud data is restored, which specifically includes:

[0085] S91), obtaining information required for decoding, including bit stream data, coordinates of the root node and edge length;

[0086] S92), construct the root node of the octree, and input the root node feature into the entropy coding model to obtain the probability distribution of the placeholder symbol of the root node Then it is used as the frequency table for arithmetic decoding;

[0087] S93), intercepting the portion of the layer where the root node is located from the bit stream and parsing it into a floating point number using a frequency table,

[0088] Get an 8-bit placeholder symbol, which represents the distribution of child nodes of the root node;

[0089] S94), reconstruct the next layer of nodes according to the child node distribution of the root node obtained in step S93), specifically, for the current root node, create an octree node for each child node with an occupied symbol of 1, generate node features; and input it into the entropy coding model to obtain the probability distribution of the placeholder symbols of the new layer generating a plurality of frequency tables for arithmetic decoding;

[0090] S95), intercepting a new layer of part from the bit stream, and parsing it into a new layer of symbol sequence using the new decoded frequency table, and then continuing to reconstruct the next layer of the octree, repeating the above process until the reconstruction of the octree is completed;

[0091] S96), by traversing the reconstructed octree structure, extracting the center points of all sub-leaf nodes, and completing the restoration of the octree-encoded point cloud data.

[0092] The beneficial effects of the present invention are:

[0093] 1. The present invention reduces quantization error, improves compression efficiency, balances computational complexity and compression effect, and significantly enhances adaptability to high-resolution point cloud data by combining a multi-level spherical coordinate octree structure, graph convolution, grouped graph attention module, and cross attention module;

[0094] 2. The present invention adopts an octree structure based on multi-level spherical coordinates to solve the problem that redundant information of point clouds cannot be fully utilized in the Cartesian coordinate system; the octree is constructed by the spherical coordinate system, and the point cloud data is hierarchically processed according to the distance, so that the points acquired by the same laser beam are more easily assigned to the same voxel, thereby making it easier to learn local features, reduce redundant information processing, and achieve higher quality and efficient encoding;

[0095] 3. The present invention uses a method of embedding contextual information through graph convolution to solve the problem that existing methods fail to effectively utilize the intrinsic correlation of nodes in the octree structure; by constructing the adjacency matrix of the parent graph and the distance graph, and using graph convolution to embed contextual information, the attention mechanism can introduce more semantic details, enhance context understanding and prediction accuracy;

[0096] 4. The present invention adopts a grouped graph attention module to solve the problem that the attention mechanism has high computational complexity and fails to fully capture local features; the grouped graph attention module reduces the computational complexity by reducing the calculation of distant points, while enhancing the local context learning ability, and sharing information between groups to capture global features, thereby improving efficiency and feature extraction capabilities;

[0097] 5. The present invention uses a cross-attention module to solve the problem that existing methods fail to effectively utilize the high-resolution information expression of the octree; through the cross-attention module, the correlation between the context of the parent node and the neighbor node is learned, the receptive field of the model is expanded, the information expression is enriched, and the feature representation capability is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a framework diagram of the point cloud encoding method of the present invention;

[0099] Figure 2 It is a framework diagram of the graph convolutional network module of the present invention;

[0100] Figure 3 A framework diagram of the grouped graph attention and cross-attention of the present invention;

[0101] Figure 4 FIG. 4 is an error color gamut diagram of an experiment of an embodiment of the present invention. DETAILED DESCRIPTION

[0102] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0103] like Figure 1 As shown, this embodiment provides a point cloud encoding method based on a multi-level ball octree and a graph-driven attention entropy model, including:

[0104] S1) Encoding the point cloud data using an octree entropy model based on multi-level spherical coordinates; specifically comprising the following steps:

[0105] S11), introduce the quantization step size of the spherical coordinate octree; the quantization step size q under the spherical coordinates (ρ, θ, φ) described in the spherical coordinates ρ ,q θ , Respectively expressed as:

[0106] q ρ =q;

[0107]

[0108] Where b represents the quantization step size q ρ The total number of bins for the ρ coordinate is expressed as ρ is the radial distance from the origin to the point in spherical coordinates; θ is the polar angle measured downward from the positive z-axis; φ is the azimuth measured along the projection of the xy plane starting from the positive x-axis; q is the octree quantization step size based on the Cartesian coordinate system; ρ max is the maximum value of radial distance;

[0109] S12), introduce the reconstruction error ε of the spherical coordinate octree s ;Right now:

[0110]

[0111] In the formula, p i is the original point cloud coordinate; is the center coordinate of the voxel;

[0112] After observation, it can be found that the upper limit of the error of the octree in spherical coordinates is linearly related to ρ; therefore, for the divided voxels, the farther away from the center point and the larger the volume, the larger the upper limit of the error;

[0113] S13), obtain the original point cloud data, and convert the Cartesian coordinate system (x, y, z) in the data into a spherical coordinate system Calculate the maximum radial distance ρ under the transformed data max , calculate the quantization step size according to the formula in step S11), and normalize the converted point cloud data; calculate the normalized maximum radial distance r in the point cloud again max , using r max Construct an octree with the processed point cloud;

[0114] S14), first according to r max Calculate the boundary range of the point cloud, and then calculate the center and maximum range of the bounding box that surrounds the entire point cloud; these values ​​are used to determine the attributes of the octree root node; starting from the root node of the octree, decide whether to continue to divide the current octant node into child nodes or keep it as a leaf node based on the artificially specified maximum depth of the tree; if the current octant node has not reached the maximum depth, it will be recursively split into eight child nodes, and the division is based on the center point of the node; each child node covers a specific sub-area of ​​the parent node space;

[0115] For each octree node, add node features, including the center sphere coordinates Quadrant (octant), tree depth (depth), occupancy symbol (occupancy); recursively build until the maximum depth to obtain an octree containing node features.

[0116] As a preferred embodiment of this invention, in step S11), the point cloud is divided into N levels, and a different number of additional quantization levels are allocated to each level; the number of additional quantization levels for each level is n, and the value range is {0, 1, ..., N-1}. On this basis, the ρ of each level is n In the interval [t n ρ max ,t n+1 ρ max ), where t n is the critical value of the partition, satisfying 0≤t n <t n+1 <1,t0=0,t N =1; the additional quantization level is expressed as a reduction in the quantization step size, specifically q n =q / 2 n ; In actual operation, the number of levels N is set to 3; the corresponding

[0117] Therefore, the reconstruction error ε s The upper limit is controlled to be less than or equal to the octree reconstruction error ε in the Cartesian coordinate system cThe upper limit of the spherical coordinate octree with the multi-level concept is introduced, and its reconstruction error ε n It is expressed as:

[0118]

[0119] S2) constructing the adjacency matrix of the parent graph and the distance graph using the octree structure and the spherical coordinate representation of the point cloud; specifically comprising the following steps:

[0120] S21), according to the hierarchical relationship of the multi-level spherical coordinate octree constructed in step S1), obtain the adjacency matrix of the parent graph. This embodiment constructs the parent graph by traversing the ancestor nodes of multiple levels of the multi-level spherical coordinate octree; and for context nodes that share the same ancestor, a unit edge weight is assigned. For nodes o in the same context i and j , the edge weight of the node in the parent graph p (o i ,o j ) is calculated as:

[0121] Edge p (o i ,o j )=∑1 m U m ;

[0122]

[0123] Where m represents the search node o i and j The ancestor node is the depth of the current octree; Respectively represent the current octree depth m, node o i and j ancestor node.

[0124] S22), construct an adjacency matrix of the distance graph based on the geometric distance relationship of the point cloud; in this embodiment, for the node o of the spherical coordinate octree i and j , the distance between them is D(o i ,o j ) is expressed as:

[0125]

[0126] In the formula, Represents node o i and j The spherical coordinate radial length ρ; Represents node o i and j The polar angle θ of the spherical coordinates; Represents node o i and j The azimuth angle φ of the spherical coordinates.

[0127] In this embodiment, for the node o of the spherical coordinate octree i and j The unit edge weight between is expressed as:

[0128]

[0129] Where D max , D min Represents node o i and j The maximum and minimum distances between them; the above regularization operation can limit the final unit edge weight to between [0,1].

[0130] S3), constructing an entropy coding model based on graph-driven attention; wherein the grouped graph attention module is used to learn the context information of the parent node, reduces the computational complexity by reducing the calculation of distant points, enhances the local context learning ability, and shares information between groups to capture global features;

[0131] The described criss-cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context.

[0132] S4), using the graph convolutional network module to use the adjacency matrix to assist in embedding context information; Figure 2 As shown, the graph convolutional network module described in this embodiment includes multiple linear layers Linear and multi-layer perceptrons MLP. Using the graph convolutional network module to embed context information into the adjacency matrix specifically includes: the graph convolutional network module maps the context information to be embedded to the target dimension through the linear layer Linear, and then multiplies it with the adjacency matrix; the obtained features are multiplied with the adjacency matrix after passing through another linear layer Linear; and the obtained features are normalized by LayerNorm; finally, the convolution output is mapped to the target output dimension through a multi-layer perceptron MLP and residual connection, so as to obtain the embedded context information.

[0133] S5), construct a grouped graph attention module and a cross attention module to learn the relevance of the parent node context and the sibling node context; Figure 3 As shown, in this embodiment, the grouped graph attention module is used to learn the context information of the parent node, reduces the computational complexity by reducing the calculation of distant points, enhances the local context learning ability, and shares information between groups to capture global features; thereby simplifying the calculation while enhancing the feature representation.

[0134] The grouped graph attention module splits the long sequence context into groups according to the edge weights of the adjacency matrix embedded with the context information, specifically comprising the following steps:

[0135] S511), alternately input the adjacency matrix of the parent graph / distance graph, and transform the tensor input as (batch_size, N, in_feature) into (batch_size*n, N / n, in_feature) accordingly; where n is the number of groups, and in_feature is the feature dimension of the input tensor;

[0136] S512), mapping the features of the context information to the target dimension out_feature through a linear layer;

[0137] S513), then through a horizontal copy and splicing, the data dimension is expanded from (batch_size*n, N / n, out_feature) to (batch_size*n, N / n, N / n, out_feature);

[0138] S514), after another vertical copy and splicing, the data dimension is expanded from (batch_size*n, N / n, N / n, out_feature) to (batch_size*n, N / n, N / n, 2*out_feature); where N represents the length of the input sequence;

[0139] S515), then use the adjacency matrix of the corresponding part to perform mask processing; extract the attention weights between nodes;

[0140] S516), multiply the attention weight by the value vector previously mapped to the target dimension through linear transformation to obtain the attention output, and finally capture the inter-group information through the multi-layer perceptron MLP to achieve more refined feature learning, reshape the data dimension, merge and restore the groups, and obtain the final output result of the group graph attention module.

[0141] The grouped graph attention module of the present invention reduces the calculation of distant nodes and focuses on the local window, with a computational complexity of Compared with the traditional attention mechanism, it reduces where k is the size of the sub-window. Since the node dependencies are more concentrated in the octree context, even with a smaller k, better prediction results can be achieved.

[0142] In this embodiment, the cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context, as follows:

[0143] First, the parent node context is projected to the same feature dimension through two linear layers to generate key K and value V vectors;

[0144] Then, the neighbor node context is compressed into feature dimensions through a linear layer to generate the query Q vector;

[0145] Next, query the parent node context through the neighbor node context, that is, multiply the query Q vector of the neighbor node context with the key K vector of the parent node context; and normalize it through the softmax activation function to obtain the attention weight;

[0146] Finally, the attention weight is multiplied by the value V vector of the parent node context to get the final result.

[0147] S6), predicting the probability of the placeholder symbol of each octree node based on the entropy coding model; specifically:

[0148] The probability distribution of the placeholder symbol of each octree node is calculated by the entropy coding model. Make a prediction, that is:

[0149]

[0150] In the formula, Represents the probability value of the entropy coding model at a given point a; n a Indicates the occupancy of the coding node; f a 、f a ′ represents the embedding of the current feature and the reference feature respectively; ω is the length of the context window, and w is the weight of the entropy model;

[0151] because The distribution of depends only on the octree node n a local context window; therefore, using the current sequence and selected reference sequences to make predictions about the distribution using all the features of , where t is the number of time steps.

[0152] In this embodiment, the entropy coding model aims to minimize the estimated distribution The cross entropy between the real distribution Q(n) is used to improve the accuracy; therefore, the objective function of the entropy coding model is:

[0153] E n~Q [-log n Q(n)];

[0154] Where E represents the expected logarithmic loss of the predicted value of node symbol n by the entropy coding model prediction distribution Q when the node symbol n follows the distribution Q; Q(n) represents the true probability distribution of the node placeholder symbol n generated by the actual data; n represents the node placeholder symbol of the octree.

[0155] S7), using arithmetic coding to compress the placeholder symbol sequence of the octree node into a binary floating point number sequence; specifically comprising the following steps:

[0156] S71), establish a frequency table; specifically includes the following steps:

[0157] S711), collect and count the placeholder symbol x of each node i The frequency of occurrence, since there may be 256 symbols in the octree node, count the number of occurrences of each symbol;

[0158] S712), divide the number of occurrences of each symbol by the total number of symbols to obtain the probability p(s) of each symbol k ), indicating the symbol s k The relative frequency of occurrence in the node symbol set;

[0159] S713), sort the symbols in order of appearance from the lowest probability to the highest probability, and calculate the cumulative probability c(s) of each symbol one by one k );Cumulative probability symbol s k The position in the frequency table forms an interval from 0 to 1, and its calculation formula is:

[0160]

[0161] S714) Based on the frequency table, associate the interval size with the probability of symbol occurrence.

[0162] S72) k (x i )=(b,v) form to create a series of nested intervals, where Φ k (x i ) represents the number axis mapping interval; b represents the starting point of the nested interval, v represents the length of the nested interval, x i The placeholder symbol for node i; the initial interval Φ0(x i ) is [0,1), and in each step, the interval is further divided; as shown in the following formula:

[0163] Φ0(x i )=(b0,v0)=(0,1);

[0164] Φ k (x i )=(b k ,vk )=(b k-1 +c(s k ) k-1 ,p(s k ) k-1 ); k = 1, 2, ..., 256;

[0165] In the formula, s k The placeholder x represents the node i i The kth format of p(s k ) indicates the format s k The probability of c(s k ) represents the cumulative probability; l k-1 is the length of the previous interval; b k-1 is the starting point of the previous interval.

[0166] S73), the placeholder symbol x of node i is i Map to a value Y on the real number interval [0,1), and use the binary expansion of Y as the compression encoding result; place the symbol x in the node to be encoded i The corresponding number axis mapping interval Φ k (x i ), select a floating point number to represent the placeholder x of node i i .

[0167] S8), converting the binary floating point number sequence into a bit stream; specifically comprising the following steps:

[0168] S81), selecting a finite floating point number within a specified range to represent a placeholder symbol of a node, and converting each floating point number into a binary form, and then converting it into a byte stream;

[0169] S82), according to the interval length v k , dynamically select the floating point number with the least number of binary digits to represent each node; for example, for the interval length v k ∈[0.25,0.5), the potential selection set is {0,0.25,0.5,0.75}, its corresponding binary form is {0.002,0.012,0.102,0.112}, and the minimum number of bits used is

[0170] S83), storing the dynamic floating point binary bits of each node and converting them into a bit stream;

[0171] S74), using breadth-first search to connect the bit streams of all nodes in the octree into a bit stream with a certain length;

[0172] S85) A predicted probability frequency table of the node placeholder symbol is established through arithmetic decoding, and is mapped to the real number interval [0,1), and the sequence number mapped to the interval is converted into binary form, thereby obtaining the decoded node placeholder symbol.

[0173] S9), using arithmetic decoding to convert the bit stream into an octree placeholder symbol sequence, and reconstructing it into an octree and restoring the point cloud data, specifically including the following steps:

[0174] S91), obtaining information required for decoding, including bit stream data, coordinates of the root node and edge length;

[0175] S92), construct the root node of the octree, and input the root node feature into the entropy coding model to obtain the probability distribution of the placeholder symbol of the root node Then it is used as the frequency table for arithmetic decoding;

[0176] S93), intercepting the portion of the layer where the root node is located from the bit stream and parsing it into a floating point number using a frequency table to obtain an 8-bit placeholder symbol, which represents the distribution of the child nodes of the root node;

[0177] S94), reconstruct the next layer of nodes according to the child node distribution of the root node obtained in step S93), specifically, for the current root node, create an octree node for each child node with an occupied symbol of 1, generate node features; and input it into the entropy coding model to obtain the probability distribution of the placeholder symbols of the new layer generating a plurality of frequency tables for arithmetic decoding;

[0178] S95), intercepting a new layer of part from the bit stream, and parsing it into a new layer of symbol sequence using the new decoded frequency table, and then continuing to reconstruct the next layer of the octree, repeating the above process until the reconstruction of the octree is completed;

[0179] S96), by traversing the reconstructed octree structure, extracting the center points of all sub-leaf nodes, and completing the restoration of the octree-encoded point cloud data.

[0180] In addition, the 3D point cloud datasets SemanticKITTI dataset and Ford dataset used in this embodiment are used to verify the method of this embodiment.

[0181] The SemanticKITTI dataset contains 22 sequences with a total of 43,504 scans, each containing more than 120,000 individual points. Sequences 00 to 10 are used for training, and sequences 11 to 21 are used for testing. The Ford dataset contains more than 1,500 scans of static point cloud scenes, using the same training / testing split as the Ford benchmark task, including 1,045 for training, 156 for validation, and 312 for testing. For the SemanticKITTI dataset, the quantization step size is set to D is the octree depth, the maximum octree depth is 14 in training, 9 to 14 in testing, and the sequence length is 32 to 1024. For the Ford dataset, the quantization step size is set to The maximum depth of the octree is 16 during training and 11 to 16 during testing. The point-to-point peak signal-to-noise ratio (D1 PSNR), point-to-plane peak signal-to-noise ratio (D2 PSNR) and Chamfer distance (CD) are used to evaluate the quality of point cloud reconstruction. Figure 4 As shown in the figure, the closer the color error is to blue, the closer the reconstructed 3D point is to the original point. This embodiment uses GPCC and EHEM as comparisons; GPCC is a point cloud compression method proposed by MPEG, an international moving picture expert group; EHEM (Efficient Hierarchical Entropy Model) is an existing point cloud compression method. The results show that the method of this embodiment significantly reduces the amount of data after compression, and because the entropy coding model based on multi-level spherical coordinate octrees and graph-driven attention encodes the 3D point cloud data more efficiently, the reconstructed point cloud data can still maintain a high degree of accuracy, while ensuring the quality of reconstruction by reducing storage and transmission overhead.

[0182] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A point cloud coding method based on multi-level ball octree and graph-driven attention entropy model, characterized in that: The following steps are involved: S1), using an octree entropy model based on multi-level spherical coordinates to encode point cloud data; S2), constructing the adjacency matrix of the parent graph and the distance graph using the octree structure and the spherical coordinate representation of the point cloud; S3), construct an entropy coding model based on graph-driven attention; S4), using the graph convolutional network module and the adjacency matrix to assist in embedding context information; S4), using the grouped graph attention module and the cross attention module to learn the relevance of the parent node context and the sibling node context; S5), predicting the probability of the placeholder symbol of each octree node based on the entropy coding model; S6), using arithmetic coding to compress the placeholder symbol sequence of the octree node into a binary floating point number sequence; S7), converting the binary floating point number sequence into a bit stream; S8), using arithmetic decoding to convert the bit stream into a placeholder symbol sequence of the octree, reconstruct it into the octree and restore the point cloud data.

2. According to claim 1, a point cloud encoding method based on a multi-level ball octree and a graph-driven attention entropy model is characterized in that: In step S1), the reconstruction error ε of the multi-level spherical coordinate octree is n It is expressed as: In the formula, q n =q / 2 n ; ρ max is the maximum value of radial distance; t n is the critical value of the partition; q is the octree quantization step based on the Cartesian coordinate system; n is the number of additional quantization levels per level.

3. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 1, characterized in that: In step S2), the adjacency matrix of the parent graph and the distance graph is constructed, which specifically includes the following steps: S21), obtaining the adjacency matrix of the parent graph according to the hierarchical relationship of the multi-level spherical coordinate octree constructed in step S1); S22), constructing an adjacency matrix of the distance graph based on the geometric distance relationship of the point cloud.

4. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 3, characterized in that: In step S21), a parent graph is constructed by traversing the ancestor nodes of multiple levels of the multi-level spherical coordinate octree; and for context nodes that share the same ancestor, a unit edge weight is assigned; for nodes o in the same context i and j , the edge weight of the node in the parent graph p (o i ,o j ) is calculated as: Where m represents the search node o i and j The ancestor node is the depth of the current octree; Respectively represent the current octree depth m, node o i and j ancestor node.

5. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 4, characterized in that: In step S22), for the node o of the spherical coordinate octree i and j , the distance between them is D(o i ,o j ) is expressed as: In the formula, Represents node o i and j The spherical coordinate radial length ρ; Represents node o i and j The polar angle θ of the spherical coordinates; Represents node o i and j The azimuth angle φ of the spherical coordinates.

6. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 5, characterized in that: In step S22), for the node o of the spherical coordinate octree i and j The unit edge weight between is expressed as: Where D max , D min Represents node o i and j The maximum and minimum distances between them; the above regularization operation can limit the final unit edge weight to between [0,1].

7. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 1, characterized in that: The entropy coding model based on graph-driven attention includes a graph convolutional network module, a grouped graph attention module and a cross attention module; The grouped graph attention module is used to learn the context information of the parent node, which reduces the computational complexity by reducing the calculation of distant points, enhances the local context learning ability, and shares information between groups to capture global features; The described criss-cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context.

8. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 7, characterized in that: In step S4), the graph convolutional network module includes multiple linear layers and multi-layer perceptrons (MLPs); the graph convolutional network module maps the context information to be embedded to the target dimension through the linear layer, and then multiplies it with the adjacency matrix; the obtained features are passed through another linear layer and then multiplied with the adjacency matrix; the obtained features are normalized by LayerNorm; finally, the convolution output is mapped to the target output dimension through a multi-layer perceptron (MLP) and a residual connection, thereby obtaining the embedded context information.

9. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 7, characterized in that: In step S5), the grouped graph attention module splits the long sequence context into groups according to the edge weights of the adjacency matrix embedded with the context information, specifically comprising the following steps: S511), alternately input the adjacency matrix of the parent graph / distance graph, and transform the tensor input as (batch_size, N, in_feature) into (batch_size*n, N / n, in_feature) accordingly; where n is the number of groups, and in_feature is the feature dimension of the input tensor; S512), mapping the features of the context information to the target dimension out_feature through a linear layer; S513), then through a horizontal copy and splicing, the data dimension is expanded from (batch_size*n, N / n, out_feature) to (batch_size*n, N / n, N / n, out_feature); S514), after another vertical copy and splicing, the data dimension is expanded from (batch_size*n, N / n, N / n, out_feature) to (batch_size*n, N / n, N / n, 2*out_feature); where N represents the length of the input sequence; S515), then use the adjacency matrix of the corresponding part to perform mask processing; extract the attention weights between nodes; S516), multiply the attention weight by the value vector previously mapped to the target dimension through linear transformation to obtain the attention output, and finally capture the inter-group information through the multi-layer perceptron MLP to achieve more refined feature learning, reshape the data dimension, merge and restore the groups, and obtain the final output result of the group graph attention module.

10. The point cloud coding method based on multi-level ball octree and graph-driven attention entropy model according to claim 7, characterized in that: In step S5), the cross attention module enhances feature extraction by learning between the encoded neighbor node context and the parent node context, as follows: First, the parent node context is projected to the same feature dimension through two linear layers to generate key K and value V vectors; Then, the neighbor node context is compressed into feature dimensions through a linear layer to generate the query Q vector; Next, query the parent node context through the neighbor node context, that is, multiply the query Q vector of the neighbor node context with the key K vector of the parent node context; and normalize it through the softmax activation function to obtain the attention weight; Finally, the attention weight is multiplied by the value V vector of the parent node context to get the final result.

Citation Information

Patent Citations

  • Multi-view fusion space-time dynamic graph convolutional network urban traffic flow prediction method

    CN116935649A

  • Traffic accident prediction systems and methods

    US20230140289A1

Cited By

  • Point cloud attribute coding method based on graph attention network

    CN119676446A

  • Point cloud compression method and device based on multi-scale octree attention mechanism

    CN120075476A

  • Semantic scene graph-based double-flow point cloud compression method and system

    CN121937547A