A point cloud data reconstruction method based on a GCN-Transformer model and an electronic device

Abstract

The application discloses a point cloud data reconstruction method and electronic equipment based on a GCN-Transformer model, belongs to the technical field of point cloud data processing, and comprises the following steps: acquiring a point cloud data set; constructing a GCN-Transformer model and optimizing and training the GCN-Transformer model by using the point cloud data set; inputting to-be-processed point cloud data into the trained GCN-Transformer model, and completing reconstruction of the to-be-processed point cloud data. The GCN-Transformer model combines the structure of a GCN encoder and a Transformer decoder, utilizes the node feature extraction of a GCN network on a graph and the coding of a self-attention mechanism of a Transformer on point cloud data, and realizes efficient feature extraction and reconstruction of point cloud data.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a point cloud data reconstruction method and electronic device based on the GCN-Transformer model. Background Technology

[0002] In recent years, point cloud data has been widely used in computer vision, autonomous driving, and robotics. Segmentation and reconstruction of point cloud data are crucial tasks in point cloud processing. Traditional methods often require manual data labeling and feature extraction algorithm design, resulting in low efficiency. Deep learning-based methods have gradually become mainstream in point cloud data processing. The Transformer model, initially used in natural language processing, is particularly effective in tasks such as text generation, language translation, and sentiment analysis. The Generative Networking Model (GCN) is primarily applied to graph data analysis, such as social network analysis and drug discovery. GCN models can perform embedding learning on graph data, enabling tasks like node feature extraction, classification, and clustering. Compared to traditional graph analysis methods, GCN models can automatically learn relationships between nodes and perform non-linear embedding, thereby improving the expressive power and prediction accuracy of graph data. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of traditional point cloud data segmentation and reconstruction methods in the prior art, which often require manual data annotation and feature extraction algorithm design, resulting in low efficiency. This invention provides a point cloud data reconstruction method and electronic device based on the GCN-Transformer model.

[0004] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0005] A point cloud data reconstruction method based on the GCN-Transformer model includes the following steps:

[0006] S1: Obtain the point cloud dataset;

[0007] S2: Construct the GCN-Transformer model and optimize and train the GCN-Transformer model using the point cloud dataset;

[0008] The GCN-Transformer model includes a GCN layer, a GCN encoder, and a Transformer decoder;

[0009] The GCN layer is used to preprocess the point cloud data in the point cloud dataset and establish the adjacency matrix of the point cloud data.

[0010] The GCN encoder is used to extract node features based on the adjacency matrix and output a node feature matrix;

[0011] The Transformer decoder is used to encode the point cloud data using a self-attention mechanism based on the node feature matrix, extract feature vectors from the encoded point cloud data, and reconstruct the point cloud data based on the feature vectors.

[0012] S3: Input the point cloud data to be processed into the trained GCN-Transformer model to complete the reconstruction of the point cloud data to be processed.

[0013] By adopting the above technical solution, and combining the GCN encoder and Transformer decoder structure, the GCN network is used to extract node features of the graphics and the Transformer's self-attention mechanism is used to encode point cloud data, thereby achieving efficient feature extraction and reconstruction of point cloud data.

[0014] As a preferred embodiment of the present invention, the GCN encoder consists of multiple graph convolutional layers;

[0015] The Transformer decoder consists of multiple Transformer modules, each of which comprises, in sequence, a feedforward network layer, a multi-head attention layer, another feedforward network layer, and a residual module.

[0016] As a preferred embodiment of the present invention, the GCN layer in step S2 is used to preprocess the point cloud data in the point cloud dataset. The establishment of the adjacency matrix of the point cloud data includes: the point cloud data is regarded as an undirected graph structure, where each point represents a node, and the distance connection relationship or proximity relationship between two nodes represents an edge. The GCN layer uses the undirected graph structure and the edges between nodes to process the point cloud data, and uses an adjacency matrix to represent the structure of the point cloud data. Each element of the adjacency matrix indicates whether there is an edge between two points.

[0017] As a preferred embodiment of the present invention, the GCN encoder described in step S2, which extracts node features based on the adjacency matrix and outputs a node feature matrix, includes:

[0018] The Laplacian matrix of the graph is constructed based on the adjacency matrix, and the formula for calculating the Laplacian matrix is ​​as follows:

[0019] L=DA

[0020] Where D is the degree matrix and A is the adjacency matrix, representing the connection between nodes;

[0021] The coordinates of the point cloud data are used as node features, and the Laplacian matrix and the node features are used as inputs to the GCN encoder. Each graph convolutional layer aggregates the node features and the features of the Laplacian matrix to generate new node features. The expression for the node feature matrix is ​​as follows:

[0022] H = GCN(X,A)

[0023] Here, H is the node feature matrix, and the GCN function uses the adjacency matrix to encode the node features.

[0024] In a preferred embodiment of the present invention, the GCN function comprises multiple graph convolutional layers. These graph convolutional layers perform convolution operations on node features, aggregate information from neighboring nodes, and update the node features. The function expression of the graph convolutional layer is as follows:

[0025]

[0026] Among them, H (l) H represents the feature matrix of the nodes in the l-th layer. (l+1) This represents the feature matrix of several points in the (l+1)th layer. I is a constant matrix. Let W be the degree matrix, σ be the activation matrix, and W be the degree matrix. (l) These are the weight parameters of the current graph's convolutional layer.

[0027] As a preferred embodiment of the present invention, step S2 involves encoding the point cloud data using a self-attention mechanism, wherein the self-attention formula is:

[0028]

[0029] Where H is the node feature matrix, W q W k W v Both are learnable parameter matrices, d k For feature dimensions.

[0030] As a preferred embodiment of the present invention, step S2 involves extracting feature vectors from the encoded point cloud data. The formula for calculating the feature vectors is as follows:

[0031] Y (m+1) =ReLU(Y (m) W (m) )

[0032] Among them, Y (m) Y is the feature vector of the m-th layer. (m+1) W is the extracted feature vector. (m) Let be the weight matrix of the m-th layer.

[0033] As a preferred embodiment of the present invention, the reconstruction of point cloud data based on the feature vector in step S2 includes: using a linear activation function or a tanh activation function to map the feature vector into the coordinates of a point, thereby completing the reconstruction of the point cloud data;

[0034] The expression for the linear activation function is:

[0035]

[0036] The expression for the Tanh activation function is:

[0037]

[0038] in, Point cloud data for reconstruction.

[0039] As a preferred embodiment of the present invention, the loss function in the GCN-Transformer model includes:

[0040] The CD loss function is used to measure the bidirectional distance between two point clouds, that is, the distance from one point to the nearest point in another point cloud:

[0041]

[0042] Where N is the number of points in the point cloud data, x i Let y be the 3D coordinates of the i-th point in the original point cloud data. i To reconstruct the 3D coordinates of the i-th point in the point cloud data;

[0043] The EMD loss function is used to measure the minimum cost required to transform one point cloud into another, which is the distance from each point to its corresponding point:

[0044]

[0045] Where φ is a bijective function that maps each point in the original point cloud data to a corresponding point in the reconstructed point cloud data;

[0046] Segmentation loss function for point cloud data segmentation tasks:

[0047]

[0048] Among them, t i p is the true label of the i-th point. i Let be the predicted probability of the i-th point;

[0049] The total loss function used to control the balance between reconstruction and segmentation tasks:

[0050] L=αLCD +βL Seg

[0051] Here, α and β represent hyperparameters.

[0052] On the other hand, an electronic device is disclosed, characterized in that it includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the point cloud data reconstruction method based on the GCN-Transformer model described above.

[0053] Compared with the prior art, the beneficial effects of the present invention are as follows: by combining the GCN encoder and Transformer decoder structure, the GCN network is used to extract node features of the graphics and the Transformer's self-attention mechanism is used to encode point cloud data, thereby achieving efficient feature extraction and reconstruction of point cloud data. Attached Figure Description

[0054] Figure 1 and Figure 2 This is a flowchart of a point cloud data reconstruction method based on the GCN-Transformer model as described in Embodiment 1 of the present invention;

[0055] Figure 3 This is a component diagram of the GCN-Transformer model in the point cloud data reconstruction method based on the GCN-Transformer model described in Embodiment 1 of the present invention.

[0056] Figure 4 This is a diagram of the GCN encoder structure of a point cloud data reconstruction method based on the GCN-Transformer model as described in Embodiment 1 of the present invention.

[0057] Figure 5 This is a Transformer decoder structure diagram of a point cloud data reconstruction method based on the GCN-Transformer model as described in Embodiment 1 of the present invention;

[0058] Figure 6 This is a structural block diagram of an electronic device according to Embodiment 2 of the present invention. Detailed Implementation

[0059] The present invention will be further described in detail below with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0060] Example 1

[0061] A point cloud data reconstruction method based on the GCN-Transformer model, such as Figure 1 and 2 As shown, it includes the following steps:

[0062] S1: Obtain the point cloud dataset;

[0063] S2: Construct the GCN-Transformer model and optimize and train the GCN-Transformer model using the point cloud dataset;

[0064] like Figure 3 As shown, the GCN-Transformer model includes a GCN layer, a GCN encoder, and a Transformer decoder;

[0065] The GCN encoder effectively utilizes the topological structure of an image to encode the node features of point cloud data, extracting both local and global features. The Transformer decoder, on the other hand, leverages self-attention mechanisms to learn the global features of the point cloud data, thereby achieving point cloud reconstruction. Therefore, this combined approach better utilizes the advantages of both GCN and Transformer, improving the processing efficiency of point cloud data.

[0066] The overall structure consisting of the GCN encoder and Transformer decoder is an end-to-end model that can directly use point cloud data as input without any preprocessing. The model's input is the adjacency matrix and node feature matrix of a point cloud dataset, and its output is the reconstructed point cloud data. During training, the model uses a reconstruction loss to optimize its parameters to minimize reconstruction error. In the testing phase, the model can be applied to various tasks involving point cloud data, such as classification, segmentation, reconstruction, and generation.

[0067] The GCN layer is used to preprocess the point cloud data in the point cloud dataset and establish the adjacency matrix of the point cloud data.

[0068] Specifically, the point cloud data is treated as an undirected graph structure, where each point represents a node, and the distance connection or proximity relationship between two nodes represents an edge. The GCN layer uses the undirected graph structure and the edges between nodes to process the point cloud data, and uses an adjacency matrix to represent the structure of the point cloud data. Each element of the adjacency matrix indicates whether there is an edge between two points.

[0069] Specifically, the GCN layer is a crucial component of the GCN-Transformer model. The GCN layer applies convolutional operations to the graph, enabling the extraction of meaningful features from nodes and edges. The input to the GCN layer consists of a graph represented by an adjacency matrix and a feature matrix, along with the input signal (i.e., feature vector) for each node in the graph. The adjacency matrix represents the connections between nodes in the graph, while the feature matrix represents the features associated with each node. The GCN layer first applies a graph convolutional operation to the input signal and the adjacency matrix to produce a new feature representation for each node in the graph. This operation involves calculating a weighted sum of the input signals of the node and its neighbors, where the weights are determined by the corresponding entries in the adjacency matrix. After the graph convolutional operation, the GCN layer applies a non-linear activation function (e.g., ReLU) to the new feature representation. This step introduces a non-linear element, enabling the model to capture more complex node relationships.

[0070] For example, distance relationships represent edges: a threshold distance is usually set according to actual needs. If the actual distance between two points is less than the threshold, they are considered to be connected by an edge.

[0071] Proximity relationships represent edges: Proximity relationships are usually based not only on distance, but may also consider the local neighborhood around a point. If two points are within the local neighborhood, they are also considered connected. This approach focuses more on the local structure of the points.

[0072] Euclidean distance or Manhattan distance is generally chosen.

[0073] Point cloud data is typically sparse, containing only a small, finite number of points. Sparse data is processed using GCN layers to preserve useful information and reduce computational complexity.

[0074] The GCN encoder is used to extract node features based on the adjacency matrix and output a node feature matrix;

[0075] like Figure 4 As shown, the GCN encoder consists of multiple graph convolutional layers. Each graph convolutional layer convolves node features and reduces their dimensionality through pooling layers. These convolutional layers continuously process and extract the node features of the point cloud data, ultimately generating global features. The graph is then encoded using GCN to extract the relationships between nodes in the graph. Specifically, this includes: editing the graph using a multi-layer graph convolutional network; and performing dimensionality reduction processing on the GCN-encoded feature vectors using pooling layers to reduce the dimensionality of the feature vectors and improve training efficiency.

[0076] Point cloud data can be represented as an N-dimensional matrix, where each row represents the coordinates of a point. For example, a point cloud dataset with M points can be represented as an N x M matrix, where each element represents the coordinates of a point.

[0077] To construct an adjacency matrix A from point cloud data, either the k-nearest neighbor algorithm or the radius-nearest neighbor algorithm can be used. Specifically, for each point, its k neighbors or those within a certain distance are considered its adjacent points. The weight of the adjacent points is set to 1, and the weights of the rest are set to 0. The size of the adjacency matrix is ​​M x M.

[0078] The Laplacian matrix L of a graph is constructed from its adjacency matrix. The Laplacian matrix can be calculated using the formula L = DA, where D is the degree matrix and A is the adjacency matrix. Specifically, the degree matrix D is a diagonal matrix whose diagonal elements represent the degree of each node, i.e., the number of neighbors of each node. The adjacency matrix A represents the connectivity between nodes, with its elements indicating whether there is a connection between node i and node j. The Laplacian matrix L represents the relationships between nodes, where the diagonal elements represent the degree of each node, and the off-diagonal elements represent the connection strength between nodes.

[0079] The coordinates of the point cloud data are used as node features, and the Laplacian matrix L and the node features are input into the GCN model for encoding. The GCN encoder consists of multiple graph convolutional layers, each of which aggregates the node features and the features of its neighboring nodes to generate new node features.

[0080] Dimensionality reduction using pooling layers: Pooling operations are performed on the output of the GCN encoder to reduce the dimensionality of the feature vectors and improve training efficiency.

[0081] Specifically, the Laplacian matrix of the graph is constructed based on the adjacency matrix, and the formula for calculating the Laplacian matrix is ​​as follows:

[0082] L=DA

[0083] Where D is the degree matrix and A is the adjacency matrix, representing the connection between nodes;

[0084] The coordinates of the point cloud data are used as node features, and the Laplacian matrix and the node features are used as inputs to the GCN encoder. Each graph convolutional layer aggregates the node features and the features of the Laplacian matrix to generate new node features. The expression for the node feature matrix is ​​as follows:

[0085] H = GCN(X,A)

[0086] Here, H is the node feature matrix, and the GCN function uses the adjacency matrix to encode the node features.

[0087] The GCN function contains multiple graph convolutional layers. These layers perform convolution operations on node features, aggregate information from neighboring nodes, and update node features. The function expression for each graph convolutional layer is as follows:

[0088]

[0089] Among them, H (l) H represents the feature matrix of the nodes in the l-th layer. (l+1) This represents the feature matrix of several points in the (l+1)th layer. I is a constant matrix. Let W be the degree matrix, σ be the activation matrix, and W be the degree matrix. (l) These are the weight parameters of the current graph's convolutional layer.

[0090] The Transformer decoder is used to encode the point cloud data using a self-attention mechanism based on the node feature matrix, extract feature vectors from the encoded point cloud data, and reconstruct the point cloud data based on the feature vectors.

[0091] like Figure 5 As shown, the Transformer decoder comprises multiple Transformer modules, each consisting of a feedforward network layer, a multi-head attention layer, another feedforward network layer, and a residual module connected in sequence. This combination enhances feature representation and improves training efficiency. In the Transformer decoder, each node's feature vector undergoes a multi-head self-attention mechanism to learn the relationships and dependencies between nodes in the point cloud data. Subsequently, a feedforward neural network further extracts key features and information. The application of residual connections helps mitigate the vanishing gradient problem, making the network easier to train and optimize. To maintain the original distribution and accuracy of the point cloud data, a linear activation function or a tanh activation function can be considered as the activation function for the output layer. To further improve the quality and visual effect of the reconstruction results, post-processing operations such as point cloud smoothing, denoising, and upsampling can be added after the output layer. These post-processing operations help optimize the reconstruction results, making them more consistent with the characteristics of point cloud data in real-world scenarios.

[0092] The node features extracted by the GCN encoder are used as the input feature vector. The node feature vector has a dimension of K, where K is the output feature dimension of the GCN encoder.

[0093] The node feature vectors are mapped through multiple fully connected layers. Each fully connected layer consists of a linear transformation and an activation function (usually ReLU), and these fully connected layers can learn the complex mapping from the node feature vectors to the output point cloud data.

[0094] Positional encoding is performed on the mapped feature vectors. Positional encoding is an important step in the Transformer decoder, used to incorporate sequence information into the feature vectors so that the decoder can understand the order information in the node feature vectors.

[0095] Decoding is performed using a Transformer decoder architecture. The decoder consists of multiple Transformer blocks, each containing a multi-head self-attention mechanism, a feedforward network, and residual connections. The decoder's function is to progressively decode the encoded feature vectors into point cloud data through multiple Transformer blocks.

[0096] The final layer considers using a linear activation function or a tanh activation function as the output layer activation function to better preserve the original distribution and accuracy of the point cloud data.

[0097] Specifically, the point cloud data is encoded using a self-attention mechanism, and the self-attention formula is as follows:

[0098]

[0099] Where H is the node feature matrix, W q W k W v Both are learnable parameter matrices, d k For feature dimensions.

[0100] Feature vectors are extracted from the encoded point cloud data. The formula for calculating the feature vectors is as follows:

[0101] Y (m+1) =ReLU(Y (m) W (m) )

[0102] Among them, Y (m) Y is the feature vector of the m-th layer. (m+1) W is the extracted feature vector. (m) Let be the weight matrix of the m-th layer.

[0103] The feature vectors are mapped to point coordinates using a linear activation function or a tanh activation function, thereby completing the reconstruction of the point cloud data;

[0104] The expression for the linear activation function is:

[0105]

[0106] The expression for the Tanh activation function is:

[0107]

[0108] in, Point cloud data for reconstruction.

[0109] S3: Input the point cloud data to be processed into the trained GCN-Transformer model to complete the reconstruction of the point cloud data to be processed.

[0110] By combining the GCN encoder and Transformer decoder structure, and utilizing the GCN network for node feature extraction of graphics and the Transformer's self-attention mechanism for encoding point cloud data, efficient feature extraction and reconstruction of point cloud data are achieved.

[0111] The Transformer model can adaptively adjust the shape and size of point cloud data, making it more robust. Furthermore, the Transformer model's decoder can automatically reconstruct point cloud data based on its features without requiring manual setting of reconstruction parameters, thus making point cloud data processing more automated and efficient. In summary, the combination of the Transformer model and the GCN model can improve the processing efficiency and accuracy of point cloud data, making it suitable for various tasks involving point cloud data, such as classification, segmentation, reconstruction, and generation.

[0112] The uniqueness of point cloud data lies in its representation as an N-dimensional matrix, where each row corresponds to the coordinates of a point. For point cloud data with M points, this forms an N x M matrix, where each element carries coordinate information in three-dimensional space. This representation makes point cloud data mathematically more complex and highly diverse.

[0113] Therefore, in order to process this kind of data, the GCN-Transformer model fully considers the three-dimensional properties of point clouds, realizes efficient feature extraction and reconstruction of point clouds, and further highlights its innovation and practical value.

[0114] The key characteristic of point cloud data is its position in three-dimensional space. This means that processing point cloud data requires considering distance, spatial relationships, and orientation, not just pixel values. This, to some extent, distinguishes point cloud data processing from traditional two-dimensional image processing.

[0115] Therefore, the GCN-Transformer model is not entirely the same as image processing when processing point cloud data. Instead, it has been adjusted and optimized according to the specific characteristics of point cloud data and application requirements. In particular, the GCN-Transformer model provides a more efficient solution in terms of the three-dimensional spatial relationships and sparsity of point cloud data, making it valuable in the field of point cloud data processing.

[0116] The loss function in the GCN-Transformer model includes:

[0117] The CD loss function is used to measure the bidirectional distance between two point clouds, that is, the distance from one point to the nearest point in another point cloud:

[0118]

[0119] Where N is the number of points in the point cloud data, x i Let y be the 3D coordinates of the i-th point in the original point cloud data. i To reconstruct the 3D coordinates of the i-th point in the point cloud data;

[0120] The EMD loss function is used to measure the minimum cost required to transform one point cloud into another, which is the distance from each point to its corresponding point:

[0121]

[0122] Where φ is a bijective function that maps each point in the original point cloud data to a corresponding point in the reconstructed point cloud data;

[0123] Segmentation loss function for point cloud data segmentation tasks:

[0124]

[0125] Among them, t i p is the true label of the i-th point. i Let be the predicted probability of the i-th point;

[0126] The total loss function used to control the balance between reconstruction and segmentation tasks:

[0127] L=αL CD +βL Seg

[0128] Here, α and β represent hyperparameters.

[0129] The CD loss function, EMD loss function, and segmentation loss function for point cloud data segmentation tasks were designed. The design of these loss functions helps to consider different influencing factors in the point cloud data reconstruction process, such as the distance between point clouds, the correspondence between point clouds, and the segmentation of point clouds, thereby achieving efficient and accurate reconstruction of point cloud data.

[0130] This invention incorporates the structures of both the GCN encoder and the Transformer decoder during training. The GCN network extracts node features from the graph, which are then input into the Transformer decoder. A self-attention mechanism is used to encode the point cloud data, extracting key features and information, thus achieving efficient feature extraction and reconstruction of point cloud data. Furthermore, this invention utilizes an adjacency matrix to establish the graphical structure of the point cloud data, enabling efficient representation and processing of the data.

[0131] This invention can effectively solve some problems in point cloud data processing, such as large data volume, high complexity, and insufficient accuracy, and has high practical value. It adopts a point cloud segmentation algorithm and an object reconstruction algorithm based on the transform model, which have high accuracy and reliability. It can be used in technical research and development in related fields and has certain economic and social value.

[0132] Meanwhile, by combining GCN encoding and Transformer decoder structures, this method demonstrates significant innovation in point cloud data processing. It effectively addresses several problems in point cloud data processing, such as large data volume, high complexity, and insufficient accuracy, and provides new ideas and methods for technological research and development in related fields. Therefore, this invention is versatile and applicable to multiple application scenarios. It can be widely applied in various fields such as power engineering design, computer vision, autonomous driving, and robotics. In power engineering design, this method can be used to reconstruct facilities such as transmission lines from laser scanning data, facilitating power engineering planning and design. In the field of computer vision, this method can be applied to tasks such as target detection and object recognition of point cloud data, improving the accuracy and efficiency of computer vision systems. In the fields of autonomous driving and robotics, this method can be used to perceive the environment and perform scene analysis, realizing intelligent driving and autonomous navigation.

[0133] The GCN-Transformer model of this invention possesses unique data sparsity handling capabilities. Point cloud data often exhibits high data sparsity, meaning that most regions are empty, with only a few points containing useful information. The GCN-Transformer model cleverly handles sparse data within graph convolutional layers, achieving efficient extraction of useful information from the data. The model automatically identifies and focuses on important points while ignoring blank areas, thereby improving the efficiency and accuracy of processing sparse point cloud data.

[0134] Furthermore, the GCN-Transformer model fully considers the spatial relationships within point cloud data. Point cloud data is three-dimensional, and the spatial relationships between nodes are crucial. The model's graph convolutional layers and Transformer decoder utilize a self-attention mechanism to capture the complex spatial relationships between nodes in point cloud data. This enables the model not only to extract node features but also to understand the positions and interactions of nodes in three-dimensional space.

[0135] Example 2

[0136] like Figure 6As shown, an electronic device includes at least one processor, a memory communicatively connected to the at least one processor, and at least one input / output interface communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform a method described in the foregoing embodiments. The input / output interface may include a display, keyboard, mouse, and USB interface for inputting and outputting data.

[0137] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0138] When the integrated units of this invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A point cloud data reconstruction method based on the GCN-Transformer model, characterized in that, Includes the following steps: S1: Obtain the point cloud dataset; S2: Construct the GCN-Transformer model and optimize and train the GCN-Transformer model using the point cloud dataset; The GCN-Transformer model includes a GCN layer, a GCN encoder, and a Transformer decoder; The GCN layer is used to preprocess the point cloud data in the point cloud dataset and establish the adjacency matrix of the point cloud data. The GCN encoder is used to extract node features based on the adjacency matrix and output a node feature matrix; The Transformer decoder is used to encode the point cloud data using a self-attention mechanism based on the node feature matrix. The self-attention formula is as follows: Where H is the node feature matrix, W q W k W v Both are learnable parameter matrices, d k For feature dimensions; Feature vectors are extracted from the encoded point cloud data. The point cloud data is reconstructed based on the feature vector, including: mapping the feature vector to the coordinates of points using a linear activation function or a tanh activation function, thereby completing the reconstruction of the point cloud data; The expression for the linear activation function is: The expression for the Tanh activation function is: Among them, Y (m) Let be the feature vector of the m-th layer. For the reconstructed point cloud data; S3: Input the point cloud data to be processed into the trained GCN-Transformer model to complete the reconstruction of the point cloud data to be processed.

2. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 1, characterized in that, The GCN encoder consists of multiple graph convolutional layers; The Transformer decoder consists of multiple Transformer modules, each of which includes, in sequence, a feedforward network layer, a multi-head attention layer, another feedforward network layer, and a residual module.

3. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 2, characterized in that, The GCN layer described in step S2 is used to preprocess the point cloud data in the point cloud dataset. The establishment of the adjacency matrix of the point cloud data includes: the point cloud data is regarded as an undirected graph structure, where each point represents a node, and the distance connection relationship or proximity relationship between two nodes represents an edge. The GCN layer uses the undirected graph structure and the edges between nodes to process the point cloud data, and uses an adjacency matrix to represent the structure of the point cloud data. Each element of the adjacency matrix indicates whether there is an edge between two points.

4. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 3, characterized in that, The GCN encoder described in step S2 is used to extract node features based on the adjacency matrix and output a node feature matrix, including: The Laplacian matrix of the graph is constructed based on the adjacency matrix, and the formula for calculating the Laplacian matrix is ​​as follows: L=DA Where D is the degree matrix and A is the adjacency matrix, representing the connection between nodes; The coordinates of the point cloud data are used as node features, and the Laplacian matrix and the node features are used as inputs to the GCN encoder. Each graph convolutional layer aggregates the node features and the features of the Laplacian matrix to generate new node features. The expression for the node feature matrix is ​​as follows: H = GCN(X,A) Here, H is the node feature matrix, and the GCN function uses the adjacency matrix to encode the node features.

5. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 4, characterized in that, The GCN function contains multiple graph convolutional layers. These layers perform convolution operations on node features, aggregate information from neighboring nodes, and update node features. The function expression for each graph convolutional layer is as follows: Among them, H (l) H represents the feature matrix of the nodes in the l-th layer. (l+1) This represents the feature matrix of the nodes in the (l+1)th layer. I is a constant matrix. For degree matrix, For the activation matrix, These are the weight parameters of the current graph's convolutional layer.

6. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 1, characterized in that, Step S2 involves extracting feature vectors from the encoded point cloud data. The formula for calculating the feature vectors is as follows: y (m+1) =ReLU(Y (m) IN (m) ) Among them, Y (m+1) W is the extracted feature vector. (m) Let be the weight matrix of the m-th layer.

7. The point cloud data reconstruction method based on the GCN-Transformer model according to claim 1, characterized in that, The loss function in the GCN-Transformer model includes: The CD loss function is used to measure the bidirectional distance between two point clouds, that is, the distance from one point to the nearest point in another point cloud: Where N is the number of points in the point cloud data, x i Let y be the 3D coordinates of the i-th point in the original point cloud data. i To reconstruct the 3D coordinates of the i-th point in the point cloud data; The EMD loss function is used to measure the minimum cost required to transform one point cloud into another, which is the distance from each point to its corresponding point: Where φ is a bijective function that maps each point in the original point cloud data to a corresponding point in the reconstructed point cloud data; Segmentation loss function for point cloud data segmentation tasks: Among them, t i p is the true label of the i-th point. i Let be the predicted probability of the i-th point; The total loss function used to control the balance between reconstruction and segmentation tasks: L=αL CD +βL Seg Here, α and β represent hyperparameters.

8. An electronic device, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the point cloud data reconstruction method based on the GCN-Transformer model as described in any one of claims 1 to 7.

Images