Method and device for connecting skeleton points into skeleton, electronic equipment and storage medium

Abstract

The invention provides a method and device for connecting skeleton points into a skeleton, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target point cloud set of an object; extracting skeleton points based on the target point cloud set; wherein the point cloud features are extracted based on the target point cloud set, a group of skeleton balls covering the object are predicted based on the point cloud features, and the centers of the skeleton balls are extracted skeleton points; constructing an adjacent matrix of the skeleton points through a local neighborhood graph; inputting node features formed by the adjacency matrix, the skeleton sphere radiuses corresponding to the skeleton points and the point cloud features into an encoder module of the graph auto-encoder model, and compressing the node features into potential features by the encoder module based on the adjacency relation between the skeleton points in the adjacency matrix; a decoder module of the graph auto-encoder model reconstructs an adjacent matrix by using the potential features; and on the basis of the adjacency relation between the skeleton points represented by the adjacency matrix reconstructed by cosine similarity threshold optimization, obtaining the skeleton topology connection relation between the skeleton points.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of point cloud, and particularly relates to a method and device for connecting skeleton points into a skeleton, an electronic device and a storage medium. BACKGROUND

[0002] In the past few decades, many researchers have conducted extensive research on object skeleton extraction. At present, the research methods of skeleton extraction mainly include a method based on two-dimensional image data and a method based on three-dimensional point cloud data. However, two-dimensional image data lacks depth information and effective description of the occluded area of an object, which makes it difficult to accurately obtain the overall structure of the object in actual application, and greatly affects the authenticity of object skeleton extraction.

[0003] Most of the current deep learning methods mainly focus on the extraction of skeleton points. As for the problem of how to topologically connect the extracted skeleton points into a skeleton, most of them are based on manual connection, which is not conducive to high-throughput data automatic processing and analysis. Therefore, some scholars have conducted related research on the problem of realizing end-to-end automatic connection of skeletons. For example, an existing technology converts the problem of object skeleton extraction into the problems of corner point positioning and corner point connection. Another existing technology converts point cloud into a wireframe deep neural network WireframeNet. The network learns geometric transformation to predict a set of edge points from the original point cloud, and finally analyzes the connectivity between the edge points to construct a complete wireframe structure. The above existing technology simplifies the connection problem of skeleton points and edges into the connection problem of edge points and wireframes of polyhedrons, and the skeleton structure extracted by the algorithm is similar to the mesh grid shape, and does not have the original topological structure of the object. SUMMARY

[0004] The application aims to provide a method and device for connecting skeleton points into a skeleton, an electronic device and a storage medium.

[0005] According to a first aspect of the application, a method for connecting skeleton points into a skeleton is provided, comprising:

[0006] obtaining a target point cloud set of an object;

[0007] extracting skeleton points based on the target point cloud set; wherein in the extraction process of the skeleton points, point cloud features are extracted based on the target point cloud set, and a set of skeleton spheres covering the object is predicted based on the point cloud features, the center of the skeleton sphere being the extracted skeleton point;

[0008] constructing an adjacency matrix of the skeleton points in the form of a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points;

[0009] inputting the node features constituted by the adjacency matrix, the skeleton point corresponding skeleton ball radius and the point cloud feature to an encoder module of a graph autoencoder model, compressing the node features to latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and reconstructing the adjacency matrix by a decoder module of the graph autoencoder model using the latent features;

[0010] optimizing the adjacency relationship between the skeleton points represented by the reconstructed adjacency matrix based on a cosine similarity threshold value to obtain a skeleton topology connection relationship between the skeleton points.

[0011] Further, the method further comprises:

[0012] initializing a plurality of standard direction vectors uniformly covering all quadrants of a three-dimensional space;

[0013] traversing the candidate edges corresponding to the skeleton points to calculate the normalized direction vectors of the candidate edges, wherein the candidate edges are edges between two skeleton points having a connection relationship in the reconstructed adjacency matrix;

[0014] matching the normalized direction vectors of the candidate edges with the standard direction vectors to obtain the matching direction vectors of the candidate edges by using cosine similarity;

[0015] selecting target edges from the candidate edges based on the matching direction vectors corresponding to all candidate edges of the skeleton points and the cosine similarity between the normalized direction vectors of the candidate edges and the matching direction vectors, and obtaining the skeleton topology connection relationship represented by the skeleton points and the target edges.

[0016] Further, the method further comprises:

[0017] determining the number of matching direction vectors of all candidate edges corresponding to each skeleton point;

[0018] determining the skeleton point as a non-isolated point when the number is less than a screening threshold value;

[0019] selecting target edges from the candidate edges based on the matching direction vectors corresponding to all candidate edges of the skeleton points and the cosine similarity between the normalized direction vectors of the candidate edges and the matching direction vectors, comprising:

[0020] for each non-isolated point, determining the candidate edge as a target edge when the cosine similarity between the normalized direction vectors of the candidate edges and the matching direction vectors is greater than a similarity threshold value.

[0021] The second aspect of the application provides a method for training a graph autoencoder model for connecting skeleton points into a skeleton, comprising:

[0022] obtaining a target point cloud set of an object;

[0023] extracting skeleton points based on the target point cloud set; wherein in the process of extracting the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points;

[0024] constructing an adjacency matrix of the skeleton points in the form of a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points;

[0025] inputting the node features composed of the adjacency matrix, the skeleton sphere radii corresponding to the skeleton points, and the point cloud features into an encoder module of a graph autoencoder model, compressing the node features to latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and reconstructing the adjacency matrix by a decoder module of the graph autoencoder model using the latent features;

[0026] constructing a loss function based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

[0027] Further, the training process of the graph autoencoder model is supervised by introducing a masked balanced cross-entropy.

[0028] The third aspect of the application provides a device for connecting skeleton points into a skeleton, comprising:

[0029] a first obtaining module configured to obtain a target point cloud set of an object;

[0030] a first extracting module configured to extract skeleton points based on the target point cloud set; wherein in the process of extracting the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points;

[0031] a first constructing module configured to construct an adjacency matrix of the skeleton points in the form of a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points;

[0032] The first prediction module is configured to input node features composed of the adjacency matrix, the skeleton point corresponding skeleton ball radius and the point cloud feature into an encoder module of a graph autoencoder model, compress the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and reconstruct the adjacency matrix by a decoder module of the graph autoencoder model using the latent features.

[0033] The optimization module is configured to optimize the adjacency relationship between the skeleton points represented by the reconstructed adjacency matrix based on a cosine similarity threshold, and obtain a skeleton topology connection relationship between the skeleton points.

[0034] The fourth aspect of the present application provides a training device of a graph autoencoder model for connecting skeleton points into a skeleton, comprising:

[0035] The second acquisition module is configured to acquire a target point cloud set of an object.

[0036] The second extraction module is configured to extract skeleton points based on the target point cloud set; wherein in the extraction process of the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton balls covering the object are predicted based on the point cloud features, and the centers of the skeleton balls are the extracted skeleton points.

[0037] The second construction module is configured to construct an adjacency matrix of the skeleton points in a local neighborhood graph manner; the adjacency matrix represents the adjacency relationship between the skeleton points.

[0038] The second prediction module is configured to input node features composed of the adjacency matrix, the skeleton point corresponding skeleton ball radius and the point cloud feature into an encoder module of a graph autoencoder model, compress the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and reconstruct the adjacency matrix by a decoder module of the graph autoencoder model using the latent features.

[0039] The training module is configured to construct a loss function based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

[0040] The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0041] In a possible design, the apparatus includes a memory and a processor. The memory is configured to store one or more computer instructions for supporting the apparatus to perform the corresponding method. The processor is configured to execute the computer instructions stored in the memory. The apparatus can further include a communication interface configured to enable the apparatus to communicate with other devices or communication networks.

[0042] The fifth aspect of the present application provides an electronic device, including a processor and a memory, the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method of the first aspect.

[0043] The sixth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor and execute the method of the first aspect.

[0044] In the seventh aspect, the embodiments of the present application provide a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, the method in any of the preceding aspects is implemented.

[0045] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by means of the structures and processes particularly pointed out in the description and appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 FIG. 1 is a flow diagram of a method for connecting skeleton points into a skeleton according to an exemplary embodiment of the present application.

[0047] Figure 2 FIG. 2 is a structural diagram of a network model for skeleton point extraction and connecting skeleton points into a skeleton according to an exemplary embodiment of the present application.

[0048] Figure 3 FIG. 3 is a structural diagram of a skeleton point extraction model according to an exemplary embodiment of the present application.

[0049] Figures 4A-4B FIG. 4 is a diagram of a comparison between skeleton structures before and after connecting an optimization strategy based on a prediction result of a graph autoencoder according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0050] To solve the problems mentioned in the background, the present application proposes a scheme of unsupervised skeleton point topology connection to skeleton. In this scheme, the core goal of the point cloud skeleton extraction task is to obtain the skeleton points and their topology connection relationship representing the main structure of the object from the discrete point cloud data, and the object skeleton is essentially a single connected tree graph composed of points and edges, where the discrete point cloud is regarded as a point and the topology connection relationship between the skeleton points is regarded as an edge. According to this characteristic of the point cloud skeleton, after obtaining the skeleton points, the present application proposes a cosine similarity threshold screening strategy based on the direction vector on the basis of the graph auto encoder (GAE) network, which measures the cosine similarity between the edges connecting the skeleton points under the condition of a set threshold, realizes more reasonable edge connection, and effectively solves the problem of low application efficiency caused by the current manual connection of skeleton points. In summary, the unsupervised skeleton point topology connection algorithm of the graph neural network proposed by the present application can realize end-to-end object point cloud skeleton extraction, which is helpful for analyzing the three-dimensional morphological characteristics of the object and greatly reduces the labor cost.

[0051] Compared with the prior art, the present application solves the problem that after the current deep learning network extracts skeleton points from the point cloud, it needs to be manually connected to form a skeleton, and it is difficult to realize end-to-end learning. The present application proposes a connection prediction network with a graph auto encoder as the core, combines prior connection strategies and topology connection optimization strategies, learns and optimizes the potential skeleton connection relationship in the point cloud, realizes end-to-end learning from the extracted skeleton points to the skeleton, reduces the labor cost of skeleton point post-processing, and significantly improves the efficiency of skeleton extraction.

[0052] The object of the present application is to connect the extracted skeleton points to a skeleton with a certain topology. The method of connecting the skeleton in the prior art relies on Euclidean distance or heuristic rules, and the algorithm complexity is high, and the effect is poor when dealing with complex topology (such as ring, bifurcation, etc.). Since the skeleton is essentially a graph structure composed of points and edges, the present application proposes a graph auto encoder connection prediction network based on a connection optimization strategy. The network uses an encoder-decoder architecture to further predict the missing links by analyzing the correlation of the skeleton points in the latent space. And combining the reconstruction loss and the optimization connection strategy of the adjacency matrix, the adjacency matrix is optimized to improve its ability to predict the connection relationship between the skeleton points.

[0053] The scheme proposed by the present application is described below through specific embodiments.

[0054] As Figure 1As shown, the present application exemplarily proposes a method for connecting skeleton points to form a skeleton, comprising: step S101: obtaining a target point cloud set of an object; step S102: extracting skeleton points based on the target point cloud set; wherein, in the process of extracting the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points; step S103: constructing an adjacency matrix of the skeleton points in the form of a local neighborhood graph; the adjacency matrix representing the adjacency relationship between the skeleton points; step S104: inputting the node features composed of the adjacency matrix, the skeleton sphere radii corresponding to the skeleton points, and the point cloud features into an encoder module of a graph autoencoder model, compressing the node features to latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and reconstructing the adjacency matrix of the skeleton points by the decoder module of the graph autoencoder model through inner product measurement of the correlation of the latent features; step S105: optimizing the adjacency relationship between the skeleton points represented by the reconstructed adjacency matrix based on a cosine similarity threshold, and obtaining the skeleton topology connection relationship between the skeleton points.

[0055] In this embodiment, the target point cloud set of the object can be a point cloud set obtained after preprocessing the original object point cloud data. The preprocessing process can adopt a point cloud data cleaning strategy and a data downsampling strategy. The function of point cloud data cleaning is to eliminate outliers, repeated points and background noise interference in the original point cloud, improve the quality of the point cloud, and ensure the effectiveness and robustness of subsequent network input; the downsampling process sparsifies the point cloud through random sampling and other methods to reduce the computational complexity, while retaining the overall structure features of the object and ensuring the geometric consistency of subsequent skeleton point extraction.

[0056] The target point cloud set obtained after preprocessing can be predicted by a skeleton point extraction model. In this embodiment, the skeleton point extraction model can extract point cloud features based on the target point cloud set, predict a group of skeleton spheres covering the object based on the point cloud features, and then obtain skeleton points based on the skeleton spheres. All forms of skeleton point extraction models conforming to the above process are suitable for the scheme of the present application.

[0057] In order to more clearly understand the skeleton point extraction process, the scheme of the present application will be introduced below in combination with a skeleton point extraction model suitable for the present application.

[0058] In this embodiment, as shown in FIG. 2, the skeleton point extraction model can be a neural network model, which can be trained by a training set composed of a plurality of object point cloud sets and the corresponding skeleton point sets. Figure 2As shown, the skeleton point extraction model can include a feature extraction encoder module and a convex combination transformation module. The feature extraction encoder module mainly includes a dynamic graph convolution layer and a multilayer perceptron layer. In the dynamic graph convolution layer, a graph structure is reconstructed in each forward propagation process according to the extracted point features. Through the construction of a dynamic graph neighborhood, more rich geometric relationships and structural features can be captured, providing more effective features for subsequent skeleton point classification. The multilayer perceptron includes multiple convolution layers, respectively an input layer, at least one intermediate layer and an output layer. The intermediate layer is used to extract features by performing convolution operations on the data input by the input layer. The multilayer perceptron layer is mainly responsible for further nonlinear mapping based on the local features extracted by the dynamic graph convolution layer. The convex combination transformation module uses the potential sparse structural features in the point cloud to find a better linear combination center in the local neighborhood of the original point cloud so that the network focuses on the key points with more obvious features. Finally, the network realizes the regression prediction from the preprocessed point cloud to the skeleton point through the constraint of the multi-dimensional unsupervised loss function.

[0059] The process of feature extraction by the dynamic graph convolution-based feature extraction encoder module is described as follows.

[0060] The core goal of the point cloud skeleton extraction task is to obtain skeleton points representing the main structure of an object and the connection relationship thereof from discrete point cloud data. The object skeleton is essentially a single-connected tree graph composed of points and edges, in which the discrete point cloud is regarded as a point and the topological connection relationship between the skeleton points is regarded as an edge. According to this characteristic of the point cloud skeleton, the graph neural network can aggregate neighborhood information through the message passing mechanism to extract point feature information by constructing a graph structure on the discrete points.

[0061] In some embodiments, the feature extraction encoder module uses a dynamic graph convolution layer to perform feature extraction on the target point cloud set and outputs the extracted point cloud features and the corresponding convex combination input points, including:

[0062] The feature extraction editor module includes multiple dynamic graph convolution layers connected in sequence. Each dynamic graph convolution layer constructs a local neighborhood graph for the input points and performs a convolution operation on the edges connecting adjacent points based on the local neighborhood graph. The input points of the first dynamic graph convolution layer are the target points in the target point cloud set, the input points of the other dynamic graph convolution layers are the outputs of the previous dynamic graph convolution layer, and the output of the last dynamic graph convolution layer includes the point cloud features and the convex combination input points.

[0063] In the feature encoder module based on the dynamic graph convolution layer, it is assumed that the input F-dimensional point cloud set (which can be a target point cloud set obtained after preprocessing of object point cloud data in embodiments of the present application) is represented as where pi represents the i-th point of the point cloud, N is the total number of points of the input point cloud, and considering the simplest case, when F = 3, each point only contains 3D coordinates p i Before performing the graph convolution operation, it is necessary to construct a local neighborhood graph for the point cloud. Therefore, it is further assumed that a directed graph G = (V, E) is used to represent the local structure of the point cloud, where represent vertices and edges, respectively. In the present application, the initial point cloud P, i.e., the object point cloud data, is first constructed into a k-neighborhood graph G using a k-neighborhood algorithm, and edge features e ij are extracted through a graph convolution operation, and finally vertex feature outputs p' are obtained by applying a channel-wise symmetric aggregation operation on the edge features related to all edges emitted by each vertex i .

[0064] where the k-neighborhood graph G is described by an adjacency matrix of N x N to describe the connection of all points. For each point in the target point cloud set , the k-neighborhood relationship is first calculated, and then the corresponding adjacency matrix can be obtained according to the k-neighborhood relationship. Specifically, for each point in the target point cloud set, a neighborhood parameter k is first set and an adjacency matrix A initialized to all 0 is set, which has a dimension of N x N. Then, the spatial distance between the point and all other points in the point cloud set P is calculated, and the first k points with the closest distance are selected, considering that the point has a connection relationship with the k points with the closest distance, so that the positions corresponding to the point and the k points with the closest distance in the adjacency matrix are set to 1. Therefore, in the adjacency matrix, assuming that the i-th row represents the i-th point and the j-th column represents the j-th point, if the number corresponding to the i-th row and the j-th column is 1, it means that the i-th point and the j-th point have a connection relationship.

[0065] The process of extracting edge features using a dynamic graph convolution layer is as follows: the target point cloud set and the neighborhood relationship of the points (i.e., the relationship between whether any two points are connected in the adjacency matrix) are input into the dynamic graph convolution layer, and edge features are obtained in the middle of the convolution, where the edge features are obtained by the coordinates p i of the point and the relative displacement vector p j -p i of the point through convolution, and finally the point cloud features corresponding to each point are obtained by summing and pooling all edge features of the point, which are used for the multi-layer perceptron. It should be noted that each dynamic graph convolution layer performs a sampling operation on the input point cloud, so the point cloud corresponding to the point cloud features output by each dynamic graph convolution layer is a part of the point cloud after sampling of the input point cloud, that is, the final output of the multiple dynamic graph convolution layers in the feature extraction encoder module includes part of the point cloud in the target point cloud set and the point cloud features corresponding to the part of the point cloud. In order to facilitate the distinction, the part of the point cloud in the final output of the multiple dynamic graph convolution layers in the present application is referred to as the convex combination input point.

[0066] The above edge feature e ij And the point feature p' i As shown in formula (1) and formula (2).

[0067] e ij = f (p i , p ji ) = f (p i , p j -p i ) (1)

[0068]

[0069] In formula (1), the edge feature function is an asymmetric edge function, which combines global shape structure (captured by the coordinates of the patch center p i ) and local neighborhood information (captured by p j -p i ), and f represents the convolution operation of the graph neural network; p' i represents part of the point cloud in the target point cloud set output by the dynamic graph convolution layer after the edge feature is aggregated, that is, the convex combination input point.

[0070] In addition, in order to further improve the expression ability of the network to the point cloud features, the application introduces a multi-layer feature fusion strategy in the skeleton point extraction module to fully integrate the local and global information extracted by different deep network layers. Specifically, the application exemplarily selects the output features of the first layer and the fourth layer of the dynamic graph convolution layer in the feature encoder, which respectively represent the shallow local structure information and the deep abstract semantic information of the point cloud. In order to facilitate the distinction, the output feature of the first layer of the dynamic graph convolution layer is called shallow semantic feature, and the output feature of the fourth layer of the dynamic graph convolution layer is called deep semantic feature. Of course, it can be understood that in other embodiments, the output feature of other shallower layer of the dynamic graph convolution layer can be selected as the shallow semantic feature, and the output feature of other deeper layer of the dynamic graph convolution layer can be selected as the deep semantic feature, and the application does not make specific limitation thereto. Then, after splicing the output features of the two layers in the channel dimension, the feature space alignment convolution layer is convolved to compress and adjust the feature expression dimension, and then fused with the output of the multi-layer perceptron layer, so that the obtained feature has stronger representation ability and semantic consistency than the feature before fusion. Through this structured multi-layer feature fusion manner, the model can capture both the detailed features and the global morphological features of the point cloud, significantly enhancing the expression ability and generalization ability of the skeleton point extraction module to complex structures. Especially when facing objects with large inter-class deformation or significant local structure change, the model can still maintain high robustness and prediction accuracy.

[0071] As described above, the point cloud features output by the plurality of dynamic graph convolution layers are further input into the multi-layer perceptron layer of the feature extraction encoder module for nonlinear fitting processing. Based on the processing result obtained through the nonlinear fitting processing, the multi-layer fusion features can be further obtained according to the specific implementation to be introduced below.

[0072] In some embodiments, the nonlinear fitting processing of the point cloud features by the multi-layer perceptron layer of the feature extraction encoder module and the multi-layer fusion features based on the processing result of the nonlinear fitting processing include that: the multi-layer fusion perceptron layer includes a plurality of fusion perception layers connected in sequence, each fusion perception layer performs convolution operation on input data, the input data of the first fusion perception layer is the point cloud features, the input data of other fusion perception layers is the output of the previous fusion perception layer, and the output of the last fusion perception layer is the processing result; the deep semantic features and the shallow semantic features output by the two non-adjacent dynamic graph convolution layers selected from the plurality of dynamic graph convolution layers are spliced and then subjected to convolution operation, and the convolution result is spliced with the processing result to obtain the multi-layer fusion features.

[0073] Figure 2 The multi-layer fusion perceptron shown includes four multi-layer fusion perception layers. It can be understood that the multi-layer fusion perceptron is not limited to four fusion perception layers, and can be set according to actual needs, which is not specifically limited in the present application.

[0074] The feature extraction encoder module further selects two non-adjacent dynamic graph convolution layers from the plurality of dynamic graph convolution layers, for example, a relatively shallow dynamic graph convolution layer and a relatively deep dynamic graph convolution layer, splices the output features of the two layers, and then performs convolution operation through the feature space alignment convolution layer, and splices the output of the last fusion perception layer. The spliced features are referred to as multi-layer fusion features. The multi-layer fusion features and the points output by the last dynamic graph convolution layer, i.e., the convex combination input points, are input into the convex combination transformation module together, so as to predict the skeleton points.

[0075] The process of predicting the skeleton points by the convex combination transformation module will be introduced below through the description of the unsupervised loss calculation function adopted by the embodiments of the present application.

[0076] In the unsupervised skeleton point extraction task, how to guide the network to accurately learn the skeleton structure in the point cloud under the premise of lacking manual annotation information is the core difficulty in loss function design. The existing technology usually constructs an unsupervised loss function from two aspects of geometric distribution constraint and structure consistency constraint: the former preserves geometric accuracy by measuring the spatial fitting degree between the skeleton points and the original point cloud, and the latter strengthens the coherence and stability of the skeleton points by introducing the sampling consistency and radius regularization term. Although these two constraints improve the accuracy of skeleton extraction to a certain extent, in the actual skeleton point data, there are often problems such as dense distribution of skeleton points, redundant structure or too much aggregation of regions, which leads to insufficient overall discriminability and clearness of expression of the skeleton representation. To further solve the above problems, the present application introduces a third key constraint dimension-semantics sparsity constraint based on the original loss function design, and further proposes an optimized unsupervised loss function with multi-dimensional feature fusion. The constraint regulates the global distribution of the skeleton points by constructing a uniform loss term, encouraging the skeleton points to present a more sparse, more uniform and more discriminative distribution pattern in the structure semantics, thereby significantly enhancing the semantic clarity and structure expression ability of the skeleton point extraction.

[0077] In some optional embodiments, the unsupervised loss calculation function comprises: calculating a point-to-sphere loss in the geometric distribution constraint level by using the convex combination input points and the predicted skeleton points, the point-to-sphere loss being a loss of the coordinates of the predicted skeleton points to the distance of the predicted skeleton points to the corresponding skeleton sphere of the predicted skeleton points, the skeleton sphere being a sphere with the predicted skeleton point as the center and the nearest distance from the predicted skeleton point to the convex combination input point corresponding to the predicted skeleton point as the radius; calculating a sampling consistency loss in the structure consistency constraint level by using the convex combination input points and the predicted skeleton points, the sampling consistency loss being a loss of constraining the sampling points on the surface of the skeleton sphere to be inside the set composed of the convex combination input points; calculating a radius regularization loss in the structure consistency constraint level by using the convex combination input points and the predicted skeleton points, the radius regularization loss being a loss of the radius of the skeleton sphere corresponding to the predicted skeleton point; and calculating a uniformity loss based on the semantics sparsity constraint by using the convex combination input points and the predicted skeleton points, the uniformity loss being a loss of the aggregation degree of the predicted skeleton points.

[0078] Next, for the unsupervised loss calculation function, the calculation method, constraint target and role in the overall optimization process of the loss term designed from each level are introduced in turn.

[0079] Firstly, based on the given input point cloud P (the convex combination input points are used by the present application in the subsequent prediction of skeleton points), the goal of the present application is to predict M skeleton points, in order to better cover the object shape by the predicted skeleton points, the present application predicts a series of skeleton sphere sets S before predicting the skeleton pointsi The set can better cover the shape of the object, and the set of centers of the skeleton spheres is regarded as the set of skeleton points C i In addition, generally, the skeleton points are regarded as the center points generated inside the local shape of the object in the point cloud. In some embodiments, the convex combination is constructed by using the points p' output after the feature extraction encoder module to predict the generated skeleton points c.

[0080]

[0081] wherein c represents the generated skeleton point, Weight T represents the network weight, and the Weight T is actually a multi-layer fusion feature represented by the model parameters of the skeleton point extraction model; p' i represents the i-th convex combination input point output by the feature extraction encoder module.

[0082] In addition, the radius R of the skeleton sphere corresponding to all the skeleton points is represented as a linear combination of the nearest distances between all the skeleton points c and the input points p' in the present application, that is, R = Weight T D. Wherein D represents a set of the nearest distances d from the input point cloud p' to the output skeleton points C, and the specific calculation formula is as follows:

[0083]

[0084] In an embodiment, the present application uses a multi-layer perceptron and a multi-classification network layer softmax layer to generate the model weight Weight T . According to the definitions of the skeleton sphere and the skeleton point described above, the present application trains the network by designing a set of loss functions to obtain the model weight Weight T .

[0085] First, the loss on the geometric distribution constraint level is the point-to-sphere loss. Since the step of predicting the skeleton point in the present application is to first predict a set of skeleton spheres capable of representing the shape of the object from the point cloud, and then estimate the final skeleton point according to the center points of the skeleton spheres, the present application first measures the skeleton sphere reconstruction error by optimizing the coordinates of the skeleton points and their radii explicitly:

[0086]

[0087] wherein L point2sphere represents the point-to-sphere loss, c i represents the predicted skeleton point, p i ' represents the input point, which refers to the i-th convex combination input point in the present application; represents the radius of the skeleton sphere corresponding to the skeleton point closest to the convex combination input point p′, r(c) represents the radius of the skeleton sphere corresponding to the skeleton point c, represents the skeleton point closest to the convex combination input point p′, and represents the minimum Euclidean distance between the skeleton point c and the convex combination input point p′. The first term constrains each convex combination input point to lie on the surface of its nearest skeleton sphere, and the second term encourages each skeleton sphere to touch its nearest convex combination input point.

[0088] Then comes the loss at the level of the structural consistency constraint: the sampling consistency loss. Only through the loss function L point2sphere may make the structure of the skeleton points cover the input point cloud, but it does not mean that the generated skeleton points can restore the original surface shape in geometry, i.e. there may be a situation where the center position and the radius of the skeleton point make the ball cover the point cloud, but the skeleton point is greatly different on the geometric surface. Therefore, the present application generates sampling points d i for representing the "approximate surface" of the object, and measuring the Chamfer Distance (CD) between the sampling points d i and the convex combination input point p i ′, for constraining the surface generated by the skeleton to also be consistent with the details of the point cloud geometry:

[0089]

[0090] where L sample is the sampling consistency loss, the sampling points d i are generated by the intersection of the spherical surface of the skeleton sphere with 8 directional vectors in the three-dimensional Cartesian coordinate system; and represents the Euclidean distance between the sampling point d and the convex combination input point p i ′.

[0091] After that comes the radius regularization loss. In practical applications, the present inventors have found that the inscribed sphere of a three-dimensional shape is sensitive to surface noise, i.e. a sphere can be inscribed in the space between several noise points, thereby producing a small radius. The purpose of the point-to-sphere loss described above is to predict a set of skeleton spheres that can represent the shape of the covered object, but there is no requirement for the radius of the skeleton sphere, which makes it possible for the center of a skeleton sphere to be used to represent the center of multiple surrounding skeleton points. In fact, more than one center of the skeleton sphere is needed to represent the multiple skeleton points, and there may also be an incomplete coverage, causing the skeleton point extraction to deviate. Therefore, the radius regularization loss encourages a larger radius by constraining the size of the radius, thereby solving the above problem. In order to focus on the basic geometry in order to better abstract and avoid instability, therefore, the present application uses the radius regularization loss Lradius To encourage greater skeleton ball radius, reduce the impact of fine noise points in the point cloud:

[0092]

[0093] Finally, the uniformity loss based on the semantic sparsity constraint. Through the joint constraint of the above loss functions, the network can estimate the position of the skeleton points from the original point cloud. However, the generated skeleton points will appear regional aggregation, which will affect the subsequent skeleton connection effect. The present application also adds a uniformity loss based on the semantic sparsity constraint to constrain the uniformity of the generated skeleton points.

[0094] First, according to the extracted skeleton points c and the corresponding ball radius r, each skeleton point is traversed to determine whether there is a redundant skeleton point around it. If there is a redundant skeleton point, the number of skeleton points in its redundant range is calculated, and the corresponding uniformity loss is calculated. Specifically, for each skeleton point c i , the present application searches for other skeleton points in the neighborhood ball with a pre-set neighborhood radius l i as the radius. If ||c j -c i ||2<l i , the redundancy in the neighborhood is increased by one. When the redundancy exceeds the threshold setting, the skeleton point region in the region is recorded as redundant. Therefore, the present application defines the redundant neighbor set as:

[0095]

[0096] Wherein, When l i = 0.5 (an empirically set threshold), it is considered that the region of the point has excessive redundancy. Next, the uniformity loss calculation needs to be performed on the region to optimize the position distribution of the skeleton points.

[0097] x i+1 = |c i -c i |2, i = 1,..., n-1

[0098]

[0099] Wherein, c 3 ∈ R i represents the generated skeleton point set; x uniform represents the Euclidean distance between the i-th skeleton point and the adjacent skeleton point i+1 in the region with excessive redundancy; represents the average value of the Euclidean distance between all adjacent skeleton points, n represents the number of skeleton points in the region with excessive redundancy, and L uniform represents the uniformity loss calculation result.

[0100] In summary, the overall loss function L of the skeleton point extraction module part skeleton-point may be expressed as:

[0101] L skeleton-point = λ1L point2sphere + λ2L sample + λ3L radius + λ4L uniform

[0102] wherein λ1, λ2, λ3, λ4 respectively represent the weight proportion of L point2sphere , L sample , L radius , L uniform loss function in the training process.

[0103] The process of connecting the skeleton points into a skeleton proposed in the present application is described below through an embodiment of a graph autoencoder model.

[0104] In this embodiment, when connecting and predicting the skeleton points based on the graph autoencoder model, a local neighborhood graph needs to be constructed first to extract the feature representation of its high-dimensional latent space by the network, and to mine the inherent structure and mapping relationship inside the data. Therefore, the following two graph neighborhood building strategies are adopted for graph initialization.

[0105] (1) k-neighbor strategy, after the skeleton points are extracted by the skeleton point extraction model, the k-neighbor algorithm is applied to the skeleton points, the k nearest nodes adjacent to the current skeleton point are calculated, and the corresponding edges between the current skeleton point and the k nodes in the adjacency matrix formed by all skeleton points are set to 1 (indicating connection). Wherein, k = 1 is taken in the present application, which means that each node only has a connection with the nearest node.

[0106] wherein the adjacency matrix A can be obtained as follows:

[0107] For each skeleton point, first set the neighborhood parameter k and an adjacency matrix A initialized as all 0, which has a dimension of N x N. Then calculate the spatial distance between all other skeleton points, and select the first k nearest points, then consider that the skeleton point has a corresponding connection with the k nearest points, and thus set the corresponding position in the adjacency matrix to 1.

[0108] (2) restoration priority strategy, given a point p in a target point cloud set and the two nearest skeleton points c1, c2 to it, a link between the skeleton points c1, c2 is formed, this strategy generates the nearest two skeleton points near the original point cloud by connection, which is equivalent to providing connection prior knowledge based on the shape of the original point cloud, and this strategy ensures that the constructed local neighborhood graph conforms to the actual topological direction of the object.

[0109] Based on the above two graph initialization strategies, the application can obtain an initialized graph with part of the connected links, and subsequent tasks determine the links that may exist in the graph and make the overall links conform to the shape structure of the object to obtain the final object skeleton.

[0110] In some optional embodiments, based on the cosine similarity threshold, the adjacency relationship between the skeleton points represented by the reconstructed adjacency matrix is optimized to obtain the skeleton topology connection relationship between the skeleton points, which includes: initializing a plurality of standard direction vectors uniformly covering all quadrants in a three-dimensional space; traversing the candidate edges corresponding to the skeleton points, and calculating the normalized direction vector of the candidate edge; wherein the candidate edge is an edge between two skeleton points having a connection relationship in the reconstructed adjacency matrix; by cosine similarity, the normalized direction vector of the candidate edge is matched with the standard direction vector to obtain the matching direction vector of the candidate edge; based on the matching direction vectors corresponding to all candidate edges of the skeleton points, and the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector, the target edge is selected from the candidate edge, and the skeleton topology connection relationship represented by the skeleton points and the target edge is obtained.

[0111] In this embodiment, after constructing a local neighborhood graph for the extracted skeleton points, the application obtains a series of discrete undirected and weightless initialization graphs corresponding to all skeleton points Wherein N skeleton points extracted from the object point cloud data, ε represents the set of edges, and further An adjacency matrix A∈{0,1} N×N Encode to represent the connectivity of the edge in the graph structure.

[0112] The network architecture of the graph autoencoder is as shown in Figure 3 , wherein A represents the adjacency matrix of the initialization graph, X is the node feature matrix, and Z represents the latent feature representation, , the solid line in the adjacency matrix A and the reconstructed adjacency matrix represents the initialization connection, and the dashed line represents the predicted connection. It can be seen that, in addition to the adjacency matrix A, the node feature information is also needed in the input data. In order to effectively utilize the geometric correlation between the skeleton and the input point cloud, a multi-source fusion method is adopted in the application to establish the node feature, that is, the node feature X is represented by the skeleton point coordinates C, the skeleton ball radius r and the feature output F of the dynamic graph convolution, that is, the node feature X=[C,r,F]; wherein the skeleton ball radius is determined when the skeleton point is extracted. In some embodiments, in the process of extracting the skeleton point, the point cloud feature is extracted based on the target point cloud set, and a group of skeleton balls covering the object is predicted based on the point cloud feature, and the center of the skeleton ball is the extracted skeleton point.

[0113] The encoder module of the GAE (Graph Autoencode) includes a multi-layer residual graph convolution structure, node features are taken as initial inputs of the encoder module, the encoder module gradually fuses topology and attribute information through multi-layer convolution, an adjacency matrix A controls information propagation, which is used to determine which node features need to be aggregated, and finally the encoder module compresses the node features into latent features Z, that is, the encoder module extracts latent features Z from node features X according to the adjacency matrix A. In some embodiments, each convolution layer of the encoder module performs convolution operation on the adjacency matrix as a convolution kernel and the corresponding input, and then obtains the output of the convolution layer. The input of the first convolution layer is the node feature, the input of the other convolution layers is the output of the previous convolution layer, and the output of the last convolution layer is the latent feature Z.

[0114] The decoder module of the GAE reconstructs a new adjacency matrix by measuring the correlation of the latent features Z in a vector inner product manner That is, the decoder performs inner product on the latent features Z, and the greater the inner product value is, the more similar the embedding of the node features is, and the higher the probability of topological connection is. As shown in the formula:

[0115] Z=GCN(X,A)

[0116] Wherein, GCN represents the graph convolution operation of the graph convolutional neural network.

[0117] When training the GAE network model, the target of the present application is to make the reconstructed adjacency matrix be able to predict the unknown links in the initialized adjacency matrix A. Therefore, the present application introduces a masked balanced cross-entropy to supervise the GAE learning:

[0118]

[0119] Wherein, represents a mask matrix, L link represents that the loss is only calculated between the specific candidate edge set; ⊙ element-wise product, which ensures that only the items within the mask range participate in the loss calculation; ξ∈R + represents the positive and negative sample weight coefficient, which is used to balance the proportion of training samples between the existence and nonexistence of edges; A∈0,1 N×N represents the input adjacency matrix; represents the adjacency matrix output by the model prediction; σ(·) represents an activation function.

[0120] That is, in the model training process of the GAE, the reconstructed adjacency matrix Afterwards, the model parameters of the encoding module and the decoding module of the GAE are trained by constructing the loss function described above. After the model training is completed, the model parameters of the encoding module and the decoding module can be directly used to reconstruct the adjacency matrix in the prediction process Afterwards, the reconstructed adjacency matrix is used to obtain the topological connection relationship between the skeleton points.

[0121] In this embodiment, in order to obtain a more reasonable and natural connection of the skeleton points into a skeleton, the connection relationship represented by the reconstructed adjacency matrix is optimized, and the details are described as follows.

[0122] In some optional embodiments, the method further comprises: determining the number of matching direction vectors of all candidate edges corresponding to each skeleton point; determining the skeleton point as a non-isolated point if the number is less than a screening threshold; and screening a target edge from the candidate edges based on the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector of the candidate edge, comprising: for each non-isolated point, determining the candidate edge as a target edge when the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector is greater than a similarity threshold.

[0123] As described above, the application uses a GAE network to perform connection prediction on an input initial graph. However, when using a graph auto-encoder network to output a skeleton topological connection, there are many wrong connections and self-loop phenomena. Such a connection structure is unreasonable and unnatural in the actual object skeleton structure. In order to make the generated skeleton conform to the topological structure of a normal object, it is usually necessary to manually correct the wrong connection edges according to the original shape of the object. However, in actual application scenarios, the shape differences of different categories, or even different objects of the same category, are very obvious, and the workload of manual correction one by one is huge, which does not conform to the high-throughput automatic processing and analysis of data.

[0124] In order to solve the above connection error problem, the application designs a connection optimization connection strategy based on cosine similarity threshold judgment, and screens out unreasonable connections by matching the cosine similarity between the direction distribution characteristics of the edges in the space and the standard direction vector. This method can effectively eliminate non-topologically consistent connection edges without relying on manual intervention, and retain the connection relationship consistent with the direction of the real skeleton structure, thereby improving the topological naturalness and accuracy of the skeleton structure.

[0125] A point cloud is a set of discrete points distributed in a three-dimensional space to represent the shape of an object, and any point in a three-dimensional Cartesian coordinate system can be represented by a combination of several unrelated vectors. Therefore, the core idea of the present application is: first, define eight unrelated standard direction vectors in a three-dimensional space with the origin of the coordinate system as the center, Directions=[±e,±e,±e], where e is a unit vector component. Then, calculate the normalized direction vector of each connected edge, i.e. candidate edge, and calculate the cosine similarity between it and the above eight direction vectors, and then judge the spatial inclination of the edge. For each skeleton point, count the number of categories of the normalized direction vectors corresponding to the connected edges (i.e. the number of categories of the standard direction vectors matched with the normalized direction vectors, for convenience of description, the standard direction vectors matched with the normalized direction vectors are referred to as matched direction vectors), and judge the rationality of the topological structure accordingly. When the matched direction vectors corresponding to a skeleton point are too discrete, it indicates that the connection structure of the point deviates from the natural skeleton direction feature, and should be considered as redundant connection or abnormal structure.

[0126] In some embodiments, the core process of the optimization strategy can be divided into three stages, which are described as follows:

[0127] First, initialize the eight standard direction vectors of the three-dimensional space, which uniformly cover all quadrants of the three-dimensional space as the basis for subsequent direction matching. At the same time, create an empty set to store the screening results, and initialize an empty set for each skeleton point to record the connection direction.

[0128] Next, traverse each candidate edge corresponding to each skeleton point, which refers to the edge between two skeleton points representing an adjacency relationship in the reconstructed adjacency matrix. Calculate the actual direction vector of each candidate edge and perform normalization processing. By cosine similarity, compare the normalized direction vector of the candidate edge with all pre-defined standard direction vectors, find the standard direction vector with the highest matching degree, and take the standard direction vector with the highest matching degree as the matching direction vector of the candidate edge, while recording the number of matching direction vectors corresponding to the index candidate edge for each skeleton point.

[0129] Finally, threshold screening is performed for each skeleton point, i.e. taking the skeleton point with a number of matching direction vectors greater than or equal to the edge screening threshold as an isolated point, thereby eliminating the skeleton point and the candidate edge corresponding to the skeleton point. This means that only those points with a number of matching direction vectors less than the edge screening threshold will be retained, and this step ensures the sufficiency of the connection of the skeleton points and avoids redundant connections or abnormal structures of the skeleton points.

[0130] The specific implementation process of the optimization strategy is shown in Algorithm 1.

[0131]

[0132]

[0133] The strategy fully considers the directional characteristics of the connection of the three-dimensional object skeleton. For an object with a long-axis structure (such as an animal torso, a tool rod body, etc.), the connection edge usually extends along the main axis direction, and therefore the correct connection relationship should be concentrated in a small number of spatial directions. In contrast, the connection direction of a point that is too dispersed is usually due to a connection error or a predicted redundancy, and through the direction vector screening mechanism, these unreasonable connections can be automatically identified and removed, thereby improving the structural expression ability and semantic consistency of the overall skeleton. Figures 4A-4B is a schematic diagram of the comparison of the skeleton structure before and after the connection optimization strategy based on the prediction result of the graph autoencoder according to the example embodiments of the present application. As Figure 4A shows that the edge connection result predicted by the GAE network has a large number of redundant edge connections and self-loop structures; Figure 4B shows the result optimized by the method of the present application, the connection structure is clear, most of the incorrect and redundant edge connections are removed, and the method has the ability to remove redundant points, significantly improving the topological rationality and shape expression ability of the skeleton.

[0134] Correspondingly, the second aspect of the present application provides a training method of a graph autoencoder model, characterized in that the graph autoencoder model is used to connect skeleton points into a skeleton, comprising: obtaining a target point cloud set of an object; extracting skeleton points based on the target point cloud set; wherein in the extraction process of the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points; an adjacency matrix of the skeleton points is constructed in a local neighborhood graph manner; the adjacency matrix represents the adjacency relationship between the skeleton points; the node features composed of the adjacency matrix, the skeleton sphere radii corresponding to the skeleton points and the point cloud features are input into an encoder module of the graph autoencoder model, and the node features are compressed into latent features by the encoder module based on the adjacency relationship between the skeleton points in the adjacency matrix, and a decoder module of the graph autoencoder model reconstructs the adjacency matrix using the latent features.

[0135] A loss function is constructed based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

[0136] In the method, the training process of the graph autoencoder model is supervised by introducing a masked balanced cross-entropy.

[0137] The construction of the loss function, the training of the model and other details in the method are described above in the method of connecting the skeleton points into a skeleton, and specific reference can be made to the description of the method of connecting the skeleton points into a skeleton, which will not be repeated here.

[0138] The present application also provides a device for connecting skeleton points into a skeleton in a third aspect, comprising: a first acquisition module configured to acquire a target point cloud set of an object; a first extraction module configured to extract skeleton points based on the target point cloud set; wherein in the extraction process of the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points; a first construction module configured to construct an adjacency matrix of the skeleton points in a local neighborhood graph manner; the adjacency matrix representing the adjacency relationship between the skeleton points; a first prediction module configured to input the node features composed of the adjacency matrix, the skeleton sphere radius corresponding to the skeleton points and the point cloud features into the encoder module of the graph autoencoder model, and based on the adjacency relationship between the skeleton points in the adjacency matrix, the encoder module compresses the node features into latent features, and the decoder module of the graph autoencoder model reconstructs the adjacency matrix using the latent features; an optimization module configured to optimize the adjacency relationship between the skeleton points represented by the reconstructed adjacency matrix based on a cosine similarity threshold, and obtain the skeleton topology connection relationship between the skeleton points.

[0139] The above device corresponds to the method of connecting the skeleton points into a skeleton described above, and specific details can be referred to the description of the training method of the graph autoencoder model above, which will not be repeated here.

[0140] The application also provides a device for training a graph autoencoder model in a fourth aspect, the graph autoencoder model being used to connect skeleton points into a skeleton, comprising: a second acquisition module configured to acquire a target point cloud set of an object; a second extraction module configured to extract skeleton points based on the target point cloud set; wherein in the extraction process of the skeleton points, point cloud features are extracted based on the target point cloud set, and a group of skeleton spheres covering the object are predicted based on the point cloud features, the centers of the skeleton spheres being the extracted skeleton points; a second construction module configured to construct an adjacency matrix of the skeleton points in a local neighborhood graph manner; the adjacency matrix representing the adjacency relationship between the skeleton points; a second prediction module configured to input node features constituted by the adjacency matrix, the skeleton sphere radii corresponding to the skeleton points, and the point cloud features into an encoder module of a graph autoencoder model, and compress the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix by the encoder module, and a decoder module of the graph autoencoder model reconstructs the adjacency matrix by using the latent features; and a training module configured to construct a loss function based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

[0141] The device described above corresponds to the training method of the graph autoencoder model described above, and specific details can be referred to the description of the training method of the graph autoencoder model above, which will not be described here.

[0142] It can be understood that the circuit structure, name and parameter described in the above embodiments are only examples. Those skilled in the art can also easily combine and adjust the structural features of the above multiple embodiments according to the needs of use, and the concept of the application should not be limited to the specific details of the above examples.

[0143] The application also provides an electronic device, comprising a processor and a memory, the memory storing a plurality of instructions, and the processor being configured to read the instructions and perform any one of the methods in the first aspect.

[0144] As still another aspect, the present disclosure also provides a computer program product, which contains computer instructions that can be loaded and executed by the processor to enable the processor to perform the methods described in Embodiment I and Embodiment II.

[0145] Although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A method for connecting skeleton points to form a skeleton, characterized in that, include: Obtain the target point cloud set of the object; Skeleton points are extracted based on the target point cloud set; wherein, during the extraction of skeleton points, point cloud features are extracted based on the target point cloud set, and a set of skeleton spheres covering the object are predicted based on the point cloud features, and the center of the skeleton sphere is the extracted skeleton point; The adjacency matrix of the skeleton points is constructed using a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points. The node features, consisting of the adjacency matrix, the skeleton sphere radius corresponding to the skeleton point, and the point cloud features, are input to the encoder module of the graph autoencoder model. The encoder module compresses the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix. The decoder module of the graph autoencoder model then reconstructs the adjacency matrix using the latent features. The adjacency relationships between the skeleton points are obtained by optimizing and reconstructing the adjacency matrix based on the cosine similarity threshold, thereby obtaining the skeleton topological connection relationships between the skeleton points.

2. The method for connecting skeleton points to form a skeleton according to claim 1, characterized in that, The adjacency relationships between the skeleton points represented by the adjacency matrix reconstructed based on cosine similarity threshold optimization are obtained, including: Initialize multiple standard direction vectors that uniformly cover all quadrants of the three-dimensional space; Traverse the candidate edges corresponding to the skeleton points and calculate the normalized direction vector of the candidate edges; wherein, the candidate edge is the edge between two skeleton points with a connection relationship in the reconstructed adjacency matrix; By using cosine similarity, the normalized direction vector of the candidate edge is matched with the standard direction vector to obtain the matching direction vector of the candidate edge; Based on the matching direction vectors corresponding to all candidate edges of the skeleton point, and the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector, the target edge is selected from the candidate edges, and the skeleton point and the target edge represent the skeleton topological connection relationship.

3. The method for connecting skeleton points to form a skeleton according to claim 2, characterized in that, The method further includes: Determine the number of matching direction vectors for all candidate edges corresponding to each skeleton point; The skeleton points whose number is less than the screening threshold are identified as non-isolated points; Based on the matching direction vectors corresponding to all candidate edges of the skeleton point and the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector, target edges are selected from the candidate edges, including: For each non-isolated point, if the cosine similarity between the normalized direction vector of the candidate edge and the matching direction vector is greater than a similarity threshold, the candidate edge is determined as the target edge.

4. A training method for a graph autoencoder model, characterized in that, The graph autoencoder model is used to connect skeleton points into a skeleton, including: Obtain the target point cloud set of the object; Skeleton points are extracted based on the target point cloud set; wherein, during the extraction of skeleton points, point cloud features are extracted based on the target point cloud set, and a set of skeleton spheres covering the object are predicted based on the point cloud features, and the center of the skeleton sphere is the extracted skeleton point; The adjacency matrix of the skeleton points is constructed using a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points. The adjacency matrix, the skeleton sphere radius corresponding to the skeleton point, and the node features formed by the point cloud features are input into the encoder module of the graph autoencoder model. The encoder module compresses the node features into latent features based on the adjacency relationships between the skeleton points in the adjacency matrix. The decoder module of the graph autoencoder model then reconstructs the adjacency matrix using the latent features. A loss function is constructed based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

5. The training method for the graph autoencoder model according to claim 4, characterized in that, The training process of the graph autoencoder model is supervised by introducing masked balanced cross-entropy.

6. A device for connecting skeleton points to form a skeleton, characterized in that, include: The first acquisition module is configured to acquire the target point cloud set of the object; The first extraction module is configured to extract skeleton points based on the target point cloud set; wherein, during the extraction of skeleton points, point cloud features are extracted based on the target point cloud set, and a set of skeleton spheres covering the object are predicted based on the point cloud features, and the center of the skeleton spheres is the extracted skeleton point. The first construction module is configured to construct the adjacency matrix of the skeleton points using a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points. The first prediction module is configured to input the node features, which are composed of the adjacency matrix, the skeleton sphere radius corresponding to the skeleton point, and the point cloud features, into the encoder module of the graph autoencoder model. The encoder module compresses the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix. The decoder module of the graph autoencoder model reconstructs the adjacency matrix using the latent features. The optimization module is configured to optimize the adjacency relationships between the skeleton points represented by the reconstructed adjacency matrix based on the cosine similarity threshold, and obtain the skeleton topological connection relationships between the skeleton points.

7. A training device for a graph autoencoder model, characterized in that, The graph autoencoder model is used to connect skeleton points into a skeleton, including: The second acquisition module is configured to acquire the target point cloud set of the object; The second extraction module is configured to extract skeleton points based on the target point cloud set; wherein, during the extraction of skeleton points, point cloud features are extracted based on the target point cloud set, and a set of skeleton spheres covering the object are predicted based on the point cloud features, and the center of the skeleton spheres is the extracted skeleton point. The second construction module is configured to construct the adjacency matrix of the skeleton points using a local neighborhood graph; the adjacency matrix represents the adjacency relationship between the skeleton points. The second prediction module is configured to input the node features, which are composed of the adjacency matrix, the skeleton sphere radius corresponding to the skeleton point, and the point cloud features, into the encoder module of the graph autoencoder model. The encoder module compresses the node features into latent features based on the adjacency relationship between the skeleton points in the adjacency matrix. The decoder module of the graph autoencoder model reconstructs the adjacency matrix using the latent features. The training module is configured to construct a loss function based on the adjacency matrix and the reconstructed adjacency matrix to train the graph autoencoder model.

8. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing multiple instructions, and the processor being used to read the instructions and execute the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions, which can be read by a processor and executed as described in any one of claims 1-5.

10. A computer program product comprising computer instructions, wherein, When executed by a processor, the computer instructions implement the method described in any one of claims 1-5.

Images