Three-dimensional model feature recognition method based on heterogeneous graph neural network

Abstract

The invention provides a three-dimensional model feature recognition method based on a heterogeneous graph neural network, and belongs to the technical field of computer-aided manufacturing. Comprising the following steps: selecting a three-dimensional model database with feature tags; respectively and sequentially obtaining a topological set, a geometric information set and a normalized geometric information set of each three-dimensional model, and respectively carrying out geometric feature extraction on each piece of normalized geometric information to obtain a corresponding geometric mapping feature set; constructing an initial heterogeneous graph neural network, dividing data of all three-dimensional models in the three-dimensional model database into a training set, a verification set and a test set by using all geometric mapping feature sets, and training, verifying and testing the initial heterogeneous graph neural network in sequence to obtain a final heterogeneous graph neural network; and performing feature recognition on the three-dimensional model by using the final heterogeneous graph neural network. According to the invention, the feature recognition capability and recognition accuracy of the complex three-dimensional model can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer-aided manufacturing technology, and in particular, to a three-dimensional model feature recognition method based on a heterogeneous graph neural network. Background Art

[0002] In the digital process of modern manufacturing, three-dimensional model feature recognition has become a key link in the field of computer-aided manufacturing technology. Three-dimensional model feature recognition is to decompose a complex three-dimensional model into independent units with clear semantics, thereby opening up the full-link data channel from conceptual design to product manufacturing. Its core value lies in transforming unstructured three-dimensional model data into computable structured data and playing a core role in the product manufacturing process. Currently, some methods for three-dimensional model feature recognition have emerged:

[0003] For example, a Chinese invention patent with the title "An Intelligent Recognition Method for Machining Features Based on Point Cloud Semantic Segmentation", application number 202010647445.4, and publication date November 10, 2020, proposes to convert the problem of intelligent recognition of machining features of a part CAD model into a semantic segmentation problem of a three-dimensional point cloud model. After converting the three-dimensional model into a point cloud, a PointNet semantic segmentation network is used as the basic framework for feature recognition. However, this method requires preprocessing of the three-dimensional model to convert it into a point cloud data format. During the conversion process, the topological connection relationship between the surfaces of the three-dimensional model is ignored, so that all the information expressed by the three-dimensional model cannot be fully considered, resulting in inaccurate results of three-dimensional model feature recognition.

[0004] Another example is a Chinese invention patent with the title "A Robust Component Segmentation Method for Three-Dimensional Point Clouds Based on the Semantic SAM Model", application number 202410849292.X, and publication date July 26, 2024. It proposes that first, multi-view projection of the three-dimensional point cloud generates a sequence of two-dimensional depth maps, and the multi-modal fusion ability of the semantic SAM model is used to extract cross-view component-level semantic features; then, a pre-trained SAM model is fine-tuned through a learnable adapter module, and the two-dimensional depth map is used as training data to optimize the segmentation boundary perception ability of the SAM model; finally, through the reverse projection of the cross-view segmentation result and the three-dimensional fusion algorithm, the two-dimensional semantic feature prediction is accurately mapped to the original point cloud space to achieve three-dimensional model feature recognition. However, this method has disadvantages in segmenting complex geometric bodies and has a high cost.

[0005] There are also some studies on methods for learning geometric representations from B-rep models. The process is to decompose B-rep data into the UV parameter domains of curves or surfaces and topological adjacency graphs, then use the fusion of image convolution and graph convolution to achieve the fusion of geometric features and topological features, and use a 3D model dataset for verification. However, the flexibility of topological modeling of this method is limited and it is difficult to meet 3D models with complex features.

[0006] In summary, it can be seen that the prior art has the following two deficiencies:

[0007] First, during the data representation process, the geometric features and topological features of the original 3D model cannot be effectively retained simultaneously;

[0008] Second, it is impossible to take into account the recognition of 3D models with complex features.

[0009] Therefore, it is necessary to propose a solution to improve one or more problems existing in the above related technical solutions.

[0010] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0011] The embodiments of the present application provide a 3D model feature recognition method based on a heterogeneous graph neural network. The method includes the following steps:

[0012] Select a 3D model database with feature labels, where the 3D model database contains multiple 3D models;

[0013] Respectively obtain all the topological information contained in each of the 3D models, jointly form all the topological sets of the 3D model database, and use all the topological sets to construct a 3D model heterogeneous graph;

[0014] Using the point cloud sampling method, respectively obtain the geometric information sets corresponding to each of the topological sets in the 3D model heterogeneous graph, and respectively normalize all the geometric information in each geometric information set to obtain the normalized geometric information corresponding to each geometric information;

[0015] Construct a feature extraction network, and use the feature extraction network to respectively extract geometric features from each of the normalized geometric information to obtain a geometric mapping feature set corresponding to each of the normalized geometric information;

[0016] Construct an initial heterogeneous graph neural network, divide the data of all the 3D models in the 3D model database into a training set, a validation set, and a test set by using all the geometric mapping feature sets, and sequentially train, validate, and test the initial heterogeneous graph neural network to obtain a final heterogeneous graph neural network;

[0017] Use the final heterogeneous graph neural network to perform feature recognition on the 3D model.

[0018] Further, the step of respectively obtaining all the topological information included in each 3D model, jointly constituting all the topological sets of the 3D model database, and constructing a heterogeneous graph of the 3D model by using all the topological sets includes:

[0019] Traverse all the surfaces of each 3D model respectively, and jointly form the face node topological set of the 3D model database with all the face node topological information of each obtained 3D model;

[0020] Determine the adjacency relationship between all the surfaces of each 3D model and all the surfaces of all the other 3D models respectively, and jointly form the adjacency edge topological set of the 3D model database with all the adjacency edge topological information of each obtained 3D model;

[0021] Traverse all the vertices of each 3D model respectively, and jointly form the vertex node topological set of the 3D model database with all the vertex node topological information of each obtained 3D model;

[0022] Determine the subordination relationship between all the vertices of each 3D model and all the surfaces respectively, and jointly form the subordinate edge topological set of the 3D model database with all the subordinate edge topological information of each obtained 3D model;

[0023] Construct the heterogeneous graph of the 3D model by using the face node topological set, the adjacency edge topological set, the vertex node topological set, and the subordinate edge topological set.

[0024] Further, the expression of the heterogeneous graph of the 3D model is:

[0025] (1)

[0026] Wherein, represents the heterogeneous graph of the 3D model, represents the node topological set, , represents the face node topological set, represents the vertex node topological set, represents the edge topological set, , represents the set of adjacent edge topologies, represents the set of subordinate edge topologies.

[0027] Further, the step of using the point cloud sampling method to respectively obtain the geometric information sets corresponding to each of the topological sets in the heterogeneous graph of the three-dimensional model, and respectively performing normalization processing on all the geometric information in each of the geometric information sets to obtain the normalized geometric information corresponding to each of the geometric information includes:

[0028] Performing a finite number of point cloud samplings on all the surfaces of each of the three-dimensional models respectively, and jointly taking the coordinates of all the surface sampling points of each of the obtained three-dimensional models and their corresponding surface node normal vectors as the surface node geometric information set corresponding to the surface node topological set;

[0029] Performing a finite number of point cloud samplings on all the adjacent edges of each of the three-dimensional models respectively, and jointly taking the coordinates of all the adjacent edge sampling points of each of the obtained three-dimensional models and their corresponding tangent vectors as the adjacent edge geometric information set corresponding to the adjacent edge topological set;

[0030] Jointly taking the coordinates of all the vertex nodes of each of the three-dimensional models and their corresponding vertex node normal vectors as the vertex node geometric information set corresponding to the vertex node topological set;

[0031] Among them, the surface node geometric information set contains multiple surface node geometric information, the adjacent edge geometric information set contains multiple adjacent edge geometric information, and the vertex node geometric information set contains multiple vertex node geometric information;

[0032] Performing normalization processing on all the surface node geometric information, all the adjacent edge geometric information, and all the vertex node geometric information respectively.

[0033] Further, the step of performing normalization processing on all the surface node geometric information, all the adjacent edge geometric information, and all the vertex node geometric information respectively includes:

[0034] Respectively taking the average value of the coordinates of all the surface sampling points of each of the three-dimensional models as the surface center coordinate of the corresponding three-dimensional model, and respectively taking the difference between the coordinate of each surface sampling point of each of the three-dimensional models and the corresponding surface center coordinate as the surface node normalization information;

[0035] The expression of the surface node normalization information is:

[0036] (2)

[0037] Among them, Indicates the The surface center coordinates of the three-dimensional model, represents the number of all surface sampling points, Indicates the Surface sampling points, Indicates the The coordinates of the surface sampling points, Indicates the Normalized information of surface nodes corresponding to each surface sampling point;

[0038] Taking the average value of the coordinates of all the adjacent edge sampling points of each of the three-dimensional models as the adjacent edge center coordinates of the corresponding three-dimensional model, and taking the difference between the coordinates of each adjacent edge sampling point of each of the three-dimensional models and the corresponding adjacent edge center coordinates as the corresponding adjacent edge normalization information;

[0039] The expression of the normalized information of the adjacent edge is:

[0040] (3)

[0041] in, Indicates the The center coordinates of the adjacent edges of the three-dimensional model, represents the number of all adjacent edge sampling points, Indicates the adjacent edge sampling points, Indicates the The coordinates of the adjacent edge sampling points, Indicates the Normalized information of adjacent edges corresponding to adjacent edge sampling points;

[0042] taking the average value of the coordinates of all the vertex nodes of each three-dimensional model as the vertex center coordinates of the corresponding three-dimensional model, and taking the difference between the coordinates of each vertex node of each three-dimensional model and the corresponding vertex center coordinates as the corresponding vertex node normalization information;

[0043] The expression of the vertex node normalization information is:

[0044] (4)

[0045] in, Indicates the The vertex center coordinates of the three-dimensional model, Represents the number of all vertex nodes, Indicates the Vertex nodes, Indicates the The coordinates of the vertex nodes, Represents the vertex node normalization information corresponding to the th vertex node.

[0046] Furthermore, the feature extraction network includes a face node feature extraction sub-network, an adjacent edge feature extraction sub-network, and a vertex node feature extraction sub-network;

[0047] The face node feature extraction sub-network and the adjacent edge feature extraction sub-network have the same structure, both including:

[0048] A first convolutional layer, the output end of the first convolutional layer is connected to the input end of a first activation function;

[0049] A second convolutional layer, the input end of the second convolutional layer is connected to the output end of the first activation function, and the output end of the second convolutional layer is connected to the input end of a second activation function;

[0050] A third convolutional layer, the input end of the third convolutional layer is connected to the output end of the second activation function, and the output end of the third convolutional layer is connected to the input end of a third activation function;

[0051] A pooling layer, the input end of the pooling layer is connected to the output end of the third activation function, and the pooling layer uses an adaptive average pooling operator;

[0052] A first fully connected layer, the input end of the first fully connected layer is connected to the output end of the pooling layer;

[0053] The structure of the vertex node feature extraction sub-network includes: a second fully connected layer, a normalization layer, and a non-linear activation layer connected in sequence;

[0054] Among them, the first convolutional layer, the second convolutional layer, and the third convolutional layer all contain convolution kernels.

[0055] Furthermore, the step of using the feature extraction network to respectively perform geometric feature extraction on each of the normalized geometric information to obtain a geometric mapping feature set corresponding to each of the normalized geometric information includes:

[0056] Respectively using all the surface sampling points of each of the 3D models to form a -dimensional tensor feature of each surface of the corresponding 3D model, where represents the number of initial channels of the face node feature extraction sub-network;

[0057] Inputting all the -dimensional tensor features into the face node feature extraction sub-network, and respectively passing each of the The feature of the d-dimensional tensor is lifted to a 64-dimensional face node feature space. Each 64-dimensional face node feature space is respectively expanded to a 128-dimensional face node high-level semantic feature by the second convolutional layer, each 128-dimensional face node high-level semantic feature is respectively deepened to a 256-dimensional face node abstract feature by the third convolutional layer, each 256-dimensional face node abstract feature is respectively subjected to adaptive average pooling processing by the pooling layer, and corresponding 64-dimensional face node vectors are respectively output by the first fully connected layer, forming a face node mapping feature set;

[0058] All the adjacent edge sampling points of each 3D model are respectively used to form the d-dimensional tensor feature of each adjacent edge of the corresponding 3D model, where represents the number of initial channels of the adjacent edge feature extraction sub-network;

[0059] All the d-dimensional tensor features are input into the adjacent edge feature extraction sub-network. Each d-dimensional tensor feature is lifted to a 64-dimensional adjacent edge feature space by the first convolutional layer, each 64-dimensional adjacent edge feature space is respectively expanded to a 128-dimensional adjacent edge high-level semantic feature by the second convolutional layer, each 128-dimensional adjacent edge high-level semantic feature is respectively deepened to a 256-dimensional adjacent edge abstract feature by the third convolutional layer, each 256-dimensional adjacent edge abstract feature is respectively subjected to the adaptive average pooling processing by the pooling layer, and corresponding 64-dimensional adjacent edge vectors are respectively output by the first fully connected layer, forming an adjacent edge mapping feature set;

[0060] All the vertex nodes of each 3D model are respectively used to form the d-dimensional tensor feature of all the vertices of the corresponding 3D model, where represents the number of initial channels of the vertex node feature extraction sub-network;

[0061] All the d-dimensional tensor features are input into the vertex node feature extraction sub-network. Each d-dimensional tensor feature is lifted to a 64-dimensional vertex node feature space by the second fully connected layer, and corresponding 64-dimensional vertex node vectors are respectively output through the normalization layer and the non-linear activation layer in sequence, forming a vertex node mapping feature set;

[0062] The types of the geometric feature set include the face node mapping feature set, the adjacent edge mapping feature set and the vertex node mapping feature set.

[0063] Further, the first activation function, the second activation function, and the third activation function are all LeaklyRelu non-linear activation functions;

[0064] The expression of the LeaklyRelu non-linear activation function is:

[0065] (5)

[0066] where represents the LeaklyRelu non-linear activation function, represents the th actual value passed to the LeaklyRelu non-linear activation function in the feature extraction network, represents the hyperparameter;

[0067] The non-linear activation layer includes the LeaklyRelu non-linear activation function.

[0068] Further, the steps of constructing the initial heterogeneous graph neural network, using all the geometric mapping feature sets to divide the data of all the 3D models in the 3D model database into a training set, a validation set, and a test set, and sequentially training, validating, and testing the initial heterogeneous graph neural network to obtain the final heterogeneous graph neural network include:

[0069] Dividing the data of all the 3D models in the 3D model database into the training set, the validation set, and the test set; the data of each 3D model in the training set, the validation set, and the test set includes the corresponding 64-dimensional face node vector, 64-dimensional adjacent edge vector, and 64-dimensional vertex node vector;

[0070] Constructing the initial heterogeneous graph neural network, the initial heterogeneous graph neural network includes:

[0071] A relational convolutional layer, the relational convolutional layer uses a two-branch graph convolutional operator, the relational convolutional layer includes a double-layer convolutional layer, the input end of the first convolutional layer is the input end of the relational convolutional layer, the output end of the second convolutional layer is the output end of the relational convolutional layer, and the input end of the first convolutional layer is connected to the output end of the feature extraction network;

[0072] A random dropout layer, the input end of the random dropout layer is connected to the output end of the first convolutional layer; the output end of the random dropout layer is connected to the input end of the second convolutional layer;

[0073] Among them, the dual-branch graph convolution operator is used to perform a dual-path feature vector aggregation operation on the feature vectors output from the output end of the second convolutional layer along the corresponding initial channels, and all the aggregated feature vectors obtained are input into a multi-layer perceptron to obtain the probability distribution of the semantic categories of each of the 3D models respectively;

[0074] Set the model parameters, and train, validate, and test the initial heterogeneous graph neural network in sequence to obtain the final heterogeneous graph neural network.

[0075] Furthermore, the expression of the dual-branch graph convolution operator is:

[0076] h i (k) =LeaklyRelu [ W (k) ∙concat( h i (k-1) , h N(i) (k) )] (6)

[0077] Among them, represents the feature vector of the -th node in the -th convolutional layer of the relational convolutional layer, represents the training weight matrix of the -th convolutional layer of the relational convolutional layer, represents the dual-path feature vector aggregation operation, represents the feature vector of the -th node in the -th convolutional layer of the relational convolutional layer, represents the aggregated feature vector of all neighbor nodes of the -th node in the -th convolutional layer of the relational convolutional layer, represents the set of all neighbor nodes of the -th node;

[0078] The model parameters include the training batch size, the number of training epochs, the learning rate, the type of optimizer, and the loss function.

[0079] This application provides a 3D model feature recognition method based on a heterogeneous graph neural network, which has at least the following beneficial effects:

[0080] (1) This application characterizes the node topology information and edge topology information of 3D models by constructing a 3D model heterogeneous graph, breaking through the limitations of traditional single geometric representation, and uses the point cloud sampling method to obtain the geometric information set corresponding to each topology set respectively, realizing the collaborative optimization of the topological structure features and geometric shape features of 3D models, thereby improving the feature recognition ability of complex 3D models.

[0081] (2) This application designs a heterogeneous graph neural network, which includes a relational convolutional layer with a double-layer convolutional layer, and uses a dual-branch graph convolutional operator to perform a dual-path feature vector aggregation operation on the output feature vectors along the corresponding initial channels, and inputs all the aggregated feature vectors obtained into a multi-layer perceptron to obtain the probability distribution of the semantic categories of each 3D model respectively, thereby realizing the construction of an independent feature transformation channel for each type of topology information and improving the accuracy of feature recognition of complex 3D models. Brief Description of the Drawings

[0082] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application. Obviously, the drawings in the following description are only some embodiments of this application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0083] Figure 1 A schematic diagram showing the steps of a 3D model feature recognition method based on a heterogeneous graph neural network in an exemplary embodiment of this application;

[0084] Figure 2 A schematic diagram showing the process of a 3D model feature recognition method based on a heterogeneous graph neural network in an exemplary embodiment of this application;

[0085] Figure 3 A schematic diagram showing the Fusion 360 Gallery database in an exemplary embodiment of this application;

[0086] Figure 4 A schematic diagram showing a 3D model with feature labels in an exemplary embodiment of this application;

[0087] Figure 5 A schematic diagram showing the structure of the feature extraction network constructed in an exemplary embodiment of this application;

[0088] Figure 6 A schematic diagram showing the structure of the initial heterogeneous graph neural network constructed in an exemplary embodiment of this application;

[0089] Figure 7Schematic diagram showing 3D model feature recognition using the final heterogeneous graph neural network in an exemplary embodiment of the present application;

[0090] Figure 8 Schematic diagram showing the training results of training the heterogeneous graph neural network in an exemplary embodiment of the present application.

[0091] Figure 4 In the figure, 101, stretching side; 102, stretching end face; 103, chamfer; 104, fillet; 105, rotation end face; 106, rotation side; 107, cutting end face; 108, cutting side. Detailed implementation manners

[0092] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0093] In addition, the drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0094] Next, a 3D model feature recognition method based on a heterogeneous graph neural network proposed in the present example embodiment will be described in more detail.

[0095] The present example embodiment provides a 3D model feature recognition method based on a heterogeneous graph neural network, as Figure 1 and Figure 2 shown, the method may include the following steps:

[0096] Step S101: Select a 3D model database with feature labels, and the 3D model database contains multiple 3D models.

[0097] In step S101 of the present embodiment, as Figure 3 shown, in the present embodiment, a Fusion 360 Gallery database is selected for illustration. There are a total of 35,680 3D models in this database, which contain 8 feature labels, as Figure 4As shown, in this embodiment, these 8 feature tags specifically refer to 8 surface tags, that is, Figure 4 the stretching side surface 101, stretching termination surface 102, chamfer 103 and fillet 104 shown in a of Figure 4 ; and Figure 4 the rotation termination surface 105, rotation side surface 106, cutting termination surface 107 and cutting side surface 108 shown in b of Figure 4 .

[0098] Step S102: Respectively obtain all topological information included in each 3D model, jointly form all topological sets of the 3D model database, and construct a heterogeneous graph of the 3D model using all topological sets. Step S102 of this embodiment may include the following steps:

[0099] Sub-step S1021: Traverse all surfaces of each 3D model respectively, and jointly form the face node topological set of the 3D model database with all face node topological information of each obtained 3D model.

[0100] Sub-step S1022: Respectively determine the adjacency relationship between all surfaces of each 3D model and all surfaces of all other 3D models, and jointly form the adjacent edge topological set of the 3D model database with all adjacent edge topological information of each obtained 3D model.

[0101] Sub-step S1023: Traverse all vertices of each 3D model respectively, and jointly form the vertex node topological set of the 3D model database with all vertex node topological information of each obtained 3D model.

[0102] Sub-step S1024: Respectively determine the subordination relationship between all vertices of each 3D model and all surfaces, and jointly form the subordinate edge topological set of the 3D model database with all subordinate edge topological information of each obtained 3D model.

[0103] Sub-step S1025: Construct a heterogeneous graph of the 3D model using the face node topological set, adjacent edge topological set, vertex node topological set and subordinate edge topological set.

[0104] Here, the types of topological information include face node topological information, adjacent edge topological information, vertex node topological information and subordinate edge topological information. The types of topological sets include face node topological set, adjacent edge topological set, vertex node topological set and subordinate edge topological set.

[0105] In sub-step S1025, the expression of the heterogeneous graph of the 3D model is:

[0106] (1)

[0107] Among them, represents the heterogeneous graph of the 3D model, Represents the set of node topologies, , represents the set of face node topologies, represents the set of vertex node topologies, represents the set of edge topologies, , represents the set of adjacent edge topologies, represents the set of subordinate edge topologies.

[0108] That is to say, the topology set of the 3D model includes the set of face node topologies, the set of adjacent edge topologies, the set of vertex node topologies, and the set of subordinate edge topologies. Among them, the set of face node topologies contains multiple face node topology information, the set of adjacent edge topologies contains multiple adjacent edge topology information, the set of vertex node topologies includes multiple vertex node topology information, and the set of subordinate edge topologies contains multiple subordinate edge topology information.

[0109] Step S103: Using the point cloud sampling method, respectively obtain the geometric information sets corresponding to each topology set in the heterogeneous graph of the 3D model, and respectively normalize all the geometric information in each geometric information set to obtain the normalized geometric information corresponding to each geometric information. Step S103 of this embodiment may include the following sub-steps:

[0110] Sub-step S1031: Respectively perform 100 times of point cloud sampling on all the surfaces of each 3D model, and jointly use the coordinates of all the surface sampling points of each obtained 3D model and their corresponding face node normal vectors as the face node geometric information set corresponding to the set of face node topologies.

[0111] Sub-step S1032: Respectively perform 100 times of point cloud sampling on all the adjacent edges of each 3D model, and jointly use the coordinates of all the adjacent edge sampling points of each obtained 3D model and their corresponding tangent vectors as the adjacent edge geometric information set corresponding to the set of adjacent edge topologies.

[0112] Sub-step S1033: Jointly use the coordinates of all the vertex nodes of each 3D model and their corresponding vertex node normal vectors as the vertex node geometric information set corresponding to the set of vertex node topologies.

[0113] Here, the face node geometric information set contains multiple face node geometric information, the adjacent edge geometric information set contains multiple adjacent edge geometric information, and the vertex node geometric information set contains multiple vertex node geometric information. Moreover, each topology information in each topology set corresponds to one geometric information in the corresponding geometric information set respectively.

[0114] Sub-step S1034: Respectively normalize all the face node geometric information, all the adjacent edge geometric information, and all the vertex node geometric information. The specific process of sub-step S1034 is as follows:

[0115] First, take the average of the coordinates of all surface sampling points of each 3D model as the surface center coordinates of the corresponding 3D model, and take the difference between the coordinates of each surface sampling point of each 3D model and the corresponding surface center coordinates as the corresponding face node normalization information.

[0116] Here, the expression for the face node normalization information is:

[0117] (2)

[0118] Among them, represents the surface center coordinates of the th 3D model, represents the number of all surface sampling points, represents the th surface sampling point, represents the coordinates of the th surface sampling point, represents the face node normalization information corresponding to the th surface sampling point.

[0119] Secondly, take the average of the coordinates of all adjacent edge sampling points of each 3D model as the adjacent edge center coordinates of the corresponding 3D model, and take the difference between the coordinates of each adjacent edge sampling point of each 3D model and the corresponding adjacent edge center coordinates as the corresponding adjacent edge normalization information.

[0120] Here, the expression for the adjacent edge normalization information is:

[0121] (3)

[0122] Among them, represents the adjacent edge center coordinates of the th 3D model, represents the number of all adjacent edge sampling points, represents the th adjacent edge sampling point, represents the coordinates of the th adjacent edge sampling point, represents the adjacent edge normalization information corresponding to the th adjacent edge sampling point.

[0123] Finally, take the average of the coordinates of all vertex nodes of each 3D model as the vertex center coordinates of the corresponding 3D model, and take the difference between the coordinates of each vertex node of each 3D model and the corresponding vertex center coordinates as the corresponding vertex node normalization information.

[0124] Here, the expression of the vertex node normalization information is:

[0125] (4)

[0126] Among them, represents the vertex center coordinates of the th 3D model, represents the total number of all vertex nodes, represents the th vertex node, represents the th vertex node coordinates, represents the vertex node normalization information corresponding to the th vertex node.

[0127] Here, all the face node normalization information forms a face node normalization information set, all the adjacent edge normalization information forms an adjacent edge normalization information set, and all the vertex node normalization information forms a vertex node normalization information set.

[0128] Step S104: As shown in Figure 5 , construct a feature extraction network, and use the feature extraction network to perform geometric feature extraction on each normalized geometric information respectively to obtain a geometric mapping feature set corresponding to each normalized geometric information. Step S104 of this embodiment may include the following sub-steps:

[0129] Sub-step S1041: Construct a feature extraction network.

[0130] In sub-step S1041, the feature extraction network includes three sub-networks: namely, a face node feature extraction sub-network, an adjacent edge feature extraction sub-network, and a vertex node feature extraction sub-network.

[0131] Figure 5 As shown in a of , the structures of the face node feature extraction sub-network and the adjacent edge feature extraction sub-network are the same, and both include:

[0132] The first convolutional layer, the output end of the first convolutional layer is connected to the input end of the first activation function;

[0133] The second convolutional layer, the input end of the second convolutional layer is connected to the output end of the first activation function, and the output end of the second convolutional layer is connected to the input end of the second activation function;

[0134] The third convolutional layer, the input end of the third convolutional layer is connected to the output end of the second activation function, and the output end of the third convolutional layer is connected to the input end of the third activation function;

[0135] The pooling layer, the input end of the pooling layer is connected to the output end of the third activation function, and the pooling layer uses an adaptive average pooling operator;

[0136] The first fully connected layer, the input end of the first fully connected layer is connected to the output end of the pooling layer.

[0137] Figure 5 As shown in b), the structure of the vertex node feature extraction sub-network includes: a second fully connected layer, a normalization layer, and a non-linear activation layer connected in sequence.

[0138] In this embodiment, the first convolutional layer, the second convolutional layer, and the third convolutional layer all contain convolution kernels.

[0139] In this embodiment, the first activation function, the second activation function, and the third activation function are all LeaklyRelu non-linear activation functions;

[0140] The expression of the LeaklyRelu non-linear activation function is:

[0141] (5)

[0142] Where represents the LeaklyRelu non-linear activation function, represents the th actual value passed to the LeaklyRelu non-linear activation function in the feature extraction network, represents the hyperparameter, usually takes 0.01.

[0143] The non-linear activation layer contains the LeaklyRelu non-linear activation function.

[0144] Sub-step S1042: Use the feature extraction network to perform geometric feature extraction on each normalized geometric information respectively, and obtain a geometric mapping feature set corresponding to each normalized geometric information. The specific process of sub-step S1042 is as follows:

[0145] First, use all the surface sampling points of each 3D model to form the -dimensional tensor feature of each surface of the corresponding 3D model, where represents the number of initial channels of the face node feature extraction sub-network.

[0146] All -dimensional tensor features are input into the face node feature extraction sub-network, and each of the The feature of the .

[0147] Secondly, all the sampled points of the adjacent edges of each 3D model are used to form the -dimensional tensor feature of each adjacent edge of the corresponding 3D model, where represents the number of initial channels of the adjacent edge feature extraction sub-network.

[0148] All the -dimensional tensor features are input into the adjacent edge feature extraction sub-network. Each -dimensional tensor feature is lifted to a 64-dimensional adjacent edge feature space through the first convolutional layer, each 64-dimensional adjacent edge feature space is extended to a 128-dimensional adjacent edge high-level semantic feature through the second convolutional layer, each 128-dimensional adjacent edge high-level semantic feature is deepened to a 256-dimensional adjacent edge abstract feature through the third convolutional layer, each 256-dimensional adjacent edge abstract feature is subjected to adaptive average pooling through the pooling layer, and the corresponding 64-dimensional adjacent edge vectors are output through the first fully connected layer respectively, forming an adjacent edge mapping feature set. In this embodiment, .

[0149] Finally, all the vertex nodes of each 3D model are used to form the -dimensional tensor feature of each vertex of the corresponding 3D model, where represents the number of initial channels of the vertex node feature extraction sub-network.

[0150] All the -dimensional tensor features are input into the vertex node feature extraction sub-network. Each -dimensional tensor feature is lifted to a 64-dimensional vertex node feature space through the second fully connected layer, and the corresponding 64-dimensional vertex node vectors are output through the normalization layer and the non-linear activation layer in sequence respectively, forming a vertex node mapping feature set. In this embodiment, .

[0151] Here, the types of the geometric mapping feature set include the face node mapping feature set, the adjacent edge mapping feature set, and the vertex node mapping feature set.

[0152] Step S105: As shown in Figure 6As shown in the figure, an initial heterogeneous graph neural network is constructed. Using all geometric mapping feature sets, the data of all 3D models in the 3D model database is divided into a training set, a validation set, and a test set. Then, the initial heterogeneous graph neural network is trained, validated, and tested in sequence to obtain the final heterogeneous graph neural network. Step S105 of this embodiment may include the following sub-steps:

[0153] Sub-step S1051: Divide the data of all 3D models in the 3D model database into a training set, a validation set, and a test set; each of the 3D models in the training set, the validation set, and the test set includes a corresponding 64-dimensional face node vector, a 64-dimensional adjacent edge vector, and a 64-dimensional vertex node vector.

[0154] In this embodiment, the ratio of the training set, the validation set, and the test set is 7:1.5:1.5.

[0155] Sub-step S1052: Construct an initial heterogeneous graph neural network, which includes:

[0156] A relational convolutional layer, which uses a dual-branch graph convolutional operator. The relational convolutional layer includes a double-layer convolutional layer. The input end of the first convolutional layer is the input end of the relational convolutional layer, and the output end of the second convolutional layer is the output end of the relational convolutional layer. The input end of the first convolutional layer is connected to the output end of the feature extraction network;

[0157] A random dropout layer, the input end of which is connected to the output end of the first convolutional layer; the output end of the random dropout layer is connected to the input end of the second convolutional layer;

[0158] Among them, using the dual-branch graph convolutional operator, the feature vectors output from the output end of the second convolutional layer are subjected to a dual-path feature vector aggregation operation along the corresponding initial channels, and all the aggregated feature vectors obtained are input into a multi-layer perceptron (MLP) to obtain the probability distribution of the semantic categories of each 3D model respectively.

[0159] It should be noted here that in this embodiment, the probability distribution of the semantic category of each 3D model means that the multi-layer perceptron includes 8 output channels, representing 8 feature labels of the 3D model. Among them, the output channel with the highest probability is the semantic category of the 3D model. The semantic category is the feature label, that is, the finally obtained feature recognition result.

[0160] Sub-step S1053: As Figure 7 shown, set the model parameters, and then train, validate, and test the initial heterogeneous graph neural network in sequence to obtain the final heterogeneous graph neural network.

[0161] This embodiment Figure 7 in represents the topological information of the th vertex node, represents the topological information of the th vertex node, represents the topological information of the th vertex node. In this embodiment, Figure 7 the represents the topological information of the th face node, represents the topological information of the th face node, represents the topological information of the th face node, represents the topological information of the th face node.

[0162] In this embodiment, a random dropout with a probability of 0.5 is applied to the random dropout layer. In this embodiment, the expression of the double-branch graph convolution operator:

[0163] h i (k) =LeaklyRelu [ W (k) ∙concat( h i (k-1) , h N(i) (k) )] (6)

[0164] where, represents the feature vector of the th node in the th convolutional layer of the relational convolutional layer, represents the training weight matrix of the th convolutional layer of the relational convolutional layer, represents the dual-path feature vector aggregation operation, represents the th node's feature vector in the th convolutional layer of the relational convolutional layer, represents the th node's aggregated feature vector of all its neighbor nodes in the th convolutional layer of the relational convolutional layer, represents the set of all neighbor nodes of the th node.

[0165] The nodes here refer to one or more of the face nodes and vertex nodes mentioned in this embodiment.

[0166] Sub-step S1054: Set the model parameters, and then train, validate, and test the initial heterogeneous graph neural network in sequence to obtain the final heterogeneous graph neural network.

[0167] In this embodiment, the model parameters are set as follows:

[0168] The training batch is 960;

[0169] The number of training epochs is 200;

[0170] The learning rate is 0.001;

[0171] The optimizer is preferably the Adam optimizer;

[0172] The loss function is preferably the cross-entropy loss function.

[0173] Figure 8 The figure shows a schematic diagram of the training results of training the initial heterogeneous graph neural network. Through Figure 8 It can be seen that the initial heterogeneous graph neural network converges at about the 110th training epoch, and the accuracy converges to about 89%. This indicates that the method proposed in this application can meet the requirements of complex 3D model feature recognition.

[0174] Step S106: Use the final heterogeneous graph neural network to identify the 3D model features.

[0175] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, "a plurality" means two or more unless otherwise specifically defined.

[0176] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0177] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

[0178] Other embodiments of the present application will be readily apparent to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application.

Claims

1. A three-dimensional model feature recognition method based on a heterogeneous graph neural network, characterized in that The method includes the following steps: Select a 3D model database with feature tags, where the 3D model database contains multiple 3D models; Respectively obtain all the topological information contained in each of the 3D models, jointly form all the topological sets of the 3D model database, and use all the topological sets to construct a heterogeneous graph of 3D models; Using the point cloud sampling method, respectively obtain the geometric information sets corresponding to each of the topological sets in the heterogeneous graph of 3D models, and respectively normalize all the geometric information in each geometric information set to obtain the normalized geometric information corresponding to each geometric information; Construct a feature extraction network, and use the feature extraction network to respectively extract geometric features from each of the normalized geometric information to obtain a geometric mapping feature set corresponding to each of the normalized geometric information; Construct an initial heterogeneous graph neural network, use all the geometric mapping feature sets to divide the data of all the 3D models in the 3D model database into a training set, a validation set and a test set, and sequentially train, validate and test the initial heterogeneous graph neural network to obtain a final heterogeneous graph neural network; Use the final heterogeneous graph neural network to perform feature recognition on the 3D models.

2. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 1, wherein The step of respectively obtaining all the topological information contained in each of the 3D models, jointly forming all the topological sets of the 3D model database, and using all the topological sets to construct a heterogeneous graph of 3D models includes: Respectively traverse all the surfaces of each of the 3D models, and jointly form the face node topological set of the 3D model database with all the face node topological information obtained for each of the 3D models; Respectively determine the adjacency relationships between all the surfaces of each of the 3D models and all the surfaces of all the other 3D models, and jointly form the adjacent edge topological set of the 3D model database with all the adjacent edge topological information obtained for each of the 3D models; Respectively traverse all the vertices of each of the 3D models, and jointly form the vertex node topological set of the 3D model database with all the vertex node topological information obtained for each of the 3D models; Respectively determine the subordination relationships between all the vertices of each of the 3D models and all the surfaces, and jointly form the subordinate edge topological set of the 3D model database with all the subordinate edge topological information obtained for each of the 3D models; Use the face node topological set, the adjacent edge topological set, the vertex node topological set and the subordinate edge topological set to construct the heterogeneous graph of 3D models.

3. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 2, wherein The expression of the heterogeneous graph of 3D models is: (1) Among them, represents a heterogeneous graph of 3D models, represents a set of node topologies, , represents a set of face node topologies, represents a set of vertex node topologies, represents a set of edge topologies, , represents a set of adjacent edge topologies, represents a set of subordinate edge topologies.

4. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 2, wherein, The step of using the point cloud sampling method, respectively obtaining the geometric information sets corresponding to each of the topological sets in the heterogeneous graph of 3D models, and respectively normalizing all the geometric information in each geometric information set to obtain the normalized geometric information corresponding to each geometric information includes: Perform finite-point cloud sampling on all the surfaces of each of the three-dimensional models, and jointly use the coordinates of all the surface sampling points of each obtained three-dimensional model and their corresponding surface node normal vectors as the surface node geometric information set corresponding to the surface node topology set; Perform finite-point cloud sampling on all the adjacent edges of each of the three-dimensional models, and jointly use the coordinates of all the adjacent edge sampling points of each obtained three-dimensional model and their corresponding tangent vectors as the adjacent edge geometric information set corresponding to the adjacent edge topology set; Jointly use the coordinates of all the vertex nodes of each of the three-dimensional models and their corresponding vertex node normal vectors as the vertex node geometric information set corresponding to the vertex node topology set; Among them, the surface node geometric information set contains multiple surface node geometric information, the adjacent edge geometric information set contains multiple adjacent edge geometric information, and the vertex node geometric information set contains multiple vertex node geometric information; Perform normalization processing on all the surface node geometric information, all the adjacent edge geometric information, and all the vertex node geometric information respectively.

5. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 4, characterized in that, The step of performing normalization processing on all the surface node geometric information, all the adjacent edge geometric information, and all the vertex node geometric information respectively includes: Respectively take the average value of the coordinates of all the surface sampling points of each of the three-dimensional models as the surface center coordinate of the corresponding three-dimensional model, and respectively take the difference between the coordinates of each surface sampling point of each three-dimensional model and the corresponding surface center coordinate as the corresponding surface node normalization information; The expression of the surface node normalization information is: (2) Among them, represents the surface center coordinates of the th 3D model, represents the number of all surface sampling points, represents the th surface sampling point, represents the coordinates of the th surface sampling point, represents the face node normalization information corresponding to the th surface sampling point; Respectively take the average value of the coordinates of all the adjacent edge sampling points of each of the three-dimensional models as the adjacent edge center coordinate of the corresponding three-dimensional model, and respectively take the difference between the coordinates of each adjacent edge sampling point of each three-dimensional model and the corresponding adjacent edge center coordinate as the corresponding adjacent edge normalization information; The expression of the adjacent edge normalization information is: (3) Among them, represents the central coordinates of the adjacent edges of the th 3D model, represents the number of all adjacent edge sampling points, represents the th adjacent edge sampling point, represents the th coordinates of the adjacent edge sampling point, represents the adjacent edge normalization information corresponding to the th adjacent edge sampling point; Respectively take the average value of the coordinates of all the vertex nodes of each of the three-dimensional models as the vertex center coordinate of the corresponding three-dimensional model, and respectively take the difference between the coordinates of each vertex node of each three-dimensional model and the corresponding vertex center coordinate as the corresponding vertex node normalization information; The expression of the vertex node normalization information is: (4) Among them, represents the vertex center coordinates of the th 3D model, represents the number of all vertex nodes, represents the th vertex node, represents the coordinates of the th vertex node, represents the vertex node normalization information corresponding to the th vertex node.

6. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 5, wherein The feature extraction network includes a surface node feature extraction sub-network, an adjacent edge feature extraction sub-network, and a vertex node feature extraction sub-network; The structures of the surface node feature extraction sub-network and the adjacent edge feature extraction sub-network are the same, and both include: A first convolutional layer, the output end of the first convolutional layer is connected to the input end of a first activation function; A second convolutional layer, the input end of the second convolutional layer is connected to the output end of the first activation function, and the output end of the second convolutional layer is connected to the input end of a second activation function; A third convolutional layer, the input end of the third convolutional layer is connected to the output end of the second activation function, and the output end of the third convolutional layer is connected to the input end of a third activation function; Pooling layer, the input end of the pooling layer is connected to the output end of the third activation function, and the pooling layer adopts an adaptive average pooling operator; First fully connected layer, the input end of the first fully connected layer is connected to the output end of the pooling layer; The structure of the vertex node feature extraction sub-network includes: a second fully connected layer, a normalization layer, and a non-linear activation layer connected in sequence; Among them, the first convolutional layer, the second convolutional layer, and the third convolutional layer all contain convolution kernels.

7. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 6, wherein, The step of using the feature extraction network to respectively perform geometric feature extraction on each of the normalized geometric information to obtain a geometric mapping feature set corresponding to each of the normalized geometric information includes: Using all the surface sampling points of each of the three-dimensional models respectively, to form the -dimensional tensor features of each surface of the corresponding three-dimensional model, where represents the number of initial channels of the surface node feature extraction sub-network; Input all the dimensional tensor features into the face node feature extraction sub-network, and respectively use the first convolutional layer to lift each dimensional tensor feature to a 64-dimensional face node feature space, respectively use the second convolutional layer to expand each 64-dimensional face node feature space to a 128-dimensional high-level semantic feature of the face node, respectively use the third convolutional layer to deepen each 128-dimensional high-level semantic feature of the face node to a 256-dimensional abstract feature of the face node, respectively perform adaptive average pooling on each 256-dimensional abstract feature of the face node through the pooling layer, and respectively output the corresponding 64-dimensional face node vectors through the first fully connected layer to form a face node mapping feature set; Using all the adjacent edge sampling points of each of the three-dimensional models respectively, to form the -dimensional tensor features of each of the adjacent edges of the corresponding three-dimensional model, where represents the number of initial channels of the adjacent edge feature extraction sub-network; Input all the -dimensional tensor features into the adjacent edge feature extraction sub-network. Respectively, through the first convolutional layer, lift each -dimensional tensor feature to a 64-dimensional adjacent edge feature space. Respectively, through the second convolutional layer, expand each 64-dimensional adjacent edge feature space to a 128-dimensional adjacent edge high-level semantic feature. Respectively, through the third convolutional layer, deepen each 128-dimensional adjacent edge high-level semantic feature to a 256-dimensional adjacent edge abstract feature. Respectively, through the pooling layer, perform the adaptive average pooling process on each 256-dimensional adjacent edge abstract feature, and respectively output the corresponding 64-dimensional adjacent edge vectors through the first fully connected layer to form an adjacent edge mapping feature set; Using all the vertex nodes of each of the three-dimensional models respectively, form the dimensional tensor features of each vertex of the corresponding three-dimensional model, where represents the number of initial channels of the vertex node feature extraction sub-network; Input all the dimensional tensor features into the vertex node feature extraction sub-network, and respectively lift each dimensional tensor feature to a 64-dimensional vertex node feature space through the second fully-connected layer, and respectively output the corresponding 64-dimensional vertex node vectors through the normalization layer and the non-linear activation layer in sequence, forming a vertex node mapping feature set; The types of the geometric mapping feature set include the face node mapping feature set, the adjacent edge mapping feature set, and the vertex node mapping feature set.

8. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 6, wherein The first activation function, the second activation function, and the third activation function are all LeaklyRelu non-linear activation functions; The expression of the LeaklyRelu non-linear activation function is: (5) Among them, represents the LeaklyRelu non-linear activation function, represents the th actual value passed to the LeaklyRelu non-linear activation function in the feature extraction network, represents a hyperparameter; The non-linear activation layer includes the LeaklyRelu non-linear activation function.

9. The 3D model feature recognition method based on the heterogeneous graph neural network according to claim 7, wherein The steps of constructing the initial heterogeneous graph neural network, using all the geometric mapping feature sets to divide the data of all the 3D models in the 3D model database into a training set, a validation set, and a test set, and sequentially training, validating, and testing the initial heterogeneous graph neural network to obtain the final heterogeneous graph neural network include: Dividing the data of all the 3D models in the 3D model database into the training set, the validation set, and the test set; the data of each 3D model in the training set, the validation set, and the test set all includes the corresponding 64-dimensional face node vector, 64-dimensional adjacent edge vector, and 64-dimensional vertex node vector; Constructing the initial heterogeneous graph neural network, the initial heterogeneous graph neural network includes: Relational convolutional layer, the relational convolutional layer adopts a two-branch graph convolutional operator, the relational convolutional layer includes a double-layer convolutional layer, the input end of the first convolutional layer is the input end of the relational convolutional layer, the output end of the second convolutional layer is the output end of the relational convolutional layer, and the input end of the first convolutional layer is connected to the output end of the feature extraction network; Random dropout layer, the input end of the random dropout layer is connected to the output end of the first convolutional layer; the output end of the random dropout layer is connected to the input end of the second convolutional layer; Among them, using the two-branch graph convolutional operator to perform a two-way feature vector aggregation operation on the feature vector output from the output end of the second convolutional layer along the corresponding initial channel, and inputting all the aggregated feature vectors into a multi-layer perceptron to respectively obtain the probability distribution of the semantic categories of each 3D model; Setting model parameters, and sequentially training, validating, and testing the initial heterogeneous graph neural network to obtain the final heterogeneous graph neural network.

10. The three-dimensional model feature recognition method based on the heterogeneous graph neural network according to claim 9, characterized in that The expression of the two-branch graph convolutional operator is: (6) Among them, represents the feature vector of the i-th node in the l-th convolutional layer of the relational convolutional layer, represents the training weight matrix of the l-th convolutional layer of the relational convolutional layer, represents the dual-path feature vector aggregation operation, represents the j-th node's feature vector in the l-th convolutional layer of the relational convolutional layer, represents the j-th node's aggregated feature vector of all neighbor nodes in the l-th convolutional layer of the relational convolutional layer, represents the set of all neighbor nodes of the j-th node; The model parameters include the training batch, the number of training epochs, the learning rate, the type of optimizer, and the loss function.

Images