A 3D model classification method based on geometric semantic pre-training and graph network

Abstract

The application discloses a kind of 3D model classification methods based on geometric semantic pre-training and graph network, comprising the following steps: STEP data input and initial feature extraction, extraction is divided into two parts, one part is the topological data of three-dimensional model, is converted into the adjacency matrix of three-dimensional model surface after processing, another part is surface information after arrangement;Surface information is input into geometric semantic pre-training model, and the word vector of each surface is obtained;All surface word vectors and surface adjacency matrix are input into graph semantic feature hybrid network, and the representation vector of the whole model, i.e., model semantic feature, is obtained;The model corresponding to the clustering result obtained by clustering model semantic features is the same class by k-means algorithm.The application realizes the mapping of parts from file to model representation space, and then realizes the accurate classification of three-dimensional model by clustering algorithm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer graphics and intelligent identification, and particularly relates to a 3D model classification method based on geometric semantic pre-training and a graph network. BACKGROUND

[0002] In recent years, with the continuous development of science and technology, the three-dimensional model classification method based on deep learning has achieved remarkable results. Compared with other 2D images, three-dimensional models have more abundant geometric information and spatial structure characteristics, and are closer to the display scene of human life. They have wide application in the fields of industrial design, three-dimensional reconstruction, film entertainment, intelligent navigation, medical treatment and the like. Meanwhile, the classification of three-dimensional models has become more important.

[0003] STEP, which is the full name of Standard for the Exchange of Product Model Data, has played an important role in geometric data interaction as a general text intermediate exchange format of three-dimensional CAD since it became an ISO international standard (ISO10303) in 1992. There is no precedent for directly starting from STEP file data in the mainstream 3D model classification method.

[0004] The graph convolution network (GCN) is a convolutional neural network that can directly act on a graph and utilize its structure information. The topological structure and geometric information in the STEP file naturally fit the edge line and node attribute in the graph data structure, so that the STEP file can be directly applied to the graph convolution network.

[0005] The full name of BERT is Bidirectional Encoder Representation from Transformers, which is a pre-trained language representation model. It emphasizes that the traditional unidirectional language model or the method of shallowly splicing two unidirectional language models is no longer used for pre-training, but a new masked language model (MLM) is used to generate deep bidirectional language representation. SUMMARY

[0006] In view of the problems in the prior art, the present application provides a 3D model classification method based on geometric semantic pre-training and a graph network, which aims to overcome the problems of sampling randomness and loss of geometric semantic information existing in the existing model classification method. The present application realizes the mapping of parts from a file to a model representation space by extracting the graph structure information and geometric semantic information in the STEP file, and then realizes the accurate classification of three-dimensional models through a clustering algorithm.

[0007] The object of the present application is achieved by the following technical solutions.

[0008] A 3D model classification method based on geometric semantic pre-training and graph network, comprising the following steps:

[0009] 1) STEP data input and initial feature extraction, which is divided into two parts, one part is the topological data of the three-dimensional model, which is converted into the adjacency matrix of the three-dimensional model surface after processing, and the other part is the surface information after arrangement;

[0010] 2) input the surface information into the geometric semantic pre-training model to obtain the word vector of each surface;

[0011] 3) input all surface word vectors and surface adjacency matrix into the graph semantic feature hybrid network to obtain the representation vector of the model as a whole, i.e. the model semantic feature;

[0012] 4) cluster the model semantic features by k-means algorithm, and the models corresponding to the obtained clustering results are of the same class.

[0013] In step 1), the STEP data input and initial feature extraction step is:

[0014] 1.1) first extract the data of the topological object in STEP, record the vertex set as V={v1, v2, …, v n}, the edge set as E={e1, e2, …, e m}, and the surface set as F={f1, f2, …, f l};

[0015] 1.2) according to the geometric topological relationship, extract the adjacency matrix A=(a ij ) 1×1， Wherein

[0016] a ij =1, when f i , f j There is at least one common edge e k ∈E

[0017] a ij =0, when f i , f j There is no common edge e k ∈E;

[0018] 1.3) according to the topological relationship, reorganize to obtain surface information FF, each surface information contains surface itself geometric data f a , and f a Surface associated edge data {e b , …, e c}, and f aCurved surface associated vertex data {v g , …, v h}.

[0019] In step 2), the geometric semantic pre-training model is constructed as follows:

[0020] 2.2) Collect a large number of unlabeled STEP data;

[0021] 2.2) Construct a pre-training model with BERT-BASE as the backbone, i.e. BERT model, using STEP files as samples, fine-tune the BERT model, and obtain sentence embedding vectors for each sentence;

[0022] 2.3) Construct a model based on SimCSE, use the sentence embedding vectors in step 2.2) as input, and perform unsupervised training, intending to use the model based on SimCSE to optimize the expression ability of the sentence embedding vectors;

[0023] 2.4) Divide the STEP files according to the described curved surface, input each content in order into the pre-trained BERT model and the model based on SimCSE in steps 2.2) and 2.3), obtain multiple sentence embedding vectors for each curved surface content, and finally use the average to obtain the embedding vector of the single curved surface information.

[0024] In step 3), the graph semantic feature hybrid network is constructed as follows:

[0025] 3.1) Design a graph semantic feature hybrid network model including graph convolution layer, MLP, residual connection and pooling layer;

[0026] 3.2) The hidden layer dimensions of all graph convolution layers are the same, and the input dimension of the convolution layer accepting curved surface semantic information is related to the batch number;

[0027] 3.3) The stacked modules in the graph are graph convolution, LayerNorm and participating connection, and the last MLP, where the MLP is 2 layers in total;

[0028] 3.4) The final model features will pass through a max pooling layer, and then output as the model feature vector.

[0029] Compared with the prior art, the advantages of the present application are that the problems of sampling randomness and loss of geometric semantic information existing in the prior model classification method are overcome, the mapping of the part from the file to the model representation space is realized by extracting the graph structure information and the geometric semantic information in the STEP file, and then the accurate classification of the three-dimensional model is realized through the clustering algorithm. Compared with the mainstream method using grid data, point cloud data or image, the STEP data (international standard format) is directly used, which can better represent the model itself while reducing the loss of model detail semantic information; the 3D model surface word vector is extracted by using the geometric semantic pre-training model; the model overall representation is obtained by using the graph convolution network to fuse the model surface adjacency graph information and the surface word vector. Compared with the traditional point cloud method, the method in the present application does not need sampling operation, the representation of each model is fixed, and the stability of classification is effectively increased; the overall representation of the 3D model is more accurately learned by using the graph convolution network to fuse the surface representation information, and the expression ability of the representation space is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The flowchart of the present application.

[0031] Figure 2 The schematic diagram of the graph semantic feature hybrid network model. DETAILED DESCRIPTION

[0032] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0033] A 3D model classification method based on geometric semantic pre-training and graph network, comprising the following steps:

[0034] 1) STEP data input and initial feature extraction, which is divided into two parts, one part is the topological data of the three-dimensional model, which is converted into the adjacency matrix of the three-dimensional model surface after processing, and the other part is the surface information after arrangement;

[0035] 2) input the surface information into the geometric semantic pre-training model to obtain the word vector of each surface;

[0036] 3) input all surface word vectors and surface adjacency matrices into the graph semantic feature hybrid network to obtain the overall representation vector of the model, i.e. the model semantic feature;

[0037] 4) the model semantic features are clustered by the k-means algorithm, and the clustering results corresponding to the models are the same class.

[0038] In step 1), the STEP data input and initial feature extraction step is:

[0039] 1.1) First, extract the data of the topological objects in STEP, record the vertex set as V = {v1, v2, …, v n}, the edge set as E = {e1, e2, …, e m}, and the face set as F = {f1, f2, …, f l};

[0040] 1.2) According to the geometric topological relationship, the adjacency matrix A = (a ij ) l×1 of the surface is extracted, where a ij = 1 when f i , f j have at least one common edge e k ∈Ea ij = 0 when f i , f j have no common edge e k ∈E;

[0041] 1.3) According to the topological relationship, the surface information FF is obtained by reorganizing, and each piece of surface information contains the geometric data f a of the surface itself, the edge data {e b , …, e c} associated with the surface f a , and the vertex data {v g , …, v h} associated with the surface fa.

[0042] In step 2), the geometric semantic pre-training model construction steps are as follows:

[0043] 2.3) Collect a large number of unlabeled STEP data;

[0044] 2.2) Construct a pre-training model with BERT-BASE as the backbone (hereinafter referred to as the BERT model), use a certain number of STEP files as samples (this method uses 5000 STEP files), fine-tune the BERT model, and obtain the sentence embedding vector of each sentence;

[0045] 2.3) Construct a model based on SimCSE, use the sentence embedding vector in step 2.2) as input, and perform unsupervised training, the intention is to use the model based on SimCSE to optimize the expression ability of the sentence embedding vector;

[0046] 2.4) Divide the STEP file according to the described surface, input each piece of content in order into the pre-trained BERT model and the model based on SimCSE in steps 2.2) and 2.3), obtain multiple sentence embedding vectors of each surface content, and finally use the average to obtain the embedding vector of a single surface information.

[0047] In step 3), the graph semantic feature hybrid network construction steps are as follows:

[0048] 3.2) The graph semantic feature hybrid network model is designed to include a graph convolution layer, an MLP, a residual connection and a pooling layer, as shown in Figure 2 .

[0049] 3.2) The hidden layer dimensions of all graph convolution layers are the same (256 in the embodiment), and the convolution layer accepting the surface semantic information is special, with an input dimension related to the batch quantity;

[0050] 3.3) The stacked modules in the graph are graph convolution, LayerNorm and the participating connection in cooperation, and the last MLP, wherein the MLP is 2 layers in total, with the two layer dimensions being [256, 1024] and [1024, 256] respectively in the embodiment;

[0051] 3.4) The final model features will pass through a max pooling layer, and then be output as a model feature vector, with a dimension of 256 in the embodiment.

[0052] In order to clearly and completely describe the technical solutions in the embodiments of the present application, a three-dimensional model in an actual test set is taken as an example, the category of which is a connecting block, and the present application is further described in detail. The three-dimensional model in the actual data set contains 20 categories and 20000 three-dimensional models.

[0053] The flowchart of the three-dimensional model classification method based on geometric semantic pre-training and graph convolution in the embodiments of the present application is shown in Figure 1 , and includes the following steps.

[0054] Step 1: Preprocess the STEP data;

[0055] Step 1-1: Obtain the adjacency matrix information [[0, 4], [0, 3], [0, 2], [0, 1], [1, 4], [1, 2], [1, 5], [2, 3], [2, 5], [3, 5], [3, 4], [4, 5]], which is represented in the form of a sparse matrix;

[0056] Step 1-2: Divide the surface data, and divide the content of each row according to a surface in one STEP file as follows:

[0057] Source file content example:

[0058] " ...

[0060] #70 = ADVANCED_FACE('X2X96F64EF651E04F554F53X0X', #69, #35,.F.) ;

[0061] #35 = PLANE('', #34) ;

[0062] #34 = AXIS2_PLACEMENT_3D('Plane Axis2P3D', #31, #32, #33) ;

[0063] #31 = CARTESIAN_POINT('Axis2P3D Location', (4594.5, -758.288210669, 1623.41050789)) ;

[0064] #32 = DIRECTION('Axis2P3D Direction', (0., 0.838670567945, -0.544639035015)) ;

[0065] #33 = DIRECTION('Axis2P3D XDirection', (-1., 0., 0.)) ;

[0066] #69 = FACE_OUTER_BOUND('', #64,.T.) ;

[0067] #64 = EDGE_LOOP('', (#65, #66, #67, #68)) ;

[0068] #65 = ORIENTED_EDGE('', *, *, #44,.F.) ; ...

[0070] #41 = VERTEX_POINT('', #40) ; ...

[0072] #39 = LINE('Line', #36, #38) ; ...

[0074] #40 = CARTESIAN_POINT('Vertex', (4552.5, -758.288210669, 1623.41050789)) ; ...

[0076] '

[0077] Example of single surface content after partitioning:

[0078]

[0079] #70 = ADVANCED_FACE('', (#69), #35,.F.) ;

[0080] #35 = PLANE('', #34) ;

[0081] #34 = AXIS2_PLACEMENT_3D('Plane Axis2P3D', #31, #32, #33) ;

[0082] #31 = CARTESIAN_POINT('Axis2P3D Location', (4594.5, -758.288210669, 1623.41050789)) ;

[0083] #32 = DIRECTION('Axis2P3D Direction', (0., 0.838670567945, -0.544639035015)) ;

[0084] #33 = DIRECTION('Axis2P3D XDirection', (-1., 0., 0.)) ;

[0085]

[0086] Step 2: input each surface information into the geometric semantic pre-training model, and obtain all corresponding surface word vectors,

[0087] All = tensor [[0.298017238, -0.142678749, -0.403173581, 0.00175128, 0.09177426, -0.43763554, -0.8014363,...], [-0.06224051, -0.33517849, 0.89017321, -0.23195508, 0.67013346, 0.11176514, 0.02147173,...]...]

[0088] Step 3: input the surface semantic information tensor and the adjacency matrix into the graph semantic hybrid network to obtain the model semantic features [-0.57351205, 0.00991702, 0.83409113, -0.37119078, -0.77581240, -0.06774036, 0.00884125,...]

[0089] ​​Step 4: Clustering the semantic features of all models by k-means algorithm, where k is the number of model types, obtaining clustering results C1, C2,..., Ck, and the models in each clustering result are of a uniform type.

[0090] The above is a detailed description of the embodiments of the present application in conjunction with the drawings. The specific embodiments are only used to help understand the method of the present application. For ordinary skilled in the art, according to the idea of the present application, the specific embodiments and application scope can be changed and modified, and should not be understood as limiting the present application, which falls within the protection of the present application.

Claims

1. A 3D model classification method based on geometric semantic pre-training and graph network, characterized in that Comprising the following steps: 1) STEP data input and initial feature extraction, which is divided into two parts, one part is the topological data of the three-dimensional model, which is converted into the adjacency matrix of the three-dimensional model surface after processing, and the other part is the surface information after sorting; 2) input the surface information into the geometric semantic pre-training model to obtain the word vector of each surface; 3) input all surface word vectors and surface adjacency matrix into the graph semantic feature hybrid network to obtain the representation vector of the whole model, i.e. the model semantic feature; 4) the clustering result obtained by clustering the model semantic feature corresponds to the same class of model; In step 1), the STEP data input and initial feature extraction step is: 1.1) First, extract the data of the topological objects in STEP, record the vertex set as }, the edge set as }, and the face set as }; 1.2) Extracting the adjacency matrix of the surface according to the geometric topological relationship wherein ； ； 1.3) reorganize the surface information FF according to the topological relationship, each piece of surface information contains the surface's own geometry data , and the edge data associated with the surface , and the vertex data associated with the surface ; In step 3), the graph semantic feature hybrid network construction step is as follows: 3.1) design the graph semantic feature hybrid network model including graph convolution layer, MLP, residual connection and pooling layer; 3.2) the hidden layer dimensions of all graph convolution layers are the same, and the input dimension of the convolution layer accepting surface semantic information is related to the batch number; 3.3) the stacked module in the graph is graph convolution, LayerNorm and participating connection, and the last MLP, wherein the MLP is 2 layers in total; 3.4) the final model feature will pass through a max pooling layer, and then output as the model feature vector.

2. The method of claim 1, wherein the method further comprises: In step 2), the geometric semantic pre-training model construction step is as follows: 2.1) collect a large amount of unlabeled STEP data; 2.2) construct a pre-training model with BERT-BASE as the backbone, i.e. BERT model, use STEP file as sample, fine-tune the BERT model to obtain sentence meaning embedding vector of each sentence; 2.3) construct a model based on SimCSE, use the sentence meaning embedding vector in step 2.2) as input, perform unsupervised training, and use the model based on SimCSE to optimize the expression ability of the sentence meaning embedding vector; 2.4) divide the STEP file according to the described surface, input each content in order into the pre-trained BERT model and the model based on SimCSE in steps 2.2) and 2.3), obtain multiple sentence meaning embedding vectors of each surface content, and finally use the average to obtain the embedding vector of the single surface information.

Images