A 3D shape segmentation method based on the fusion of voxel and grid representation
Through the combination of voxelized three-dimensional grid model and V-SegNet network, the accuracy and computational complexity problems of the three-dimensional shape segmentation method in the prior art are solved, and a more efficient three-dimensional shape segmentation effect is achieved.
Patent Information
- Application Number
- CN202210625850.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-02
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-02
AI Technical Summary
The existing three-dimensional shape segmentation method is difficult to take into account the connection and positional relationship between the various parts of the model when dealing with complex shapes, resulting in low segmentation accuracy and high computational complexity.
Using a method based on the fusion of voxel and mesh representation, the surface and internal characteristics of shapes are learned by using a voxelized three-dimensional mesh model, using the V-SegNet network combined with the voxel encoder-decoder architecture and voxel residual blocks. , reduces computing costs and improves segmentation performance.
It realizes more accurate three-dimensional shape segmentation, reduces the computational complexity, can effectively process complex shapes, and improves segmentation performance without losing information.
Smart Images

Figure CN115063554B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer graphics application, and more particularly to a three-dimensional shape segmentation method based on the fusion of voxel and grid representation modes. Background Art
[0002] In recent years, a large amount of 3D shape data has appeared on the Internet, and the segmentation of 3D shapes has become a fundamental task in shape analysis. These 3D shapes are usually acquired by depth sensors using 3D geometric representations in the form of polygonal meshes and point clouds. The goal is to segment the 3D shapes into meaningful (semantic) parts and label them. These labeled 3D objects have been shown to be crucial for many computer graphics applications, such as skeleton tracking, skeleton extraction, object detection, 3D object reconstruction, 3D object style transfer, etc. Existing 3D shape segmentation methods mainly rely on manually labeled geometric descriptors. A common idea for segmentation is to directly use manually labeled geometric descriptors to calculate the geometric features on each face, then cluster them into several groups in the feature space, and finally project the labels onto the 3D shape to obtain the final segmentation result. However, the 3D shape segmentation task remains challenging in many aspects, including the ambiguity between subtle creases and boundaries, the diversity of shape parts, and the integration of local and global features.
[0003] In order to overcome these challenges and improve the segmentation performance, two technical trends have been developed. One trend is to use multiple geometric descriptors to combine patch features on the model. The motivation is that it is better to describe the shape from different angles than to segment from a single aspect. However, it is still challenging to fuse multiple unrelated features or use them effectively. The other trend is to use learning-based techniques and prior knowledge to improve the segmentation results. In the algorithms proposed by previous researchers, these techniques are widely used, such as formulating the objective function as a conditional random field (CRF) model, using an extreme learning machine (ELM) as a classifier for initial segmentation, training a convolutional neural network (CNN) to generate a grid representation from low-level features, etc. However, the semantic information of each grid also requires the connection and position information between the various parts of the model. It is difficult to learn this information only through the feature descriptors of each grid. Therefore, some methods encode the grid model into voxels and apply 3D CNN to learn the position features. However, on the one hand, large-resolution voxels will bring high computational complexity, which may make the segmentation method unable to handle complex 3D shapes. On the other hand, small-resolution voxels will cause serious information loss, which prevents the segmentation method from achieving satisfactory performance.
[0004] Therefore, how to balance the connection and positional relationship between the various parts of the three-dimensional model, improve the accuracy of three-dimensional model segmentation, and reduce the complexity of data processing is an urgent problem that technical personnel in this field need to solve. Summary of the invention
[0005] In view of this, the present invention provides a three-dimensional shape segmentation method based on the fusion of voxel and grid representation, combining voxels and feature descriptors, using voxel grids to divide the three-dimensional shape grid into voxels before segmentation, and providing a feature descriptor containing the grid for each voxel, thanks to the use of voxel neural networks, taking the geometric features of the shape surface and the three-dimensional voxel data as input, considering the entire shape of the three-dimensional model, including the surface and the internal space, and obtaining a more satisfactory segmentation result; and in the three-dimensional voxelized model, since the voxels passed through the surface only occupy a small part of the voxel space, those voxels without feature vectors inside do not participate in the calculation of the loss function, thereby reducing the computational cost and promoting convergence.
[0006] In order to achieve the above object, the present invention adopts the following technical solution:
[0007] The three-dimensional shape segmentation method based on the fusion of voxel and grid representation includes the following steps:
[0008] Step 1: Collect 3D mesh model data;
[0009] Step 2: Calculate the geometric features of each face in the 3D mesh model, obtain the feature vector of each face, and voxelize the 3D mesh model; the face is generally a triangle;
[0010] Step 3: Take the average value of the patch feature vectors contained in each voxel of the three-dimensional mesh model as the feature vector of the voxel;
[0011] Step 4: construct a V-SegNet network and perform network training; the V-SegNet network consists of a voxel encoder-decoder architecture and a voxel residual block, and the main framework is a voxel-based encoder-decoder architecture;
[0012] Step 5: Input the feature vector of each voxel into the trained V-SegNet network, output the label probability distribution of each voxel, and use the label with the maximum value in the label probability distribution to mark the corresponding voxel to determine the final label value of the voxel;
[0013] Step 6: Label each facet on the 3D mesh model according to the final label value of the voxel. Segment the 3D mesh model based on the labeling result and apply the graph cut algorithm to output the labeled 3D shape. 3D shape segmentation essentially assigns a semantic label to all triangular faces on the 3D shape.
[0014] Preferably, in step 2, three artificially designed feature descriptors, namely Gaussian curvature, average geodesic distance and shape diameter function, are used to obtain geometric information contained in the three-dimensional shape and obtain feature vectors. Three feature descriptors are selected to characterize the shape from different angles, and the feature vectors are calculated using the selected descriptors on all grids. Different feature vectors can be regarded as different channels in a two-dimensional image. Gaussian curvature (GC) describes the curvature of the surface at each vertex; average geodesic distance (AGD) is obtained by calculating the average geodesic distance between a vertex and all other vertices; shape diameter function (SDF) is a scalar function defined on the surface, which calculates the domain diameter of the model at each vertex on the grid.
[0015] Preferably, voxelization is to project and discretize the feature vector of each facet on the three-dimensional mesh model onto the corresponding voxel; the specific process is:
[0016] Step 21: placing the three-dimensional mesh model in a bounding box, where the bounding box is a voxel grid of size M×N×K;
[0017] Step 22: determining a resolution of the voxel grid, and dividing the three-dimensional grid model into voxels according to the resolution;
[0018] Perform point sampling on each patch, find the voxel unit of each sampled point according to the coordinates, and finally find the intersection of the voxels of all points to obtain the voxelization result of the corresponding patch, thus completing the voxelization;
[0019] Step 23: For voxels penetrated by one or more surfaces, the average value of the eigenvectors of all surfaces is assigned to the corresponding voxel.
[0020] The 3D mesh model is placed within its bounding box, which is actually a voxel grid of size M×N×K. The resolution of the voxel grid is carefully selected and a balance is found between network size and performance; the resolution is set to 80×80×80 in the experiment. At this resolution, V-SegNet can obtain rich spatial context information without making the network too complicated, and the resolution of 80×80×80 can already divide all 3D mesh models in the experimental dataset (most of which contain no more than 30K faces) into sufficiently small voxels so that each voxel contains about one face; it is worth pointing out that when processing 3D mesh models with more faces, it is recommended to use a higher resolution. In practice, the input 3D shape with a large number of faces can also be simplified to a coarser shape before segmentation; for voxels penetrated by one or more faces, the average value of the face feature vector is assigned to the voxel, otherwise, the face that the voxel does not pass through in the internal space of the 3D shape will be set to zero. In the 3D voxelized model, the voxels penetrated by the surface only occupy a small part of the voxel space, and those voxels without feature vectors inside do not participate in the calculation of the loss function, which can reduce the computational cost and promote convergence; through the voxelization process, there is a three-channel voxel data of size 3×80×80×80 as the input of the V-SegNet network.
[0021] Preferably, the V-SegNet network includes a voxel basic block, a voxel residual block, a voxel downsampling block and a voxel upsampling block; the voxel basic block consists of a convolution layer, a batch normalization layer and an activation function, and the activation function adopts ReLU; the V-SegNet network uses a voxel residual block extended from a two-dimensional residual block, extending it from two dimensions to three dimensions, thereby learning the three-dimensional features of the voxel structure, and marking the three-dimensional features of the voxel cube in a voxel-to-voxel manner.
[0022] The processing process of the V-SegNet network described in step 5 is:
[0023] Step 51: Input the feature vector of each voxel, use 16 3×3×3 voxel basic blocks in the feature channel, and then use maximum pooling for downsampling. The obtained features are processed by three consecutive voxel residual blocks to double the number of feature channels and extract local information of the object;
[0024] The encoder uses voxel basic blocks to obtain scale-invariant local features and uses maximum pooling to pass the local features to the voxel residual block for processing; after each voxel convolution, batch normalization and activation function are used to accelerate learning;
[0025] Step 52: At the bottleneck layer, a voxel residual block is used to improve the learning performance of the V-SegNet network; first, the voxel basic block is used to extract lower-level features, and then the voxel residual block is used to short-circuit the features obtained by downsampling and the local features obtained by the encoder, and the feature values are output to compensate for the detail loss in the voxel basic block;
[0026] Step 53: The eigenvalue is used as the label of the corresponding voxel. The eigenvalue passes through two consecutive voxel residual blocks and two 1×1×1 voxel basic blocks to output the label probability distribution of each voxel. The label with the maximum value in the label probability distribution is selected as the final label value of the corresponding voxel to achieve the final label value prediction.
[0027] Preferably, in step 6, the final label value predicted by the V-SegNet network on each voxel is converted back to each face of the corresponding three-dimensional mesh model; if a face intersects with multiple voxels, a voting mechanism is used to determine the final label value of the face; and a graph cut algorithm is used to further smooth the boundaries between segments, and a smooth output labeled three-dimensional shape is obtained. The objective function of the graph cut algorithm is based on the smoothness of the boundary.
[0028] Preferably, the specific process of the graph cut algorithm segmenting the three-dimensional network model and outputting the marked three-dimensional shape is:
[0029] Step 61: establishing an objective function according to the final label value corresponding to the facet of the three-dimensional network model;
[0030] The objective function is expressed as:
[0031]
[0032] in,
[0033]
[0034] L represents the final label value corresponding to the face of the 3D network model; F represents the set of all faces on the 3D network model; G represents the set of any two adjacent faces; l f represents the segmentation label value of the face f; P(f|l f ) represents the label l of the patch f f The probability of θ(f,g) represents the dihedral angle between face f and face g; λ is a weight constant, and the optimal λ value for the three-dimensional mesh model is obtained from the neural network training samples, so as to ensure that the graph cut algorithm is applicable to the current three-dimensional mesh model data; ε is a very small probability value; e 1 and e 2 They represent two optimization items in the objective function, respectively, representing that the angle between the predicted label and the segmented patch should be as close to the concave edge as possible;
[0035] Step 62: Use the α-expansion algorithm to optimize and solve the objective function, and adjust the final label value of the patch;
[0036] Step 63: Segment the three-dimensional network model according to the final label value after the facet adjustment, and output a labeled three-dimensional shape.
[0037] Preferably, the three-dimensional linear data is rotated 90 degrees along the x, y, and z axes to expand it into shapes at multiple different positions, and the data is further enhanced to obtain enhanced three-dimensional shape data to train the V-SegNet network.
[0038] Preferably, the V-SegNet network training details are:
[0039] The weights in V-SegNet are initialized using a zero-mean Gaussian distribution with σ=0.001. To train all variants, V-SegNet is implemented in PyTorch and uses ADAM with a fixed learning rate of 0.005 and a beta range of (0.9, 0.99). We trained the variants for 100 epochs because overfitting occurred if more epochs were applied. The training set was shuffled before each epoch, and each mini-batch (4 shapes) was picked to ensure that each model was only used once in an epoch. To train V-SegNet, we used the cross entropy loss over all non-empty voxels in a batch as the objective function. The loss function L is defined as follows:
[0040]
[0041] where y n is the true label, is the predicted probability value on the true label in mini-batch n, N represents the size of the mini-batch, and H represents the coordinate set of the voxel model that is not empty.
[0042] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a three-dimensional shape segmentation method based on the fusion of voxel and grid representation, designs a voxel encoder-decoder architecture V-SegNet network, trains a deep neural network, voxelizes a given three-dimensional object, and then predicts the label of each voxel located inside the given shape, and finally generates the label information of each triangular face. By promoting the surface-based geometric features to closed voxels, and V-SegNet working in a surface-voxel-surface manner, the segmentation performance can be further improved. The voxel-based method can learn the internal spatial information of the shape, and can simultaneously consider the geometric features of the shape surface and its spatial information, so as to more accurately segment each part to obtain a more satisfactory segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0044] Figure 1 The accompanying drawing is a schematic diagram of a three-dimensional shape segmentation process based on the fusion of voxel and grid representation provided by the present invention;
[0045] Figure 2 The accompanying drawing is a schematic diagram of the V-SegNet network framework structure provided by the present invention;
[0046] Figure 3 The accompanying drawing is a schematic diagram of the V-SegNet network segmentation results under different training times provided by the present invention;
[0047] Figure 4 The accompanying drawings are schematic diagrams of some results of the training data set and the test data set provided by the present invention;
[0048] Figure 5 The accompanying drawing is a schematic diagram comparing the actual segmentation results provided by the present invention and the segmentation results of the existing algorithm;
[0049] Figure 6 The accompanying drawings are schematic diagrams of actual segmentation results provided by the present invention;
[0050] Figure 7 The accompanying drawing is a schematic diagram of the segmentation results provided by the present invention and the existing algorithm. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] The embodiment of the present invention discloses a 3D shape segmentation method based on the fusion of voxel and grid representation. A new voxel-based method is proposed, and a deep neural voxel convolutional network is used to segment the 3D shape. By voxelizing the 3D grid model, the component position information on the voxelized model and the feature descriptor information on the grid are used to explore clues from the surface and internal space.
[0053] Example
[0054] Various shapes of selected datasets (including the Princeton Segmentation Benchmark (PSB) dataset and the COSEG dataset) are used to compare the segmentation results of the method of the present invention with those of existing algorithms.
[0055] 1. Dataset.
[0056] The benchmark dataset used is intended to evaluate the proposed method and the algorithms proposed by previous researchers. It consists of two different datasets: Princeton Segmentation Benchmark (PSB) and Shape COSEG Dataset. The PSB dataset contains 380 3D mesh models in 19 categories, with approximately 20 models in each category. Three categories (bearings, machinery, and bust) are also omitted in the PSB dataset because they lack meaningful correspondence between segmented parts. In the COSEG dataset, the other four categories of models are used for comparative experiments, including 18 shapes in Iron, 20 shapes in Lamp, 28 shapes in Candelabra, and 44 shapes in Guitar. The categories used range from the simplest models (such as cups and tables) to more complex shapes (such as armadillos and humans). The PSB and COSEG datasets are divided into two parts. The detailed statistics of each category are shown in Table 1. All categories use 12 shapes as training sets, and the rest are used for testing.
[0057] Table 1 Statistics of training and testing shapes for each category of the dataset
[0058] category Number of training Number of tests Marking section Human 12 8 8 cup 12 8 2 Glasses 12 8 3 airplane 12 8 5 Ant 12 8 5 Chair 12 8 4 octopus 12 8 2 Chair 12 8 2 Teddy Bear 12 8 5 hand 12 8 6 pliers 12 8 3 fish 12 8 3 bird 12 8 5 Armadillo 12 8 11 vase 12 8 5 Four-legged centaur 12 8 6 Candlestick 12 16 4 Iron 12 6 3 Guitar 12 32 3 Lighting 12 8 3
[0059] The number of segmentation components also varies from category to category. Examples of models include Figure 4As shown in the figure. The model on the left is the training model, and the model on the right is the test model. In the same category, there are various types of objects, animals in various postures, and humans of different genders and ages. In each category, the segmentation experiment with different random initializations was repeated 3 times to calculate the average accuracy of the V-SegNet labeling results. The final accuracy of the method of the present invention is compared with the algorithms proposed by predecessors, including analogy-based and deep learning-based methods. It is worth noting that the comparison method is also improved by the graph cut algorithm.
[0060] The method of the present invention is implemented in both PyTorch and Matlab. The entire network is trained on an NVIDIA GeForce GTX2080 Ti GPU, and the Matlab framework is run on an Intel CoreTM i9 3.60 GHz CPU, 32GB RAM. It takes 2 hours to train a class of shapes, each of which contains 20K to 30K faces.
[0061] 2. Algorithm comparison
[0062] The method of the present invention is compared with the following six traditional and previous algorithms, including Kalogerakis et al., Wang et al., ShapePFCN, MeshCNN, Guo et al. and MeshWalker. Kalogerakis et al. simulated the objective function as a conditional random field (CRF) model to consider the consistency of patches and labels using multiple shape geometric descriptors. Guo et al. used a two-dimensional convolutional neural network to segment shapes by nonlinearly combining and compressing low-level geometric descriptors into a compact and efficient mesh representation. Wang et al. and ShapePFCN converted the three-dimensional shape into several two-dimensional projections and applied image-based methods to segment the mesh. MeshCNN applied edge convolution and pooling to segment the mesh. MeshWalker represents the three-dimensional shape with a sequence of multiple random walks along the surface and trains a recursive neural network to segment the three-dimensional mesh. The accuracy is calculated as:
[0063]
[0064] where a t is the area of triangle t, l t is the predicted segmentation label. g t (l t ) is a function that is equal to 1 if the prediction is correct and 0 otherwise.
[0065] Table 2 lists the statistical comparison of the segmentation accuracy of the proposed method and the algorithms proposed by the above-mentioned predecessors. It is obvious that the categories with the least number of annotated parts, such as cups, octopuses, and tables, have the highest accuracy among all categories because the meanings of different parts are clear. The accuracy of categories such as humans and hands is lower because they have the most ambiguity and more parts. It is more difficult to distinguish each finger than to distinguish the palm from the fingers, and the clear boundary between the upper arm and the lower arm is also difficult to distinguish. It can be seen that the accuracy of our proposed algorithm is comparable to the algorithms proposed by predecessors: Kalogerakis et al., Wang et al., ShapePFCN, Guo et al., MeshCNN, and MeshWalker. Among the algorithms proposed by predecessors, the algorithm of Guo et al. performed the best. In addition, our overall average segmentation accuracy is better than all other algorithms.
[0066] Table 2 Statistics of training and testing shapes for each category of the dataset
[0067]
[0068]
[0069] More comparisons between 3D shape segmentation results are shown in Figure 5 As shown in Figure 2. Observe that Guo et al. failed to separate the horse's ears from the head in these examples and mislabeled the area around the seagull's neck, while our proposed 3D shape segmentation algorithm was able to separate them well. Similarly, the fish's fins were not correctly segmented using Guo et al.'s method. This is because the feature vectors of these parts are similar to those of the seagull's body parts. Without learning the relative positional relationship between each part, it would be difficult to distinguish these faces using only feature descriptors.
[0070] Figure 6 A comprehensive comparison between the proposed method and the ground-truth labels is shown. It is clear that our method produces segmentation results that are very similar to the ground-truth, even on complex categories such as armadillos and four-legged centaurs.
[0071] Figure 7 A quantitative comparison of the results of the method of the present invention with six existing segmentation algorithms in the glasses, chair and pliers models is provided, including CoreExtra, FitPrim, Normalized cut, Randomized cut, Randomwalks and ShapeDiam. The results show that the segmentation part of the present invention is complete and meaningful, and is better than the other six compared algorithms.
[0072] This example demonstrates the effectiveness of using several geometric descriptors as mesh feature extractors. The input to V-SegNet consists of three heuristically designed geometric features, including Gaussian curvature (GC), average geodesic distance (AGD), and shape diameter function (SDF). Table 3 shows the average segmentation accuracy of 16 categories in the PSB dataset. The accuracy is highest when all three descriptors are used to generate the input of the network, while the accuracy is lower when no descriptor is used.
[0073] Table 3 Average segmentation accuracy of 16 categories in the PSB dataset
[0074]
[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0076] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A three-dimensional shape segmentation method based on the fusion of voxel and grid representation, characterized in that: The following steps are involved: Step 1: Collect 3D mesh model data; Step 2: Calculate the geometric features of each facet in the three-dimensional mesh model, obtain the feature vector of each facet, and voxelize the three-dimensional mesh model; Step 3: taking the average value of the patch feature vectors contained in each voxel of the three-dimensional mesh model as the feature vector of the voxel; Step 4: construct a V-SegNet network and perform network training; the V-SegNet network consists of a voxel-based encoder-decoder architecture and a voxel residual block; Step 5: Input the feature vector of each voxel into the trained V-SegNet network, output the label probability distribution of the voxel, and use the label with the maximum value in the label probability distribution to mark the corresponding voxel to determine the final label value of the voxel; Step 6: Mark each face on the three-dimensional mesh model accordingly according to the final label value of the voxel, segment the three-dimensional mesh model according to the marking result and use a graph cut algorithm to output a marked three-dimensional shape.
2. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: In the step 2, three feature descriptors, namely, Gaussian curvature, average geodesic distance and shape diameter function, are used to obtain the geometric information contained in each facet in the three-dimensional network model, and obtain a feature vector.
3. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: Voxelization is to project and discretize the feature vector of each facet on the 3D mesh model to the corresponding voxel; the specific process is: Step 21: placing the three-dimensional mesh model in a bounding box, where the bounding box is a voxel grid of size M×N×K; Step 22: determining a resolution of the voxel grid, and dividing the three-dimensional grid model into voxels according to the resolution; Perform point sampling on each patch, find the voxel unit of each sampled point according to the coordinates, and finally find the intersection of the voxels of all points to obtain the voxelization result of the corresponding patch, thus completing the voxelization; Step 23: For voxels penetrated by one or more surfaces, the average value of the eigenvectors of all surfaces is assigned to the corresponding voxel.
4. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: The V-SegNet network includes a voxel basic block, a voxel residual block, a voxel downsampling block and a voxel upsampling block; the voxel basic block consists of a convolutional layer, a batch normalization layer and an activation function, and the activation function adopts ReLU; The V-SegNet network uses a voxel residual block extended from a two-dimensional residual block to extend it from two dimensions to three dimensions, thereby learning the three-dimensional features of the voxel structure; The processing process of the V-SegNet network described in step 5 is: Step 51: Input the feature vector of each voxel, use 16 3×3×3 voxel basic blocks in the feature channel, and then use maximum pooling for downsampling, and the obtained features are input into three consecutive voxel residual blocks for processing; the encoder uses the voxel basic block to obtain scale-invariant local features, and uses maximum pooling to pass the local features to the voxel residual block for processing; after each voxel convolution, use batch normalization and activation function to accelerate learning; Step 52: At the bottleneck layer, first use the voxel basic block to extract lower-level features, and then use the voxel residual block to short-circuit the features obtained by downsampling and the local features obtained by the encoder to output feature values; Step 53: The eigenvalue is used as the label of the corresponding voxel. The eigenvalue passes through two consecutive voxel residual blocks and two 1×1×1 voxel basic blocks to output the label probability distribution of each voxel. The label with the maximum value in the label probability distribution is selected as the final label value of the corresponding voxel to achieve the final label value prediction.
5. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: The three-dimensional shape data is rotated 90 degrees along the x, y, and z axes to expand it into shapes at multiple different positions, and the data is further enhanced to obtain enhanced three-dimensional shape data to train the V-SegNet network.
6. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: In step 6, the final label value on each voxel predicted by the V-SegNet network is converted back to each face of the corresponding three-dimensional mesh model; if a face intersects with multiple voxels, a voting mechanism is used to determine the final label value of the face; a graph cut algorithm is used to further smooth the boundaries between segments, and a smooth output labeled three-dimensional shape is produced.
7. The three-dimensional shape segmentation method based on the fusion of voxel and grid representation according to claim 1, characterized in that: The specific process of the graph cut algorithm segmenting the three-dimensional network model outputting the marked three-dimensional shape is as follows: Step 61: establishing an objective function according to the final label value corresponding to the facet of the three-dimensional network model; Step 62: Use the α-expansion algorithm to optimize and solve the objective function, and adjust the final label value of the patch; Step 63: Segment the three-dimensional network model according to the final label value after the facet adjustment, and output a labeled three-dimensional shape.
