3D Mesh Feature Extraction Training with Patch Mask Embedding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D mesh model datasets have insufficient samples, leading to poor accuracy in feature extraction networks trained with limited annotated data, and manual annotation is inefficient and costly.
Innovation Solution
A training method that divides 3D mesh models into non-overlapping patches, uses mask embedding for feature encoding, and adjusts network parameters based on predicted and actual geometric and coordinate information to improve accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised training with cross entropy loss function is used, then the feature extraction network can be trained with limited annotated data, but the accuracy is poor due to insufficient 3D mesh model samples
Solution Approach 1:
The 3D mesh model is divided into multiple non-overlapping patches, where each patch represents a local region of the model. This segmentation allows the network to learn from localized geometric features more effectively, improving feature extraction accuracy even with limited annotated samples. The patches are processed independently and then aggregated to form the complete model representation.
2Quantity of substance
If manual annotation is used to increase training samples, then more training data can be obtained, but the process is inefficient and costly
Solution Approach 1:
The network uses mask embedding to automatically handle occluded regions without requiring manual annotation of these areas. The mask embedding serves as a learnable representation that enables the network to infer information about occluded parts from visible regions, eliminating the need for time-consuming manual annotation of complete 3D mesh models while still providing sufficient training data.
3Measurement precision
If complete 3D mesh models are processed, then comprehensive feature extraction can be achieved, but computational complexity increases
Solution Approach 1:
By dividing the 3D mesh model into non-overlapping patches, the computational complexity is reduced as each patch can be processed independently through the network. This segmentation strategy maintains comprehensive feature extraction capability while lowering the overall computational burden compared to processing the complete model as a single unit.
Solution Approach 2:
The network processes only the visible, non-occluded patches rather than attempting to process the entire model including occluded regions. This partial action approach reduces computational complexity while still achieving sufficient feature extraction for downstream tasks, as the mask embedding handles the occluded portions without requiring full model processing.
Data Source
AI summary
A training method, apparatus and system for a feature extraction network of a 3D mesh model are provided. The method includes dividing a training 3D mesh model into a plurality of patches which do not overlap with each other, dividing the plurality of patches into first-type patches and second-type patches, and using mask embedding as a feature encoding of each second-type patch; inputting geometric representation information and positional representation information of each first-type patch into a feature extraction network; determining predicted geometric representation information of each face based on a feature encoding of each first-type patch output from the feature extraction network, the mask embedding, and positional representation information of each second-type patch, and adjusting parameters of the feature extraction network based on differences between the predicted geometric representation information and geometric representation information of the each face.