3D Feature-Embedded Semantic Segmentation for Image Components

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image object component-level semantic segmentation methods fail to accurately segment image components under varying lighting conditions, relative poses, and different surface materials and textures, as they do not effectively account for the three-dimensional nature of objects.

Innovation Solution

A method and device that acquire three-dimensional feature information of a two-dimensional image and perform component-level semantic segmentation by combining this information with two-dimensional feature information, using techniques such as convolutional neural networks and voxel models to enhance segmentation accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If three-dimensional feature information is integrated into the segmentation process, then segmentation accuracy is improved, but computational complexity and processing time increase

Engineering Contradiction:
Improvesegmentation accuracyVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by performing depth map generation and three-dimensional feature extraction before the main segmentation process. The depth information is pre-processed and integrated into the feature pyramid structure, allowing the segmentation network to receive enriched features without adding complexity during the core segmentation operation. This pre-computation of 3D features from 2D images and depth maps resolves the contradiction by preparing the computational work in advance.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements dimensionality change by transitioning from traditional two-dimensional image features to three-dimensional feature representations. By incorporating depth information and constructing 3D feature pyramids, the system adds a third dimension to the feature space, enabling more accurate segmentation while managing computational complexity through efficient 3D convolution operations and feature fusion strategies.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Measurement precision

If multiple feature types (2D and 3D) are combined for segmentation, then segmentation accuracy is improved, but processing time increases

Engineering Contradiction:
Improvesegmentation accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies merging by integrating 2D image features and 3D depth features into a unified feature pyramid structure. The multi-scale feature fusion network combines features from different sources and scales simultaneously, allowing parallel processing of multiple feature types. This fusion approach improves segmentation accuracy while managing processing time by consolidating operations into a single integrated network rather than sequential processing.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The system performs preliminary extraction and preparation of both 2D and 3D features before the main segmentation task. Depth maps are pre-computed from images, and feature pyramids are pre-constructed at multiple scales, enabling the segmentation network to operate with enriched features without incurring excessive processing delays during the critical segmentation phase.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS10740897B2Method and device for three-dimensional feature-embedded image object component-level semantic segmentation
Publication Date: 2020.08.11 BEIHANG UNIV
  • US10740897B2 patent drawing
  • US10740897B2 patent drawing
  • US10740897B2 patent drawing

AI summary

Embodiments of the present invention provide a method and a device for three-dimensional feature-embedded image object component-level semantic segmentation, the method includes: acquiring three-dimensional feature information of a target two-dimensional image; performing a component-level semantic segmentation on the target two-dimensional image according to the three-dimensional feature information of the target two-dimensional image and two-dimensional feature information of the target two-dimensional image. In the technical solution of the present application, not only the two-dimensional feature information of the image but also the three-dimensional feature information of the image are taken into consideration when performing the component-level semantic segmentation on the image, thereby improving the accuracy of the image component-level semantic segmentation.