3D Map Creation Excluding Moving Object Voxels

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current SLAM technologies fail to accurately form three-dimensional maps when input images contain moving objects, as they assume stationary objects, leading to incorrect map formation and the need for map recreation upon object movement.

Innovation Solution

An image processing device and method that restricts three-dimensional position recognition of pixels based on attached labels, dividing the image into regions for separate recognition and using weight coefficients to minimize errors, allowing for accurate map creation even with moving objects by excluding their voxels from the map.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If three-dimensional position recognition is executed for all pixels in the captured image, then the three-dimensional map can be formed, but the map becomes inaccurate when moving objects are included

Engineering Contradiction:
Improvethree-dimensional map accuracyVSAvoidmap correctness in dynamic environments
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The captured image is divided into multiple regions based on object type recognition results. Pixels are classified into different regions (e.g., stationary objects, moving objects, background) and three-dimensional position recognition is selectively applied to specific regions, excluding moving objects from map formation.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Different processing strategies are applied to different regions of the image. Stationary object regions undergo three-dimensional position recognition and contribute to map formation, while moving object regions are excluded or handled differently, ensuring local optimization of map accuracy.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If the three-dimensional map is recreated whenever an object is moved, then the map remains accurate, but the processing time and computational load increase

Engineering Contradiction:
Improvemap accuracyVSAvoidmap recreation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

Object type recognition is performed in advance on the captured image to identify moving objects before three-dimensional position recognition. This preliminary classification allows the system to selectively exclude moving objects from map formation, avoiding the need for complete map recreation when objects move.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the three-dimensional position recognition process based on real-time object detection results. When moving objects are detected, the system adaptively modifies the processing to exclude these regions, maintaining map accuracy without requiring full map recreation.

Inventive Principle:
Principle #15Dynamics

3Reliability

If object type recognition is performed for each pixel, then moving objects can be identified, but the computational complexity increases

Engineering Contradiction:
Improvemoving object detection accuracyVSAvoidprocessing system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The object type recognition system serves multiple functions: it identifies moving objects, classifies pixels into different regions, and provides guidance for selective three-dimensional position recognition. This multi-functional approach reduces the need for separate processing systems and minimizes overall computational complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS12014459B2Image processing device, image processing method, and program for forming an accurate three-dimensional map
Publication Date: 2024.06.18 SONY INTERACTIVE ENTERTAINMENT LLC
  • US12014459B2 patent drawing
  • US12014459B2 patent drawing
  • US12014459B2 patent drawing

AI summary

Provided are an image processing device, an image processing method, and a program capable of forming an accurate three-dimensional map even in a case where a moving object is included in an input image. The image processing device includes an image acquiring section that sequentially acquires two-dimensional captured images, an object type recognition executing section that attaches, to each pixel in the sequentially acquired captured images, a label indicating the type of an object represented by the pixel, and a three-dimensional map creating section that executes three-dimensional position recognition of each pixel of the captured images to create a three-dimensional map, on the basis of the sequentially acquired captured images, and the three-dimensional map creating section restricts the three-dimensional position recognition of each pixel of the captured images according to the label attached to the pixel.