Multi-Sensor 3D Skeleton Positioning via Orientation Classification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The accuracy of recognizing human posture and motion decreases due to occlusion, where parts of the person are hidden by objects or self-occlusion, when using a single distance sensor, leading to difficulties in recognizing complex motions.

Innovation Solution

A recognition system utilizing multiple distance sensors to acquire positional relations, provisionally classify object orientations, calculate likelihoods of sensor combinations, and classify postures based on these calculations to prevent accuracy drops due to occlusion, integrating data from multiple sensors to estimate three-dimensional skeleton positions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single distance sensor is used to recognize human posture and motion, then the device complexity is reduced, but the recognition accuracy decreases due to occlusion

Engineering Contradiction:

Improvenumber of sensorsVSAvoidrecognition accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent combines data from multiple distance sensors to compensate for occlusion. By merging the sensing results from several sensors positioned at different locations, the system can reconstruct complete skeleton information even when parts of the subject are occluded from any single sensor's view.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent transitions from single-sensor two-dimensional depth mapping to multi-sensor three-dimensional skeleton reconstruction. By adding spatial dimensions through multiple sensor positions, the system can infer skeleton positions that are occluded in any single sensor's view.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If multiple distance sensors are used to improve recognition accuracy, then the measurement precision increases, but the device complexity increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidnumber of sensors

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent performs preliminary classification of object orientation relative to each sensor before final skeleton extraction. By pre-classifying the orientation into discrete categories, the system reduces the computational complexity of processing multi-sensor data while maintaining high recognition accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the parameter representation from continuous orientation angles to discrete orientation classes. This parameter transformation simplifies the integration of multiple sensor measurements and reduces computational complexity while preserving the essential directional information needed for accurate skeleton recognition.

Inventive Principle:

Principle #35Parameter changes

3Loss of information

If multiple sensors are used to capture complete object information, then the loss of information due to occlusion is reduced, but the difficulty of detecting and measuring increases

Engineering Contradiction:

Improveinformation loss due to occlusionVSAvoidprocessing complexity

Core Design Contradiction:

Loss of informationVSDifficulty of detecting and measuring

Solution Approach 1:

The patent replaces complex mechanical integration of multiple sensor data streams with a classification-based computational approach. By substituting the measurement integration process with orientation classification and likelihood calculation, the system reduces information loss while simplifying the detection and measurement process.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces orientation classification as an intermediary step between raw sensor data and final skeleton extraction. This intermediate representation simplifies the integration of multiple sensor measurements by reducing the data to discrete orientation categories that are easier to process and combine.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Measurement precision

If provisional classification and likelihood calculation are performed for each sensor combination, then the recognition accuracy is improved, but the loss of time increases

Engineering Contradiction:

Improveposture classification accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs provisional classification into a limited set of discrete orientation categories rather than attempting continuous orientation estimation. This partial classification approach reduces computational time while providing sufficient accuracy for subsequent skeleton extraction, avoiding the need to process all possible orientation variations.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11620857B2Method, device, and medium for determining three-dimensional position of skeleton using data acquired by multiple sensors

Publication Date: 2023.04.04 FUJITSU LTD

AI summary

A recognition method executed by a processor, includes: acquiring positional relation between a plurality of sensors each of which senses a distance to an object; provisionally classifying an orientation of the object relative to each individual sensor included in the sensors into one of a plurality of classifications based on sensing data acquired by the individual sensor; calculating likelihood of each combination corresponding to the positional relation between the sensors based on a result of provisional classification of the orientation of the object relative to the individual sensor; and classifying the orientation of the object corresponding to each individual sensor in accordance with the calculated likelihood of each combination.