Noise Skeleton Sequence Training for Robust Behavior Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Behavior recognition models degrade in performance when faced with skeleton data distributions different from training data, particularly due to inaccurate joint extraction by skeleton extraction models, leading to misclassification and high annotation costs.

Innovation Solution

Train behavior recognition models using skeleton sequences that include noise, such as first noise from the extraction model and second noise based on motion statistics, to align with inference data distributions, reducing the performance gap.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If behavior recognition model is trained with manually annotated skeleton data (ground truth), then training accuracy is improved, but annotation cost and time increase significantly

Engineering Contradiction:

Improvetraining accuracyVSAvoidannotation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent uses skeleton extraction model outputs as copies of ground truth data for training the behavior recognition model. Instead of requiring manual annotation for every training sample, the system extracts skeleton data automatically using a pre-trained extraction model, creating synthetic training data that mimics the quality and distribution of manually annotated data.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The skeleton extraction model serves itself by using its own outputs as training data for the behavior recognition model. This self-service approach eliminates the need for external manual annotation labor, allowing the system to train autonomously on automatically extracted skeleton sequences.

Inventive Principle:

Principle #25Self-service

2Reliability

If behavior recognition model is trained with accurate manually annotated skeleton data, then recognition performance on training data is improved, but performance degrades on inference data with different distribution

Engineering Contradiction:

Improverecognition performance on training dataVSAvoidrecognition performance on inference data

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent changes the distribution parameters of the training data by using skeleton extraction model outputs that have similar characteristics to inference data. The extraction model's skeleton data includes natural variations and errors comparable to real-world inference conditions, effectively adjusting the training data distribution to match inference data distribution.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system performs preliminary training with skeleton data extracted by the extraction model before final behavior recognition. This preliminary action using extraction model outputs as training data prepares the behavior recognition model to handle the actual distribution of inference data, bridging the gap between training and inference environments.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If skeleton extraction model extracts joints with large motion variation, then motion capture capability is improved, but extraction accuracy decreases due to incorrect joint detection

Engineering Contradiction:

Improvemotion capture capabilityVSAvoidjoint extraction accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent converts the harmful effect of extraction model errors into a beneficial training characteristic. Instead of discarding skeleton data with extraction errors, the system uses these erroneous extractions as training data, teaching the behavior recognition model to be robust to extraction inaccuracies and handle real-world variability.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The system provides beforehand cushioning by training the behavior recognition model on skeleton data that already contains extraction errors and noise. This pre-exposure to imperfect data cushions the model against future inference errors, making the model more resilient to the natural variability and errors present in real-world applications.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Data Source

PatentUS20260030928A1Apparatus and method for behavior recognition based on noise skeleton sequence

Publication Date: 2026.01.29 SK TELECOM CO LTD

AI summary

In a method and apparatus for behavior recognition based on noise skeleton sequence, the method includes preparing a first skeleton sequence extracted by a skeleton extraction model from a first training image sequence and a behavior label of the first skeleton sequence, wherein the first skeleton sequence includes first noise caused by the skeleton extraction model; and training a behavior recognition model based on the first skeleton sequence and the behavior label of the first skeleton sequence.