Keypoint Detection System with Stability Module for Low Jitter

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing object and keypoint detection systems require significant processing resources and power, leading to high latency, instability due to spatial jitter, and limited applicability in resource-constrained environments like mobile devices.

Innovation Solution

A neural network-based keypoint detection system with stability modules and temporal filtering, which includes a keypoint detector, stability module, persistence detector, and filter system, to reduce noise and jitter, and optimize resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If existing object detection techniques are used, then detection reliability is improved, but processing resources and power consumption increase significantly

Engineering Contradiction:

Improvedetection reliabilityVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system segments the detection process into multiple stages: a lightweight initial detection stage that filters candidates, followed by a more computationally intensive verification stage only for promising candidates. This segmentation allows the system to achieve high detection reliability while minimizing overall power consumption by avoiding full processing of all image data.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies partial action by processing only the most relevant portions of the image data through intensive computation, while using lighter processing for the remaining data. The stability module applies temporal filtering only to keypoint trajectories rather than all image frames, reducing power consumption while maintaining detection reliability.

Inventive Principle:

Principle #16Partial or excessive action

2Measurement precision

If existing object detection techniques are used, then detection accuracy is improved, but processing time increases leading to high latency

Engineering Contradiction:

Improvedetection accuracyVSAvoidprocessing latency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary actions by using a fast initial detection model to identify potential objects and their locations before applying more complex verification and stability analysis. This preliminary filtering reduces the amount of data that requires intensive processing, thereby reducing latency while maintaining high detection accuracy through the subsequent verification stage.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the level of processing applied to different detected objects based on their confidence scores and temporal stability. High-confidence detections receive full processing, while low-confidence detections receive reduced processing, allowing the system to maintain accuracy for critical detections while reducing overall processing time and latency.

Inventive Principle:

Principle #15Dynamics

3Stability of the object's composition

If existing object detection techniques are used, then detection stability is improved, but spatial jitter increases causing instability

Engineering Contradiction:

Improvedetection stabilityVSAvoidspatial jitter

Core Design Contradiction:

Stability of the object's compositionVSMeasurement precision

Solution Approach 1:

The system implements feedback through the stability module that continuously monitors keypoint trajectories across multiple frames and adjusts detections based on temporal consistency. The system feeds back stability information to the detection process, rejecting detections that exhibit high spatial jitter while confirming stable detections, thereby improving detection stability without increasing spatial jitter.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system applies beforehand cushioning by using temporal filtering and stability analysis to compensate for and reduce spatial jitter before it affects final detections. The stability module anticipates and corrects jittery movements by analyzing patterns across multiple frames, cushioning against the harmful effects of spatial variation while maintaining detection accuracy.

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

4Reliability

If existing object detection techniques are used, then detection reliability is improved, but device complexity increases making it unsuitable for mobile devices

Engineering Contradiction:

Improvedetection reliabilityVSAvoidprocessing resource requirements

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system applies local quality by using different processing complexities in different parts of the pipeline: the initial detection uses lightweight operations suitable for mobile devices, while only the verification and stability analysis stages use more computationally intensive operations. This localized application of complexity allows high detection reliability while keeping overall device complexity manageable for mobile platforms.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS11847823B2Object and keypoint detection system with low spatial jitter, low latency and low power usage

Publication Date: 2023.12.19 APPLE INC

AI summary

Video object and keypoint location detection techniques are presented. The system includes a detection system for generation locations of an object's keypoints along with probabilities associated with the locations, and a stability system for stabilizing keypoint locations of the detected objects. In some aspects, the generated probabilities are two-dimensional array correspond locations within input images, and stability system fits the generated probabilities to a two-dimensional probability distribution function.