Image Processing Apparatus for Pose-Based Video Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current surveillance systems face challenges in accurately searching for moving images that include a desired scene, as they struggle to flexibly recognize and classify specific poses or actions of individuals, especially when parts of the body are hidden or when the desired state is unknown.
Innovation Solution
An image processing apparatus and method that acquires a series of frame images, detects keypoints in the skeletal structure of objects, computes feature values, and analyzes the direction of change in these values over time to improve search accuracy by using the computed direction of change as a key for searching moving images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional face-based search methods are used, then the search process is simple, but the search accuracy is insufficient when parts of the body are hidden or the desired state is unknown
Solution Approach 1:
The patent segments the human body into multiple keypoints (joints, limbs, head) and extracts skeletal structure information from images. This segmentation allows the system to search for poses and actions even when parts of the body are hidden, as the skeletal structure can be inferred from visible keypoints. The feature value computation unit processes each keypoint independently and combines them to form comprehensive pose descriptors.
Solution Approach 2:
The patent transitions from traditional 2D image-based search to incorporating temporal dimension by computing the direction of change in feature values over time. This adds a time-based dimension to the search key, enabling differentiation between static and dynamic poses, and improving search accuracy for action recognition by analyzing how keypoints move and change relative positions across multiple frames.
2Adaptability or versatility
If skeletal structure detection is implemented, then recognition flexibility improves, but computational requirements increase
Solution Approach 1:
The patent extracts only the essential skeletal structure information (keypoints and their relative positions) from full images, discarding unnecessary visual details. This extraction approach maintains high recognition flexibility for various poses and actions while significantly reducing computational energy requirements by processing only critical structural features rather than entire images.
Solution Approach 2:
The patent transforms image data into parameter-based representations by computing feature values for each keypoint (such as relative positions, distances, and angles between keypoints). This parameter transformation enables efficient computational processing while maintaining adaptability for recognizing diverse poses and actions, as numerical parameters are more computationally efficient than processing raw pixel data.
3Measurement precision
If direction of change computation is added to the search key, then search accuracy for dynamic scenes improves, but processing time increases
Solution Approach 1:
The patent computes the direction of change in feature values as part of the search key preparation process, performing this computation in advance during query processing. By pre-computing these temporal features for the query image sequence, the system enables efficient comparison with database entries without requiring intensive real-time computation during the actual search matching phase, thus improving accuracy for dynamic scenes while managing processing time.
Data Source
AI summary
The present invention provides an image processing apparatus (100) including: a query acquisition unit (109) that acquires a plurality of first frame images in time series; a skeletal structure detection unit (102) that detects a keypoint of an object included in each of a plurality of the first frame images; a feature value computation unit (103) that computes a feature value of the detected keypoint for each of the first frame images; a change computation unit (110) that computes a direction of change in the feature value along a time axis of a plurality of the first frame images in time series; and a search unit (111) that searches for a moving image by using the computed direction of change in the feature value as a key.