Graph Convolutional Network for Person Re-Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current person re-identification methods, especially video-based, face limitations in computational efficiency and memory usage due to the requirement of 3D convolutions for capturing spatial and temporal information effectively.
Innovation Solution
The method involves training a convolutional neural network with a graph convolutional neural network as a parallel branch during training, using skeleton representations to form spatial-temporal graphs, which process keypoint features across images, allowing for more efficient person re-identification by leveraging spatial and temporal information without the need for extensive computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If 3D convolutions are used to capture spatial and temporal information simultaneously, then re-identification performance is improved, but computational resources and memory usage increase
Solution Approach 1:
The patent segments the feature extraction process into two distinct components: a convolutional neural network for spatial feature extraction from individual frames, and a graph convolutional neural network for temporal relationship modeling across keypoint sequences. This segmentation allows each component to be optimized independently, reducing overall computational burden compared to monolithic 3D convolutions while maintaining the ability to capture both spatial and temporal information necessary for accurate re-identification
2Reliability
If 3D convolutions are used to capture spatial and temporal information simultaneously, then re-identification performance is improved, but memory usage increases
Solution Approach 1:
The patent divides the processing architecture into separate spatial (CNN) and temporal (GCN) components, allowing memory resources to be allocated and managed independently for each function. The CNN processes individual frames with moderate memory requirements, while the GCN processes extracted keypoint sequences with optimized graph structures, avoiding the exponential memory growth associated with full 3D convolutional operations
Solution Approach 2:
The patent extracts only the essential temporal information from video sequences by identifying and processing keypoint features at critical moments, rather than processing all temporal data through computationally intensive 3D convolutions. This extraction approach reduces memory usage by focusing computational resources on the most informative temporal patterns while discarding redundant information
3Productivity
If graph convolutional neural network is used to process spatial-temporal graphs, then computational efficiency is improved, but device complexity increases
Solution Approach 1:
The patent segments the complex graph convolutional processing into manageable stages: keypoint detection from individual frames, graph construction from detected keypoints, and sequential graph convolution operations. This segmentation transforms an otherwise intractably complex monolithic operation into a series of simpler, more efficient steps that can be implemented with standard computational resources
Data Source
Figure 1
Figure 2A~2C
Figure 3
AI summary
A method for training a convolutional neural network to deliver an identifier of a person visible on an image when the image is inputted to the convolutional neural network, the method comprising: a. obtaining a training dataset including images of a person and skeleton representations, b. obtaining feature maps from the convolutional neural network, c. extracting features using the skeleton representations, d. forming graphs using the features and processing them in a graph convolutional neural network, e. calculating a loss, f. jointly training the two neural networks.