Cross-Modality Person Re-Identification Using Dual-Stream ResNet50
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing person re-identification technologies face challenges in accurately identifying individuals across different viewing angles, postures, and modalities, particularly when combining visible light and infrared images due to differences in imaging characteristics and feature representation.
Innovation Solution
A cross-modality person re-identification method based on local information learning using a dual-stream ResNet50 convolutional neural network architecture with self-attention and local feature joint learning modules, which enhances specific and shared feature extraction between RGB and infrared images through data enhancement, self-attention learning, and loss function training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cross-modality person re-identification is performed by combining visible light and infrared cameras, then identification capability in limited conditions or dark environments is improved, but identification accuracy deteriorates due to huge differences in imaging characteristics and feature representation between the two modalities
Solution Approach 1:
The patent divides the feature extraction process into two separate streams: one for RGB images and one for infrared images. Each stream independently extracts features specific to its modality, avoiding direct comparison of incompatible features. The self-attention learning module further segments the feature extraction by paying differential attention to different regions and features within each modality, capturing modality-specific characteristics while maintaining separate feature representations throughout the network.
2Quantity of substance
If RGB images with three channels containing color information are used, then feature richness is improved, but compatibility with infrared images having only one channel deteriorates
Solution Approach 1:
The patent designs a universal dual-stream network architecture that can process both RGB images (three channels) and infrared images (one channel) through the same framework. The self-attention learning module provides a modality-agnostic mechanism that adapts to the specific characteristics of each input type, making the system universally applicable to both modalities while preserving their unique features. The separate processing streams ensure each modality's specific characteristics are maintained while enabling cross-modality re-identification.
3Reliability
If different wavelength ranges are used for visible light and infrared images, then modality-specific information is preserved, but feature alignment and comparison between modalities becomes more difficult
Solution Approach 1:
The patent introduces a new dimension of processing by applying self-attention mechanisms that operate independently of the original image wavelength characteristics. Instead of trying to align features in the traditional spatial and spectral dimensions, the self-attention module creates an additional attention-weighted feature space where correlations can be learned across modalities without requiring direct wavelength alignment. This transforms the problem from one of feature alignment to one of attention-based feature correlation.
Data Source
AI summary
Disclosed is a cross-modality person re-identification method based on local information learning, the method comprising the following steps: acquiring a standard data set and performing data enhancement on the standard data set; dividing the enhanced standard data set into a training set and a test set; constructing a cross-modality person re-identification training network based on a dual-stream ResNet50 convolutional neural network architecture; inputting the training set into the cross-modality person re-identification training network to obtain a cross-modality person re-identification test network through training; randomly selecting an image to be queried from the test set, and inputting the image to be queried and a candidate database from the test set into the cross-modality person re-identification test network to obtain an identification accuracy value corresponding to the image to be queried.