Composite Image CNN Feature Extraction Without Crop Leakage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When performing object re-identification or classification using concatenated images in a CNN, information leakage occurs between image crops due to convolution operations, leading to increased processing time and reduced capacity for parallel computation.
Innovation Solution
A method involving a grid pattern to concatenate images, where convolutional filters are selectively applied to specific regions while ignoring others by nullifying weights using a grid pattern and binary masks, maintaining data separation and enhancing parallel computation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple image crops are concatenated into a composite image for batch processing in a CNN, then processing efficiency and throughput are improved, but information leakage occurs between crops during convolution, compromising data separation and analysis accuracy
Solution Approach 1:
The patent applies segmentation by dividing the composite image into distinct crop regions and introducing mask tensors that segment the convolutional filter weights corresponding to each region. This allows the CNN to process multiple crops in batch while maintaining data separation through selective weight masking, resolving the contradiction between batch processing efficiency and data separation integrity
Solution Approach 2:
The patent implements local quality by making the convolutional filter weights location-dependent through mask tensors. Different regions of the composite image (different crops) have different weight masks applied, so that each crop is processed with appropriate weights while preventing information leakage to other crops. This enables differentiated processing quality for different regions while maintaining overall batch processing efficiency
2Loss of information
If padding is applied to prevent information leakage between crops, then data separation is maintained, but processing time increases and the number of crops that can be combined is reduced
Solution Approach 1:
The patent extracts the data separation function from the traditional padding approach and implements it through mask tensors that selectively enable or disable weight applications. Instead of adding unnecessary padding data that must be processed, the mask tensors directly prevent information leakage by zeroing out inappropriate weight applications, thereby maintaining data separation without the time penalty of processing extra padded regions
Solution Approach 2:
The patent changes the parameter control mechanism from spatial expansion (padding) to weight parameter modification (masking). By dynamically adjusting which weight parameters are applied based on the mask tensors, the system maintains data separation without altering the spatial dimensions of the input data, thus avoiding the processing time increase that would result from handling larger padded images
3Loss of information
If the quantity of padding is increased to maintain data separation through more convolutional layers, then data separation is improved, but the maximum input dimensions constraint reduces the number of crops that can be combined
Solution Approach 1:
The patent applies segmentation by introducing mask tensors that divide the weight space into crop-specific regions. This allows each crop to be processed independently with its own weight mask, enabling more crops to be combined in the composite image without requiring excessive padding, thus increasing the number of crops that can be processed in batch while maintaining data separation
Solution Approach 2:
The patent transitions from solving the data separation problem in the spatial dimension (through padding that increases image dimensions) to solving it in the weight parameter dimension (through mask tensors). This dimensional shift allows more crops to be combined within the maximum input dimension constraints while maintaining data separation, as the masks operate in the weight space rather than requiring spatial expansion
Data Source
AI summary
A method for feature extraction of detected objects, comprising the steps of: receiving a plurality of images, each depicting an object detected by the object detecting application; concatenating the plurality of images into a composite image according to a grid pattern; feeding the composite image through a convolutional neural network (CNN) trained for feature extraction, wherein each convolutional layer of the CNN is configured to, while convolving input data to the convolutional layer using a convolutional filter: determine a currently convolved image of the plurality of images by determining a centre coordinate of a subset of the input data currently covered by the convolutional filter, and mapping the centre coordinate to the grid pattern; and selectively nullifying all weights of the convolutional filter that cover input data derived from any of the plurality of images not being the currently convolved image.