Query-Adaptive Sample Selection for Efficient Data Item Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network approaches for processing large numbers of data item samples, such as long video data, are either computationally expensive or limited by the number of samples they can process, often resulting in irrelevant or redundant data samples due to uniform downsampling or fixed subset selection, leading to variable performance and resource inefficiency.
Innovation Solution
Adaptive selection techniques using a selection neural network to generate relevance scores for both data item samples and placeholder samples, allowing for a dynamic selection of the most relevant subset to be processed by a task neural network, reducing unnecessary computation and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If uniform downsampling is applied to reduce computational cost, then processing speed improves, but data item sample relevance deteriorates
Solution Approach 1:
The patent changes the selection criterion from uniform random sampling to relevance-based sampling using selection scores. The selection neural network computes relevance scores for each data item sample based on the query, and samples are selected according to their scores rather than uniformly, thereby maintaining relevance while controlling the number of processed samples.
Solution Approach 2:
The patent introduces dynamic adaptability by allowing the number of selected data item samples to vary based on query complexity and relevance. Instead of a fixed downsampling rate, the system dynamically determines how many samples to process by comparing selection scores against a baseline, enabling flexible adjustment between processing speed and relevance based on specific query requirements.
2Device complexity
If a fixed size subset of data item samples is selected, then computational complexity is reduced, but task performance deteriorates
Solution Approach 1:
The patent implements a threshold-based selection mechanism where data item samples are selected if their selection score exceeds a baseline score. This partial selection approach processes only the necessary subset of samples above the threshold rather than a fixed size, balancing computational complexity reduction with maintaining sufficient task performance by including all potentially relevant samples.
Solution Approach 2:
The system changes from fixed-size subset selection to variable-size selection based on relevance scores. The number of processed samples becomes a dynamic parameter determined by query-sample relevance rather than a static configuration, allowing the system to adapt the subset size to actual task requirements while controlling computational complexity.
3Measurement precision
If all data item samples are processed, then accuracy is improved, but computational cost increases
Solution Approach 1:
The patent extracts and processes only the relevant data item samples by computing selection scores and filtering out irrelevant samples before processing. The selection neural network identifies and extracts the subset of samples that are actually relevant to the query, excluding irrelevant samples from the main task neural network processing, thereby reducing computational cost while maintaining accuracy on relevant data.
Solution Approach 2:
The system performs preliminary relevance assessment by computing selection scores for all data item samples before the main task processing. This preliminary filtering action identifies relevant samples in advance, allowing the main task neural network to focus computational resources only on relevant samples rather than processing all samples, thus reducing overall computational cost while preserving accuracy.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for receiving a query relating to a data item that includes multiple data item samples and processing the query and the data item to generate a response to the query. In particular, the described techniques include adaptively selecting a subset of the data item samples using a selection neural network conditioned on features of the data item samples and the query. Then processing the subset and query using a downstream task neural network to generate a response to the query. By adaptively selecting the subset of data item samples according to the query, the described techniques generate responses to queries that are more accurate and require less computation resources than would be the case using other techniques.