Parallel Inference Apparatus for Reducing Machine Learning Processing Time
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In object detection systems, the sequential computation of multiple machine learned models increases processing time, particularly in battery-powered devices like cameras, where processing speed is limited, leading to reduced accuracy and longer computation times.
Innovation Solution
The inference apparatus performs computations of multiple machine learned models in parallel, allowing for the initiation of subsequent models before the first has completed, and adjusts processing based on likelihood calculations at different time points to ensure accurate and efficient inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple machine learned models are computed sequentially, then computation accuracy is maintained, but processing time increases significantly
Solution Approach 1:
The system performs preliminary computation of subsequent machine learned models in parallel while the first model is still being processed. Intermediate results are calculated and stored beforehand, so when the first model completes, the second model can immediately continue from the intermediate state without waiting for sequential completion, thereby reducing total processing time while maintaining accuracy through checkpointing and resumption capability
2Measurement precision
If the number of machine learned models is increased, then detection accuracy improves, but computation amount and processing time increase
Solution Approach 1:
The computation of each machine learned model is divided into multiple stages or chunks, with intermediate results stored at each stage. This segmentation enables parallel processing where different stages of different models can be executed simultaneously, allowing the system to maintain high detection accuracy through ensemble learning while improving processing speed by avoiding sequential execution of all models
3Reliability
If successive computation is performed, then comprehensive inference is achieved, but computation time accumulates
Solution Approach 1:
The system implements continuous computation where multiple machine learned models are processed continuously in parallel rather than sequentially. Intermediate computations are maintained and resumed without interruption, ensuring that comprehensive inference is achieved while computation time is reduced through overlapping execution of multiple models
Data Source
AI summary
An inference apparatus includes a first machine learned model including a plurality of learners and outputting likelihoods of a plurality of classes, and a plurality of second machine learned models corresponding to the plurality of classes, performs computation of a second machine learned model corresponding to a first class selected from the plurality of classes based on likelihoods calculated in the middle of computation of the plurality of learners in the first machine learned model, in parallel with remaining computation of the plurality of learners in the first machine learned model, and in a case where a second class selected based on likelihoods when computation of the plurality of learners in the first machine learned model is fully completed is coincident with the first class, continues the computation of the second machine learned model corresponding to the first class, thereby outputting an inference result.