Processor Assignment Circuitry for Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image analysis systems face challenges in efficiently processing object detection and recognition across multiple cameras with varying object recognition tasks, leading to increased costs and processing time due to the need for multiple GPUs for different neural network models, resulting in frame drops and recognition failures.
Innovation Solution
An image analysis device and system that utilizes a processor assignment circuitry to allocate inference processors based on inference time and frequency of use for each neural network model, allowing for efficient object recognition across multiple cameras with different recognition tasks using a limited number of processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple GPUs are assigned to different neural network models for object recognition across multiple cameras, then object recognition accuracy is improved, but system cost increases significantly
Solution Approach 1:
A single GPU is designed to execute multiple different neural network models through dynamic model loading and switching mechanisms. The system can load different object recognition models (e.g., for detecting persons, vehicles, animals) onto the same GPU sequentially or concurrently, allowing one GPU to perform multiple recognition functions that previously required separate dedicated GPUs for each camera type.
Solution Approach 2:
The system dynamically assigns and switches between different neural network models on the GPU based on real-time requirements. When a camera needs to switch from detecting persons to detecting vehicles, the system dynamically loads the appropriate model onto the GPU, optimizing resource utilization while maintaining recognition accuracy for varying detection needs.
2Speed
If a fixed number of GPUs are assigned to neural network models, then processing speed is improved, but the system cannot adapt to varying recognition tasks across different cameras
Solution Approach 1:
The system implements dynamic model assignment where the same GPU can be assigned to different neural network models based on the current detection requirements. The control system monitors which objects need to be detected by which cameras and dynamically loads or switches models on the GPU accordingly, maintaining high processing speed while adapting to varying recognition tasks.
Solution Approach 2:
The system changes the operational parameters of the GPU by switching between different pre-trained neural network models. Each model is optimized for specific object types, and the system changes which model is active on the GPU based on the detection needs of different cameras, thereby adapting recognition capabilities without adding hardware.
3Reliability
If object recognition is performed for all detected objects in a frame image, then recognition completeness is improved, but frame drops occur when many objects are detected
Solution Approach 1:
The system applies different processing priorities to different objects detected in the same frame. Based on the object type, detection confidence level, and camera importance, the system selectively performs recognition on high-priority objects first. This allows the system to maintain recognition completeness for critical objects while processing frames efficiently, reducing frame drops by not uniformly processing all objects with the same computational resources.
Data Source
AI summary
An image analysis device has: an image analysis circuitry to analyze images input from each of cameras using instances of an image analysis program including a learned neural network model for object detection and learned neural network models for object recognition; inference processors to perform inference processes in the learned neural network model for object detection and each learned neural network model for object recognition; and a processor assignment circuitry to assign, from the inference processors, inference processors to be used for the inference process in the learned neural network model for object detection and the inference process in each learned neural network model for object recognition, based on inference time and frequency of use required for the inference process in each of the learned neural network models for object detection and object recognition included in each instance of the image analysis program.