Embedded Device Object Detection Using Local Deep Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for video-based object detection on embedded devices with restricted resources are inefficient, requiring extensive hardware and compromising data privacy and security, especially in cloud-based scenarios where video data transmission and storage pose privacy and security risks.
Innovation Solution
Deploying deep learning models on resource-restricted embedded devices for autonomous object detection without server access, using models like object detection and tracking, re-identification, change detection, and segmentation models, which are trained locally and executed on devices like smartphones or camera-chipset-modules, ensuring data privacy and security by processing data offline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cloud-based object detection is used with backend servers, then processing power and accuracy are improved, but data privacy and security are compromised due to video data transmission and storage requirements
Solution Approach 1:
The patent extracts the object detection processing capability from the cloud server and implements it directly on the embedded device. The deep learning model is deployed locally on the device, eliminating the need to transmit video data to remote servers for processing, thus resolving the privacy and security concerns while maintaining detection accuracy
Solution Approach 2:
The patent creates a local copy of the deep learning model on the embedded device that replicates the cloud-based detection functionality. This local model copy enables the device to perform object detection autonomously without requiring continuous connection to remote servers, thereby eliminating data transmission risks
2Object-affected harmful factors
If deep learning models are deployed on embedded devices with restricted resources, then data privacy is improved by processing locally, but hardware resource constraints limit model performance
Solution Approach 1:
The patent optimizes the deep learning model parameters to be suitable for embedded devices with limited resources. This includes model quantization, pruning, and architecture modifications that reduce computational requirements while maintaining detection accuracy, enabling privacy-preserving local processing on resource-constrained hardware
Solution Approach 2:
The patent implements a simplified version of the full deep learning model that focuses on the most critical detection functions. By using a reduced model architecture that processes only essential video frames and uses minimal computational resources, the system achieves adequate detection performance on embedded devices without requiring extensive hardware capabilities
3Power
If video data is transmitted to backend servers for processing, then centralized processing capability is improved, but transmission time and network dependency increase
Solution Approach 1:
The patent pre-loads the deep learning model and necessary processing capabilities onto the embedded device before actual object detection is needed. This preliminary setup eliminates the need for real-time data transmission to servers, enabling immediate local processing and significantly reducing detection time while maintaining centralized model updates when needed
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In one aspect the present invention relates to method for object detection based on computer vision by processing video data of at least one camera (Ca) by using at least one processing unit (PU) of an embedded device (MD). The method comprises: - Providing a registration procedure for registering (S1) the embedded device (MD); - Upon receiving (S2) a trigger signal on the processing unit (PU) of the embedded device (MD) in reply to a successful execution of the registration procedure: - Connecting (S3) the embedded device (MD) to a basket structure (B) by means of attachment means (AM); - Executing (S4) an object detection algorithm (ODA) locally on the processing unit (PU) of the embedded device (MD) for automatically detecting objects in the basket structure (B) and providing (S5) a result dataset (rd), wherein executing (S4) the object detection algorithm comprises: - Receiving a video stream (S41) of the at least one camera (Ca) during aggregation of objects in the basket structure (B) via an image input interface (II) of the processing unit (PU); and in parallel: - Accessing (S42) a first storage on the embedded device (MD) for locally executing a trained object detection and tracking model based on the received video stream and - Accessing (S43) a second storage on the embedded device (MD) for locally executing a trained re-identification model and - Accessing (S44) a third storage on the embedded device (MD) for locally executing a change detection model and - Accessing (S45) a fourth storage on the embedded device (MD) for locally executing a movement direction classification model.