Edge Anomaly Detection Using Server-Trained Models and NPU Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing edge devices face limitations in memory and processing resources, constraining the size of training datasets and models, leading to lower quality models and longer runtimes, and lack user involvement capabilities in machine learning operations.
Innovation Solution
A machine learning deployment system enables users to create, refine, and deploy machine learning models on edge devices, supporting both users without models and those with existing models, using a neural processing unit (NPU) for inference and integrating models for non-traditional data types like images and audio, with iterative threshold adjustments for improved outlier detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models are deployed on edge devices with limited memory and processing resources, then the models can operate locally for anomaly detection, but the model quality and training efficiency deteriorate due to constrained dataset size and processing power
Solution Approach 1:
The patent segments the machine learning workflow into two distinct parts: model training is performed on a centralized server with sufficient computational resources, while model inference (anomaly detection) is executed on the edge device with limited resources. This segmentation allows each component to operate in its optimal environment, resolving the contradiction between local deployment capability and model quality.
Solution Approach 2:
The patent introduces a neural processing unit (NPU) as an intermediary component on the edge device that is specifically designed to accelerate neural network inference. This NPU acts as a bridge between the limited edge device resources and the demanding requirements of machine learning models, enabling high-quality model deployment without requiring full training capabilities on the edge device.
2Measurement precision
If larger training datasets are used to improve model quality, then detection accuracy improves, but the training time and computational resources required increase
Solution Approach 1:
The patent separates the time-consuming model training phase (performed on a centralized server with access to large datasets) from the rapid inference phase (performed on the edge device). This segmentation allows large datasets to be utilized for training without impacting the real-time performance requirements of anomaly detection, as training occurs separately and does not need to complete within the operational runtime constraints.
3Device complexity
If more processing resources are allocated to the edge device to improve model training capability, then model quality improves, but the device cost and power consumption increase
Solution Approach 1:
The patent extracts the resource-intensive model training function from the edge device and relocates it to a centralized server. This extraction allows the edge device to maintain minimal processing resources dedicated solely to inference operations, significantly reducing power consumption and device cost while still achieving high model quality through server-based training.
Solution Approach 2:
The neural processing unit (NPU) serves as an intermediary that enables efficient inference on the edge device without requiring full training capabilities. This specialized hardware component provides the necessary computational power for anomaly detection while consuming far less energy than would be required for complete model training on the edge device.
Data Source
AI summary
Techniques for detecting anomalies at an edge device integrated with a data intake system are disclosed. Sensor data captured by a set of edge devices is received at a system. The system is remote from the set of edge devices. A subset of the sensor data is selected based on a query. The machine learning model is trained to detect anomalies using the subset of the sensor data. After training the machine learning model, the machine learning model is deployed on the edge device. The machine learning model is executed at the edge device to detect one or more anomalies based on runtime sensor data captured at the edge device.