Cascade Neural Network Abnormality Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data analysis systems are unable to automatically generate rules for detecting abnormal behavior in data streams, particularly in video analysis, and are limited in handling large datasets with varying cluster sizes and flow rates, leading to inefficient abnormality detection.
Innovation Solution
A novel technique utilizing neural network modules trained in a cascade fashion, with a unique thresholding method based on statistical parameters and flow rates, to accurately identify abnormal data pieces by learning normal behavior patterns and adjusting thresholds for controlled alert generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If traditional rule-based systems are used for abnormality detection, then the system can detect specific predefined abnormal events, but the system cannot automatically generate detection rules and requires a priori knowledge of abnormal event types
Solution Approach 1:
The system performs self-learning by automatically generating detection rules from normal data without requiring external programming or a priori knowledge. The neural network modules learn normal behavior patterns autonomously and automatically identify deviations, enabling the system to serve itself in rule generation rather than requiring manual configuration
Solution Approach 2:
The patent replaces traditional mechanical rule-based systems with neural network modules that learn patterns through statistical analysis. Instead of manually programmed rules, the system uses machine learning algorithms to automatically detect abnormal patterns, substituting mechanical rule formulation with intelligent pattern recognition
2Measurement precision
If clustering is applied to large datasets with varying cluster sizes, then the system can identify normal behavior patterns, but it becomes difficult to accurately determine whether data pieces belong to learned clusters when cluster sizes differ significantly
Solution Approach 1:
The system dynamically adjusts detection thresholds based on the characteristics of each cluster rather than using fixed thresholds. The neural network modules adapt their decision boundaries according to the varying sizes and distributions of different clusters, allowing accurate classification whether clusters are large or small
Solution Approach 2:
The patent changes the parameters used for cluster evaluation by employing multiple neural network modules with different detection thresholds. Each module is trained to recognize patterns at different sensitivity levels, and the system adjusts which module's threshold applies based on the specific data characteristics, enabling accurate detection across varying cluster sizes
3Reliability
If multiple neural network modules are used in cascade to improve detection accuracy, then the system can reduce false alarms, but the complexity of training and operating multiple modules increases
Solution Approach 1:
The system segments the abnormality detection task into multiple specialized neural network modules, each responsible for detecting specific types of patterns or operating at different sensitivity levels. This segmentation allows each module to be optimized for its specific function while working together in cascade to provide comprehensive detection with reduced false alarms
Solution Approach 2:
The patent applies partial action by having multiple neural network modules operate in cascade where not all modules need to agree for an abnormality to be detected. The system uses a hierarchical approach where earlier modules filter obvious cases and later modules handle more subtle patterns, reducing the need for excessive computational effort while maintaining high reliability
4Productivity
If the system generates alerts for all detected abnormal data pieces, then comprehensive monitoring is achieved, but alarm overflowing occurs with excessive alert rates
Solution Approach 1:
The system dynamically adjusts alert generation based on the confidence level of detection and the characteristics of detected patterns. Rather than generating alerts for all detected abnormalities equally, the neural network modules prioritize alerts based on the significance and confidence of detection, dynamically controlling the alert rate to prevent overflow while maintaining comprehensive monitoring
Solution Approach 2:
The patent applies different detection thresholds and alert generation rules to different types of detected patterns rather than using a uniform approach. Critical abnormalities trigger immediate alerts while less critical patterns may be monitored without alerting, allowing the system to maintain comprehensive monitoring while generating alerts only when necessary to avoid alarm overflow
Data Source
AI summary
A system and method for use in data analysis are provided. The system comprises a data processing utility configured to receive and process input data, comprising: plurality of neural network modules capable for operating in a training mode and in a data processing mode in accordance with the training; a network training utility configured for operating the neural network modules in the training mode utilizing selected set of training data pieces for sequentially training of the neural network modules in a cascade order to reduce an error value with respect to the selected set of the training data pieces for each successive neural network module in the cascade; and an abnormality detection utility configured for sequentially operating said neural network modules for processing input data, and classifying said input data as abnormal upon identifying that all the neural network modules provide error values being above corresponding abnormality detection thresholds.