Big Data Anomaly Detection via Node Segmentation and Circulation Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data detection methods in big data processing links have a limited detection range, failing to effectively capture anomalies in the circulation process between processing nodes, which affects the anomaly detection rate and complicates the data processing link logic, increasing storage and computational requirements.
Innovation Solution
A data detection method that obtains and analyzes processing information from multiple nodes in a data processing link, determines circulation information, and collects data when event trigger conditions are met, expanding the detection range and improving anomaly detection by capturing events in both service processing and content circulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data detection is performed only on content processing situation inside processing nodes, then the detection logic is simple, but the detection range is small and anomaly detection rate is affected
Solution Approach 1:
The patent segments the data processing link into multiple processing nodes and further segments detection into two dimensions: processing information (what each node does) and circulation information (how data moves between nodes). This segmentation allows comprehensive detection without overwhelming complexity by organizing detection into structured, manageable components.
Solution Approach 2:
The detection method is designed to be universally applicable across different processing nodes and various types of data processing links. By creating a unified detection framework that can monitor both processing operations and circulation events across multiple nodes, the system achieves high anomaly detection rates without requiring node-specific complex logic.
2Reliability
If data detection expands to include circulation information between processing nodes, then the detection range increases, but the data processing complexity and storage requirements increase
Solution Approach 1:
The patent extracts only the essential circulation information needed for anomaly detection rather than processing all possible data. By selectively extracting key circulation events and processing information from multiple nodes, the system expands detection range while controlling data volume through focused, relevant data collection.
Solution Approach 2:
The detection system implements partial monitoring of circulation information by focusing on specific event trigger conditions rather than continuously analyzing all data flows. This approach achieves sufficient detection coverage without the excessive data processing burden of complete surveillance.
3Measurement precision
If multiple processing nodes are monitored for both processing and circulation information, then anomaly detection accuracy improves, but computational overhead increases
Solution Approach 1:
The system performs preliminary actions by pre-defining event trigger conditions and circulation patterns before actual anomaly detection begins. This allows the system to quickly compare incoming data against predetermined criteria, improving detection accuracy while reducing real-time computational overhead through efficient pattern matching.
Solution Approach 2:
The detection method incorporates feedback mechanisms where detection results from multiple processing nodes are aggregated and analyzed collectively. This feedback loop improves anomaly detection accuracy by cross-validating information across nodes while optimizing computational resources through coordinated analysis rather than independent processing.
Data Source
AI summary
Provided are a data detection method performed by an electronic device. The method includes: obtaining, in response to a detection trigger instruction, processing information of a plurality of processing nodes in a data processing link for a target content, and an event trigger condition, determining circulation information of the target content in the plurality of processing nodes using the respective processing information of the plurality of processing nodes; determining, when at least one of the processing information and the circulation information hits the event trigger condition, occurrence of a detection event corresponding to the event trigger condition in the data processing link; collecting data for the detection event to obtain detection data corresponding to the detection event; and displaying the detection event in a detection event region of a detection interface, the detection data in a data display region of the detection interface.