APM Plugin Priority via ML Data Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Application Performance Management (APM) systems face challenges in automatically identifying and analyzing data from various sources due to the lack of explicit plugins for unknown data types, leading to incomplete monitoring and resource inefficiencies.
Innovation Solution
A system and method using machine learning procedures, including natural language processing and convolutional neural networks, to classify data received by APM agents, assign appropriate plugins, and store data based on classification, enabling efficient handling of unknown data types and reducing resource provisioning costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual configuration is used for each data type upon receipt, then data can be read and analyzed, but the system complexity and time consumption increase significantly
Solution Approach 1:
The system performs preliminary classification of data types using machine learning models before detailed analysis. The APM agent automatically detects and categorizes incoming data types, pre-configuring the appropriate plugins and analysis methods, thereby eliminating the need for manual configuration upon data receipt and significantly reducing configuration time while maintaining accurate data identification
Solution Approach 2:
The system implements self-service through automatic data type detection and classification mechanisms. The machine learning model autonomously identifies incoming data types, selects appropriate plugins, and configures analysis parameters without human intervention. This self-configuration capability resolves the contradiction by enabling rapid automated setup that maintains high identification accuracy while eliminating manual configuration time
2Adaptability or versatility
If all possible data types are pre-configured with plugins, then complete data coverage is achieved, but resource provisioning costs increase
Solution Approach 1:
The system implements a universal machine learning classification model that can handle multiple data types through a single platform. Instead of provisioning separate plugins for each possible data type, the ML model provides multi-functional capability by automatically classifying and routing different data types to appropriate processing pipelines, achieving comprehensive data type coverage while minimizing resource provisioning requirements
Solution Approach 2:
The system dynamically adjusts its configuration parameters based on the classified data type. Rather than maintaining static pre-configured plugins for all data types, the ML model changes system parameters (plugin selection, analysis methods, processing priorities) dynamically according to the detected data type. This parameter-based adaptability achieves full data type coverage while avoiding the resource overhead of pre-provisioning all possible plugins
3Measurement precision
If machine learning procedures are used for data classification, then unknown data types are accurately identified, but processing complexity increases
Solution Approach 1:
The system introduces a machine learning classification model as an intermediary layer between raw data ingestion and the plugin-based processing system. This ML intermediary automatically classifies incoming data types and translates them into appropriate plugin selections and processing configurations. While the ML component adds complexity, it acts as a smart mediator that simplifies the overall system by enabling automatic, accurate classification without requiring complex manual configuration or pre-provisioning of all possible data type handlers
Data Source
AI summary
An exemplary system, method, and computer-accessible medium for classifying data associated with an application performance management (“APM”) agent(s) deployed on a server(s), can include, for example, receiving first information related to data to be read by the APM agent(s) located at the server(s), generating second information by assigning a score to the first information using machine learning procedure(s), classifying the data based on the second information. A data plugin(s) associated with the data can be determined based on the classification. A determination can be made as to whether the data is labeled as a data type. The second information can be generated based on (i) a log(s) associated with the data, (ii) transaction data associated with the data, (iii) server metrics associated with the data, (iv) a time stamp(s) associated with the data, (v) infrastructure data associated with the data, (vi) domain knowledge, or (vii) environment knowledge.