NLP Event Detection in Software Testing Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated software testing methods require defined criteria to identify events of interest, which is resource-intensive and slows down the onboarding process for software products.
Innovation Solution
The technology uses natural language processing to automatically detect events of interest in software testing by analyzing log files and telemetry events without the need for predefined criteria, allowing for efficient use of resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If predefined criteria are used to identify events of interest, then detection accuracy is improved, but resource consumption increases and onboarding time increases
Solution Approach 1:
The system automatically generates and refines event detection criteria through machine learning without requiring manual definition by users. The NLPS autonomously analyzes log files, identifies patterns, and creates detection rules, allowing the system to serve itself rather than requiring user intervention for criteria definition.
Solution Approach 2:
The system transitions from static predefined criteria to dynamic, automatically generated criteria that evolve based on analyzed data patterns. The machine learning model continuously refines detection parameters based on observed events, enabling adaptive detection accuracy without manual reconfiguration.
2Reliability
If predefined criteria are used to identify events of interest, then detection reliability is improved, but device complexity increases
Solution Approach 1:
The system replaces manual mechanical definition of detection criteria with automated machine learning processes. Instead of requiring users to manually create and maintain complex detection rules, the system uses AI models to automatically identify patterns and generate detection logic, simplifying the overall system architecture.
Solution Approach 2:
The machine learning model acts as an intermediary between raw log data and detection criteria. Rather than requiring direct manual translation of requirements into detection rules, the ML model processes and transforms data automatically, reducing the complexity burden on users while maintaining reliable detection.
3Reliability
If automated testing with defined criteria is implemented, then testing thoroughness is improved, but productivity decreases
Solution Approach 1:
The system performs preliminary analysis of log files and event data to automatically generate detection criteria before actual testing begins. By pre-identifying relevant event patterns through machine learning, the system prepares detection rules in advance, eliminating the need for manual criteria definition during the testing process and improving overall productivity.
Solution Approach 2:
The automated testing system combined with self-service criteria generation creates a feedback loop where testing results automatically refine detection capabilities. The system learns from tested events and automatically updates its detection criteria, making the testing process more efficient while maintaining thorough coverage without requiring repeated manual configuration.
Data Source
AI summary
A system and method for providing automated event of interest detection and reporting for software testing. The technology described herein uses natural language processing to identify events of interest based on natural language included in log files messages and/or telemetry event items of software under test. Identified events of interest are reported to one or more downstream services. In some examples, events of interest are mapped to action sequences of interest that can be replayed to reproduce events of interest without requiring event of interest criteria to be defined. In some examples, events of interest are reported to a health/status dashboard and/or a listener that responds to reported events of interest.