Software Test Tagging Through Failure Analysis and NLP
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of tagging software tests is time-consuming and requires substantial effort whenever source code or tests are revised, updated, or edited, especially in collaborative software development environments where complex software is being developed.
Innovation Solution
A method and system that utilizes natural language processing and artificial intelligence to automatically generate tags for software tests based on failure analysis, leveraging previous executions and regression analysis to systematically and continuously revalidate tags for new or updated source code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual tagging of test corpuses is performed, then tags can identify concepts and relationships in source code, but the process is time-consuming and requires substantial effort
Solution Approach 1:
The system performs self-service by automatically generating tags through execution analysis without requiring manual human intervention. The computing device autonomously executes test cases, analyzes failures, and generates tags based on the analysis results, eliminating the need for manual tagging while maintaining high accuracy.
Solution Approach 2:
The manual mechanical process of tagging is replaced with an automated computational system. Instead of manually analyzing and tagging test corpuses, the system uses computing devices to automatically execute tests, analyze failures, and generate tags through programmed algorithms, substituting human effort with automated processing.
2Reliability
If tags are continuously updated with each source code revision, then tags remain current and accurate, but the effort required increases substantially
Solution Approach 1:
The system implements continuous automatic tagging by executing test cases and generating tags whenever source code changes occur. This continuous process ensures tags remain current and accurate without requiring manual intervention at each revision stage, maintaining reliability while improving productivity through automation.
Solution Approach 2:
The system uses feedback from test execution results to automatically update tags. When test cases are executed and failures are detected, the system analyzes the failure information and generates or updates tags accordingly, creating a closed-loop feedback mechanism that keeps tags current without manual effort.
3Reliability
If extensive testing is performed on new source code, then errors are detected early, but the time and resources required increase
Solution Approach 1:
The system performs preliminary error detection by automatically executing test cases against new source code before formal release. Test cases are prepared in advance and automatically run when source code is submitted, detecting errors early in the development cycle without requiring extensive manual testing time.
Data Source
AI summary
Tagging a corpus of tests based upon failure analysis. Access tags associated with test cases known to execute without error against an original corpus of source code. Access a test set associated with the accessed tags, the test set containing test cases. Access a new corpus of source code to be tested. Execute the new corpus of source code using test cases from the accessed test set. Access errors generated by execution of the new corpus of source code using the test cases. Determine locations in the new corpus of source code associated with the accessed errors. Execute natural language processing software to generate new tags associated with the locations in the new corpus of source code associated with the accessed errors.