Automated Test Vector Generation from Technical Documentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of modern technical products with vast and highly technical features makes it difficult and laborious to check for errors in documentation, leading to frustrated customers, increased technical support costs, and impacted sales due to mismatches between product functionality and documentation.
Innovation Solution
Automated systems and methods for generating test vectors from technical documentation, including building a command template database, command vector table, and command template sequence graph, to identify valid inputs and generate test vectors for testing products.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated test vector generation is implemented, then productivity and documentation verification efficiency are improved, but device complexity increases due to the need for command template databases, vector tables, and sequence graphs
Solution Approach 1:
The system performs preliminary actions by pre-building command template databases, command vector tables, and command template sequence graphs from technical documentation before actual product testing. These pre-processed structures contain extracted commands, parameters, data types, and valid input ranges that are reused during test vector generation, eliminating the need to re-analyze documentation during testing and thereby improving productivity while managing complexity through upfront preparation
Solution Approach 2:
The patent introduces intermediary data structures (command template database, command vector table, and command template sequence graph) that mediate between the technical documentation and the test execution system. These intermediaries store processed command information and serve as a bridge that translates documentation into executable test vectors, reducing the direct complexity interaction between the documentation verification system and the testing framework
2Measurement precision
If comprehensive test vectors are generated to verify all product features, then measurement precision and documentation accuracy are improved, but loss of time increases due to the voluminous nature of technical documentation
Solution Approach 1:
The system replaces manual mechanical review processes with automated computational methods. Natural language processing algorithms automatically extract commands, parameters, and constraints from voluminous technical documentation, and computational algorithms generate test vectors based on command template sequence graphs. This substitution of manual analysis with automated processing maintains high measurement precision for documentation accuracy while dramatically reducing the time required to verify comprehensive product features
Solution Approach 2:
The system changes parameters by transforming unstructured documentation text into structured data representations with defined parameters including command templates, data types, valid input ranges, and parameter constraints. This parameter transformation enables precise measurement and verification while automating the extraction process to reduce time loss, as the structured parameters can be systematically processed and reused for test vector generation
Data Source
AI summary
Aspects of the present disclosure relate to systems and methods that help automate the generation of a test vector. Every functional product, be it a service, device, or combination thereof, has one or more documents associated with it. These documents may include such documentation as: (1) Release Notes; (2) Configuration Guides; (3) command line interfaces (CLIs)/application program interfaces (APIs); (4) Data Sheets; (5) Installation Guides; (6) User Manuals; (7) Errata notices; and (8) other documentation. In embodiments, command data-models and ranges and sequence of valid inputs to the data-model are extracted from natural language expressions in documentation related to a specific product. In embodiments, this extracted information is used to generate one or more table of attributes and properties or constraints about which a test vector may be generated for testing a command.