Quality Assurance for Example-Based Systems via Input Space Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Example-based systems, such as neural networks, face challenges in quality assurance due to their opaque nature, lack of verification, and uncertainty in determining the number of examples needed across the input space, leading to potential differences between training and application datasets, which can impact their reliability and security, especially in safety-critical applications.
Innovation Solution
A method for quality assurance that involves determining and comparing quality ratings representing the coverage of the input space by example sets, using distribution-based mappings to guide targeted example acquisition, and employing statistical measures to assess and improve the quality of example sets, ensuring consistent and reliable knowledge bases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If example-based systems are used without verification and inspection, then the system development is simpler and faster, but the reliability and security of the system deteriorates
Solution Approach 1:
The patent applies preliminary action by performing quality rating determination and input space coverage analysis before the example-based system is deployed or further developed. The method calculates quality ratings for example sets and compares them to identify potential issues early in the development process, allowing verification without slowing down the overall development timeline.
2Reliability
If the number of examples collected across the input space is increased, then the coverage and reliability of the example-based system improves, but the time and resources required for example acquisition increases
Solution Approach 1:
The patent implements feedback by calculating quality ratings that represent the coverage of the input space by examples. These quality ratings are compared between different example sets to provide feedback on whether additional examples are needed and where gaps exist in the input space coverage, enabling targeted example acquisition rather than random collection.
Solution Approach 2:
The patent applies local quality by analyzing the distribution of examples across different regions of the input space. The quality rating comparison identifies specific areas where coverage is insufficient, allowing focused acquisition of examples in those local regions rather than uniformly increasing examples across the entire input space.
3Adaptability or versatility
If example sets are acquired in multiple iterative steps, then the acquisition process becomes more manageable and adaptable, but differences between example sets may occur that affect system consistency
Solution Approach 1:
The patent uses feedback by comparing quality ratings between example sets acquired at different iterative steps. This comparison provides feedback on whether the example sets are consistent in terms of input space coverage, allowing the acquisition process to be adjusted to maintain consistency while preserving the flexibility of iterative acquisition.
Data Source
AI summary
A quality assurance method for an example-based system improves quality assurance by creating and training the example-based system based on collected examples forming an example set. The respective example in the example set includes an input value in an input space. A first example set including a plurality of examples and a second example set including a plurality of examples are collected. A first quality rating representing coverage of the input space by the examples in the first example set is determined based on distribution of the input values in the input space. A second quality rating representing coverage of the input space by the examples in the second example set is determined based on distribution of the input values in the input space. The first and second quality ratings are compared to one another. A computer program and a computer-readable storage medium are also provided.