Random QUBO Generation for Quantum Annealer Benchmarking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users of quantum annealers face difficulties in comparing the performance of different vendors due to lack of transparent reporting of negative fidelity data and stochastic nature of annealers, making it challenging to determine the best solution for their specific problems.
Innovation Solution
A broker generates randomized QUBO problems that are distributed to various vendors for execution, allowing the broker to collect performance telemetry and provide it to customers for comparison, thereby enabling apples-to-apples comparison of vendor performance without vendors anticipating specific problems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If vendors report only positive results and features, then vendor marketing appeal is improved, but user ability to compare performance objectively deteriorates
Solution Approach 1:
The patent introduces a broker as an intermediary that collects performance data from multiple vendors, standardizes the data collection process, and presents comprehensive comparisons to users. This mediator ensures all vendors are evaluated using identical benchmark procedures, capturing both positive and negative performance aspects without requiring vendors to self-report selectively.
Solution Approach 2:
The system changes the parameter of data reporting from vendor-controlled selective disclosure to broker-controlled comprehensive measurement. By controlling the benchmarking process and data collection methodology, the system ensures consistent parameter measurement across all vendors, including fidelity metrics that may reveal negative performance aspects.
2Measurement precision
If users purchase annealing systems from each vendor to compare performance, then comparison accuracy is improved, but cost and complexity deteriorates
Solution Approach 1:
Instead of requiring users to acquire multiple physical annealing systems, the broker creates virtual copies of vendor systems through standardized benchmarking. Multiple vendors execute identical benchmark workloads on their respective systems, and the broker collects and compares the results, effectively copying the evaluation process across vendors without requiring users to physically possess multiple systems.
Solution Approach 2:
The broker platform serves multiple functions: it acts as a centralized collection point for benchmark data from numerous vendors, provides standardized comparison tools, and delivers comprehensive performance analysis to users. This universal platform eliminates the need for users to individually acquire and manage multiple vendor systems while maintaining accurate comparison capabilities.
3Manufacturing precision
If vendors reconfigure systems to perform well on specific QUBO problems, then performance on targeted problems is improved, but generalizability and fairness deteriorates
Solution Approach 1:
The broker distributes benchmark QUBO problems to vendors in advance of any vendor-specific optimization. Vendors execute these pre-distributed benchmarks on their standard configurations, ensuring comparison fairness. The preliminary distribution of standardized test cases prevents vendors from tailoring their systems to specific known problems, maintaining both fairness and generalizability in the evaluation.
Solution Approach 2:
The benchmarking system dynamically generates and distributes QUBO problems across multiple categories and difficulty levels. Rather than using static, predetermined test cases, the system employs dynamic problem generation that adapts to the vendor pool, ensuring that no single vendor can optimize for all possible problem types simultaneously. This dynamic approach maintains both fairness and assessment of generalizability.
Data Source
AI summary
One method includes receiving a request concerning resolution of a QUBO (quadratic unconstrained binary optimization) problem, randomly generating benchmark QUBOs, obtaining information, including telemetry and solutions, concerning execution of the benchmark QUBOs on each annealer in a group of annealers, and comparing respective performances of each of the annealers. Each of the QUBOs is associated with a respective QUBO matrix, and each QUBO matrix having the same size and density.