Secure Bidding Platform for AI Model Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current bidding systems require organizations to share their data with bidders, compromising security and fairness, as bidders must access proprietary data to develop solutions, which is not feasible or secure.
Innovation Solution
A secure and fair competitive bidding system that enables bidders to develop solutions without direct access to the organization's data by using a model-to-data paradigm, where bidders create and submit data science models to a secure cloud platform for evaluation against common metrics, ensuring data security and fairness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If bidders are given direct access to the organization's data to develop solutions, then the quality and accuracy of bidder solutions improve, but data security and confidentiality are compromised
Solution Approach 1:
The patent introduces an intermediary evaluation platform that acts as a mediator between the organization's data and the bidders. The platform receives data from the organization, evaluates bidder solutions against this data without exposing the actual data to bidders, and returns performance metrics. This intermediary system enables accurate solution evaluation while maintaining data security and confidentiality.
2Measurement precision
If the organization shares proprietary data with bidders for solution development, then bidder performance evaluation becomes more reliable, but fairness between bidders is compromised
Solution Approach 1:
The evaluation platform serves as a neutral intermediary that uniformly evaluates all bidder solutions against the organization's data without any bidder having direct access to the data itself. This ensures that all bidders operate under the same conditions and are judged by the same criteria, maintaining fairness while achieving reliable evaluation through centralized, consistent assessment.
Solution Approach 2:
The system segments the evaluation process into distinct phases: data submission by the organization, solution development by bidders using only challenge specifications, centralized evaluation by the platform, and result reporting to bidders. This segmentation prevents any single bidder from gaining unfair advantage through data access while maintaining evaluation integrity.
3Reliability
If bidders develop models without accessing the organization's data, then data security is maintained, but the ability to evaluate model performance accurately is reduced
Solution Approach 1:
The evaluation platform acts as an intermediary that receives trained models from bidders, evaluates them against the organization's proprietary data in a secure environment, and returns performance metrics without exposing the data to bidders. This enables accurate model assessment while maintaining data security throughout the process.
Data Source
AI summary
There are provided a system, a method and computer program product providing a data-to-model challenge platform for enabling bidders publicly build, test, evaluate, validate and optimize AI models on proprietary data while at the same time avoiding the need to grant them access to the data itself. Rather, the bidder develops analytics on the enterprise’s data to solve a task desirable to the organization and submits the analytics for evaluation against other bidders. The offering organization evaluates all submissions from the bidders and rank submissions against each other using the same metrics, wherein the metrics for selecting the winning bidder can include response time, number and rate of attempts to solve the analytics, quality of results, code compactness, team size etc. Allowing the submissions of the bidders to be visible only to the offering organization can occur subject to an agreement that the offering organization and the bidder sign.