Trusted Rating Function for AI Model Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In communication networks, there is a lack of a trusted and fair rating system for AI/ML models and analytics services, particularly in multi-vendor scenarios, where consumers struggle to select the best-performing models and services, and producers face challenges in improving their offerings without biased ratings.
Innovation Solution
A Trusted Rating Function (TRF) is introduced to manage and store ratings, ensuring only genuine consumers can rate AI/ML models and services, preventing self-rating by producers, and providing a rating format that includes key usage information, allowing consumers to select the best-performing models and services based on aggregated, weighted ratings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple AI/ML models and analytics services from different producers are available, then consumers have more choices and competition drives improvement, but it becomes difficult to select the best-performing model due to lack of trusted rating information
Solution Approach 1:
The patent introduces a Trusted Rating Function (TRF) as an intermediary entity that collects, verifies, and aggregates ratings from multiple consumers about different AI/ML models and analytics services. This mediator resolves the information asymmetry by providing centralized, trusted rating information that helps consumers make informed selections among multiple producers without requiring them to independently evaluate each model's performance
Solution Approach 2:
The system implements a feedback mechanism where consumers provide ratings about their experiences with analytics services, and this feedback is aggregated by the TRF to create composite ratings. The feedback loop enables continuous improvement as producers can see how their models are rated and consumers can see aggregated performance information to guide their selections
2Reliability
If producers want to improve their AI/ML models based on rating information, then model quality can be enhanced, but self-rating by producers would create biased and untrusted ratings
Solution Approach 1:
The Trusted Rating Function acts as an independent intermediary that verifies consumer identities and validates that ratings come from genuine users who have actually consumed the service. The TRF prevents producers from submitting their own ratings by verifying consumer credentials and checking that the rater is not the service provider, thus eliminating self-rating bias while still allowing producers to access aggregated rating information for improvement
Solution Approach 2:
The system enables genuine consumers to self-serve by submitting their own ratings about services they have used. The TRF automatically verifies consumer identities and processes these self-submitted ratings without requiring producer involvement, ensuring that the feedback comes from independent users rather than the producers themselves
3Reliability
If a centralized rating system is implemented to ensure trust and fairness, then rating reliability improves, but system complexity and infrastructure requirements increase
Solution Approach 1:
The Trusted Rating Function is designed as a multi-functional entity that can be co-located with existing network repositories such as NRF (Network Repository Function) or UDM/UDR. By making the TRF universal and compatible with existing infrastructure components, the system achieves centralized rating management without requiring completely new standalone infrastructure, thus reducing overall system complexity while maintaining reliability
Data Source
AI summary
A trusted rating function in a communication network system obtains at least one verification information associated with at least one of an analytics function identifier, a service identifier and a service consumer identifier, receives, from a service consumer, rating information related to at least one rated service and consumer verification information associated with the service consumer, accepts the rating information based on a comparison between the obtained verification information and the consumer verification information, and updates a rating stored for the rated service based on the rating information.