Joint Training Model Evaluation via Tagged Sample Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing joint training models face challenges in efficiently evaluating and optimizing model performance without exposing original data, particularly when participants lack tagged samples.
Innovation Solution
A method and apparatus for evaluating joint training models that generate model evaluation data using tagged sample sets, dividing them into training and test sets, and generating index change information over time, while ensuring data security by not sharing original samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If joint training model is used to solve data island and data privacy problems, then data security is improved, but model evaluation capability deteriorates
Solution Approach 1:
The patent introduces an intermediary mechanism (model evaluation data request and sample set matching system) that allows participants to evaluate model performance without directly sharing original data. The server acts as a mediator that receives evaluation requests, matches appropriate sample sets, and returns evaluation results while maintaining data privacy boundaries.
Solution Approach 2:
The patent uses tagged sample sets as copies or representations of the actual training data. Instead of sharing original sensitive data, participants work with matched sample sets that preserve the statistical properties needed for model evaluation while eliminating direct exposure to proprietary or sensitive information.
2Productivity
If participants share model evaluation data, then model optimization is improved, but data privacy protection deteriorates
Solution Approach 1:
The patent segments the model evaluation process into distinct components: model evaluation data requests, sample set matching, tagged data processing, and result generation. This segmentation allows each participant to contribute specific elements (like tagged samples) without exposing their complete data sets, enabling collaborative optimization while maintaining privacy boundaries.
Solution Approach 2:
The patent transforms the evaluation process by changing parameters from sharing raw data to sharing structured model evaluation data requests and tagged samples. This parameter transformation enables efficient model optimization through standardized interfaces while protecting underlying data privacy through the tagged sample mechanism.
3Measurement precision
If tagged sample sets are used for model evaluation, then evaluation accuracy is improved, but data exposure risk increases
Solution Approach 1:
The server acts as an intermediary that receives model evaluation data requests, performs sample set matching based on evaluation dimensions, and returns results without participants directly exchanging tagged samples. This intermediary mechanism enables accurate evaluation using tagged data while eliminating direct data exposure risks between participants.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a method and apparatus for evaluating a joint training model. A specific implementation of the method for evaluating a joint training model comprises: receiving a model evaluation data request sent by a target device (201), wherein the target device comprises a participant of a joint training model; acquiring a sample set matching the model evaluation data request (202), wherein the matching sample set is labeled data associated with the joint training model; and generating model evaluation data of the joint training model according to the matching sample set (203). By means of the implementation, an effect index of a joint training model can be shared on the premise of not exposing original sample data. Accordingly, a timely and effective data reference basis is provided for the optimization and improvement of the joint training model.