Document Data Evaluation in Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models lack sufficient methods to derive the contribution value for each document data when using a group of document data as input, particularly in interpreting the classification results, as seen in techniques like LIME, which are insufficient for document data groups.
Innovation Solution
An information processing apparatus and method that derives evaluation values for each document data within a document data group by using a machine learning model, allowing for the specification of display targets and orders based on these values, and updates the model using a loss function calculated from correct answer data and output data for training document data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a machine learning model uses a document data group including multiple document data as input, then the model can process and analyze multiple documents simultaneously, but it becomes insufficient to derive the degree of contribution for each individual document data
Solution Approach 1:
The patent segments the overall evaluation into two levels: first evaluating each individual document data separately to obtain document-level evaluation values, then evaluating combinations of document data to obtain interaction-level evaluation values. This segmentation allows the system to maintain both processing efficiency (by handling documents in manageable units) and contribution information (by capturing both individual and interactive contributions).
Solution Approach 2:
The patent implements a nested evaluation structure where individual document evaluations are nested within combination evaluations. The document-level evaluation values serve as foundational components that are then combined and re-evaluated in pairs or groups. This nested approach enables the system to derive contribution information at multiple levels of granularity while maintaining computational feasibility.
2Measurement precision
If the model derives evaluation values for each document data individually, then the interpretability improves, but the computational complexity increases
Solution Approach 1:
The patent applies partial action by focusing evaluation efforts on the most relevant document combinations rather than exhaustively evaluating all possible combinations. The system prioritizes deriving evaluation values for document pairs or small groups that have the most significant interaction, thereby achieving sufficient interpretability without the full computational burden of exhaustive combination evaluation.
Solution Approach 2:
The patent changes the evaluation parameters dynamically based on the combination of document data. Instead of using a fixed evaluation approach for all documents, the system adjusts evaluation parameters when assessing document combinations, allowing for more precise measurement of interaction effects while adapting computational resources to the specific evaluation needs of each document group.
Data Source
AI summary
An information processing apparatus includes at least one processor, in which the processor is configured to: for a machine learning model that uses a document data group including a plurality of document data as input and outputs output data, derive an evaluation value in the machine learning model for each document data included in the document data group.