Machine Learning Model Reliability Assessment and Training Data Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The preparation of training data for machine learning models is labor-intensive and requires a large amount of data, which can lead to accuracy issues if the data quantity is reduced.

Innovation Solution

An information processing system that includes a machine learning model, reliability output means, generation means, and learning control means. The system outputs the reliability of the model's output for input data, generates new training data based on the input data if the reliability meets a predetermined condition, and trains the machine learning model with the new data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a large amount of training data is used to train the machine learning model, then the accuracy of the model is improved, but the labor required for data preparation increases

Engineering Contradiction:

Improveaccuracy of machine learning modelVSAvoidlabor required for data preparation

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system automatically generates new training data using the machine learning model itself and an estimation model, eliminating the need for manual data preparation. The model serves itself by producing training data from its own outputs and reliability assessments, thereby reducing external labor input while maintaining data quantity and quality

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements a feedback loop where the machine learning model's outputs are evaluated by reliability output means, and high-reliability outputs are fed back as new training data. This closed-loop feedback mechanism continuously improves the model using self-generated data, reducing the need for external data preparation labor

Inventive Principle:

Principle #23Feedback

2Loss of time

If the amount of training data is reduced to decrease labor, then the labor required for data preparation is reduced, but the accuracy of the machine learning model cannot be ensured

Engineering Contradiction:

Improvelabor required for data preparationVSAvoidaccuracy of machine learning model

Core Design Contradiction:

Loss of timeVSMeasurement precision

Solution Approach 1:

The system creates copies of existing training data by generating new training data from the machine learning model's outputs. Instead of reducing data quantity, the system replicates and augments training data through automated generation processes, maintaining sufficient data volume for model accuracy while eliminating manual preparation labor

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs preliminary actions by pre-processing the machine learning model's outputs through reliability assessment and estimation models before generating new training data. This preliminary filtering and processing ensures that only high-quality data is used for training, maintaining model accuracy without requiring extensive manual data preparation

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250165862A1Information processing system, information processing method, and program

Publication Date: 2025.05.22 SONY INTERACTIVE ENTERTAINMENT LLC

AI summary

The accuracy of a machine learning model is improved while the maintenance of training data is facilitated. An information processing system outputs, on the basis of an output of a machine learning model being trained with training data when input data is received to the machine learning model as an input, reliability of the output for the input data, generates new training data on the basis of the input data in a case where the reliability satisfies a predetermined condition, and trains the machine learning model with the new training data.