ML Data Division Criteria Determination via Prediction Performance Matching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods fail to accurately identify the division method used for training and validation data in machine learning, leading to imprecise data division, which affects the accuracy of machine learning models and can result in prediction performance degradation, especially when new data is added or when manual methods are used.

Innovation Solution

A determination program and information processing device that generates multiple division candidate datasets using different criteria from a combined training and validation dataset, calculates prediction performances for each, and identifies the division method that produces prediction performances closest to the original, thereby determining the optimal division criteria.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If manual division methods are used for training and validation data, then flexibility in data processing is improved, but measurement precision of the division method deteriorates

Engineering Contradiction:

Improveflexibility in data processingVSAvoidprecision of division method identification

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent creates a copy of the divided dataset by combining training data and validation data, then generates multiple candidate division datasets using different division methods. By comparing the copied data against the original division results, the system automatically identifies which division method was used, thereby capturing precision without manual intervention.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs self-identification of the division method by automatically generating candidate divisions, training machine learning models, and evaluating prediction performances. This automated self-service process eliminates the need for manual tracking of division methods while maintaining high precision in identifying the correct division approach.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If the division method is changed when new data is added, then adaptability to new data is improved, but reliability of prediction performance deteriorates

Engineering Contradiction:

Improveadaptability to new dataVSAvoidreliability of prediction performance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent performs preliminary identification of the division method used for existing training and validation data before adding new data. By determining the correct division method in advance through automated evaluation, the system ensures that new data is integrated using the same proven method, maintaining reliability while enabling adaptability.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If multiple division methods are tested, then measurement precision of division method identification is improved, but productivity of the process deteriorates

Engineering Contradiction:

Improveprecision of division method identificationVSAvoidspeed of division method identification

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent generates multiple candidate division datasets using different division methods, which is more than the single method approach. However, it efficiently evaluates these candidates by training machine learning models and comparing prediction performances, achieving high precision in identifying the correct method while managing the computational burden through systematic evaluation.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20230289657A1Computer-readable recording medium storing determination program, determination method, and information processing device

Publication Date: 2023.09.14 FUJITSU LTD

AI summary

A program for causing a computer to execute processing including: generating division candidate datasets divided in accordance with different criteria from each other, from a combined dataset obtained by combining training data and validation data in a divided dataset that has been divided into the training data and the validation data used for machine learning; generating respective machine learning pipelines that execute machine learning, separately for each of the divided dataset and the division candidate datasets; using each of the divided dataset and the division candidate datasets to calculate respective prediction performances when the respective machine learning pipelines are executed; identifying division candidate datasets that have the prediction performances closest to the respective prediction performances calculated using the divided dataset, from among the division candidate datasets; and determining division criteria used for the identified division candidate dataset to be the division criteria used for the divided dataset.