State Transition Estimation with Stratified Data Regularization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data estimation methods suffer from over-learning when estimating state transition probabilities between stratified data sets, leading to reduced estimation accuracy due to the influence of some data points.

Innovation Solution

The data estimation device employs an algorithm related to optimal transport to estimate state transition probabilities by stratifying data sets based on attributes, using a weighted loss function to regularize the transition probabilities and maintain overall tendencies, thereby suppressing over-learning.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If data sets are stratified based on attributes to improve estimation accuracy for specific groups, then the precision of transition probability estimation for each stratum is improved, but over-learning occurs due to reduced data quantities in each stratified set

Engineering Contradiction:

Improveestimation accuracyVSAvoidover-learning suppression

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent segments the overall data set into multiple stratified data sets based on attributes (e.g., age groups, disease types). This segmentation allows for more precise estimation within each stratum while the regularization technique prevents over-learning by maintaining connection to the overall distribution pattern.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a regularization parameter that controls the balance between fitting the stratified data and maintaining the overall distribution pattern. By adjusting this parameter, the system can prevent over-learning while still capturing stratum-specific characteristics, effectively changing the estimation parameters to avoid overfitting.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If stratification is performed to capture heterogeneous data characteristics, then the adaptability of the estimation model is improved, but the complexity of the estimation process increases

Engineering Contradiction:

Improvedata characteristic captureVSAvoidestimation process complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The data set is segmented into stratified subsets based on relevant attributes, allowing the model to adapt to heterogeneous characteristics within each stratum. This segmentation approach improves adaptability while maintaining manageable complexity through structured organization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent uses feedback from the overall distribution pattern to regularize the stratified estimation process. The estimated transition probabilities from the full data set are used as a reference to guide the stratified estimation, creating a feedback mechanism that simplifies the overall process while maintaining adaptability.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If the state transition probability is estimated directly from stratified data sets, then the measurement precision for each stratum is improved, but the loss of information about overall data trends occurs

Engineering Contradiction:

Improvestratum-specific estimation precisionVSAvoidoverall data trend information

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent implements feedback by using the state transition probability estimated from the full data set as a reference pattern. This overall pattern provides feedback to the stratified estimation process, ensuring that stratum-specific estimates remain consistent with the global data trends and preventing information loss.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent combines information from both the full data set and the stratified data sets. By merging the overall distribution pattern with stratum-specific characteristics through regularization, the method preserves both global trends and local precision without losing information from either source.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS20260057266A1Data estimation device, data estimation method, and recording medium

Publication Date: 2026.02.26 NEC CORP

AI summary

A data estimation device includes an acquisition unit, a stratification unit, an estimation unit, and an output unit. The acquisition unit acquires data sets including pieces of data indicating mutually different probability distributions and an attribute used for stratification of the data sets. The stratification unit stratifies the data sets based on the attribute. The estimation unit estimates a state transition probability between the data sets after the stratification based on a difference in distribution between a state transition probability between the data sets before the stratification and a state transition probability between the data sets stratified for each of the attributes. The output unit outputs the state transition probability between the data sets after the stratification. The use of the state transition probability estimated in this manner enables the data estimation device to support decision making based on an estimation result of a transition destination of data.