Posterior Distribution Estimation via Prior Sampling

Solution Overview

Problem

Existing methods for estimating characteristics of posterior distributions, such as differential entropy, are computationally expensive due to repetitive Markov Chain Monte Carlo (MCMC) sampling and entropy computation.

Innovation Solution

An apparatus and method that generate samples from a prior distribution, calculate parameters related to density, and estimate posterior distribution characteristics without sampling the posterior distribution, reducing computational resources by using a sampling section, obtaining section, calculation section, and estimation section to derive parameters and likelihoods.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If Markov Chain Monte Carlo (MCMC) algorithms are used to generate samples from the posterior distribution for every possible observation, then the estimation of differential entropy can be performed, but the computation is time consuming and computationally expensive

Engineering Contradiction:

Improvedifferential entropy estimation accuracyVSAvoidcomputation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent pre-generates a large set of samples from the prior distribution before any observation is made. These samples are stored and reused for multiple observations, eliminating the need to perform MCMC sampling for each new observation. This preliminary action significantly reduces computation time while maintaining estimation accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of generating new samples from the posterior distribution for each observation, the patent uses copies of the pre-generated prior samples and applies importance weighting based on the likelihood of each observation. This copying approach avoids repetitive MCMC computation while preserving the ability to estimate differential entropy for multiple observations.

Inventive Principle:

Principle #26Copying

2Measurement precision

If Markov Chain Monte Carlo (MCMC) algorithms are used to generate samples from the posterior distribution for every possible observation, then the estimation of differential entropy can be performed, but the computation is computationally expensive

Engineering Contradiction:

Improvedifferential entropy estimation accuracyVSAvoidcomputational resources

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent pre-generates a large set of samples from the prior distribution before any observation is made. These samples are stored and reused for multiple observations, eliminating the need to perform MCMC sampling for each new observation. This preliminary action significantly reduces computation time while maintaining estimation accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of generating new samples from the posterior distribution for each observation, the patent uses copies of the pre-generated prior samples and applies importance weighting based on the likelihood of each observation. This copying approach avoids repetitive MCMC computation while preserving the ability to estimate differential entropy for multiple observations.

Inventive Principle:

Principle #26Copying

3Use of energy by moving object

If samples are generated from the prior distribution and posterior distribution characteristics are estimated without sampling the posterior distribution, then computational resources are reduced, but the method must accurately estimate density parameters

Engineering Contradiction:

Improvecomputational resourcesVSAvoiddensity parameter estimation accuracy

Core Design Contradiction:

Use of energy by moving objectVSManufacturing precision

Solution Approach 1:

The patent uses the prior distribution samples as an intermediary to estimate posterior distribution characteristics. By applying importance weighting with the likelihood function, the method bridges the gap between prior samples and posterior estimation without requiring actual posterior sampling, thus reducing computational resources while maintaining accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent transforms the estimation problem by changing from direct posterior sampling to importance sampling using prior samples with weighted likelihoods. This parameter transformation allows accurate density estimation without the computational burden of posterior MCMC sampling.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20230206099A1Computational estimation of a characteristic of a posterior distribution

Publication Date: 2023.06.29 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

An apparatus for implementing a computing system to predict preferences includes at least one processor device operatively coupled to a memory. The at least one processor device is configured to calculate a parameter relating to a density of a prior distribution at each sample of a set of samples associated with the prior distribution. The at least one parameter including a distance from each sample to at least one neighboring sample. The at least one processor device is further configured to estimate, for the plurality of samples, at least one differential entropy of at least one posterior distribution associated with at least one observation based on the parameter relating to the density of the prior distribution at each sample and the likelihood of observation for each sample. The estimation is performed without sampling the at least one posterior distribution to reduce consumption of resources of the computing system.