Automated Neural Network Compilation via Probabilistic Factor Graphs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current machine learning technologies lack a systematic procedure for translating machine learning problems into neural network structures and face challenges in training neural networks due to the intensive computational effort required, especially with large numbers of parameters.

Innovation Solution

A probabilistic framework based on a Bayesian generative model is developed, which captures nuisance variables and can convert probabilistic models into deep learning systems using the BrainFactory procedure, enabling the automated construction of neural networks for solving inference tasks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If a neural network is designed using ad hoc procedures guided by intuition and trial-and-error experiments, then the neural network can be constructed for solving machine learning problems, but the process is labor intensive and inhibits proliferation of solutions

Engineering Contradiction:

Improveease of neural network constructionVSAvoidtime for neural network construction

Core Design Contradiction:

Ease of manufactureVSLoss of time

Solution Approach 1:

The system enables automated neural network construction by having the computer system itself perform the design process through executable instructions that automatically translate problem definitions into neural network specifications, eliminating the need for manual ad hoc design procedures

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the manual mechanical process of neural network design (intuition and trial-and-error experiments) with an automated computational system that uses executable instructions to systematically construct neural networks from problem definitions

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If the number of parameters in a neural network is increased to solve complex machine learning problems, then the neural network's capability is improved, but the training process becomes computationally laborious

Engineering Contradiction:

Improveneural network capabilityVSAvoidcomputational effort for training

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary action by automatically generating optimized neural network architectures and parameter configurations before training begins, using the automated compilation process to prepare the neural network structure in advance, which reduces the computational burden during the actual training phase

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10846589B2Automated compilation of probabilistic task description into executable neural network specification

Publication Date: 2020.11.24 WILLIAM MARCH RICE UNIVERSITY

AI summary

A mechanism for compiling a generative description of an inference task into a neural network. First, an arbitrary generative probabilistic model from the exponential family is specified (or received). The model characterizes a conditional probability distribution for measurement data given a set of latent variables. A factor graph is generated for the generative probabilistic model. Each factor node of the factor graph is expanded into a corresponding sequence of arithmetic operations, based on a specified inference task and a kind of message passing algorithm. The factor graph and the sequences of arithmetic operations specify the structure of a neural network for performance of the inference task. A learning algorithm is executed, to determine values of parameters of the neural network. The neural network is then ready for performing inference on operational measurements.