Interconnected Neural Network Clusters for Forecasting

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional deep neural networks (DNNs) are limited in their applicability and efficiency, particularly in tasks such as data series extrapolation and forecasting, due to high computational requirements and memory constraints, and lack effective methods for ongoing training to adapt to changing trends in input data over time.

Innovation Solution

The system employs a cluster of computing devices, including cloud-based resources, to train and operate multiple interconnected DNNs with parallelized training across processing units, allowing for efficient parameter adjustment and ongoing training to improve forecasting accuracy by maintaining historical data relationships without excessive memory or computational overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional deep neural networks are used for data series extrapolation and forecasting, then modeling capability is improved, but computational requirements and memory constraints increase

Engineering Contradiction:
Improveforecasting accuracyVSAvoidcomputational requirements
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent divides the neural network into multiple specialized networks, each handling a specific time period or forecasting task. This segmentation reduces the computational burden on each individual network while maintaining overall forecasting accuracy through coordinated operation of multiple smaller networks.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements a hierarchical structure where smaller neural networks are nested within a larger framework. Each nested network processes specific temporal patterns, and their outputs are integrated to produce the final forecasting result, reducing memory requirements while preserving modeling capability.

Inventive Principle:
Principle #7Nested doll (Nesting)

2Measurement precision

If conventional deep neural networks are used for data series extrapolation and forecasting, then modeling capability is improved, but memory constraints increase

Engineering Contradiction:
Improveforecasting accuracyVSAvoidmemory requirements
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

By segmenting the neural network into multiple smaller specialized networks, each handling specific time periods or forecasting aspects, the patent reduces the memory footprint of individual networks while maintaining collective forecasting accuracy through their coordinated operation.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts and separates different temporal patterns into distinct neural network components, removing the need for a single large network to store all patterns simultaneously. This extraction approach reduces memory constraints while preserving the ability to model complex temporal relationships.

Inventive Principle:
Principle #2Taking out (Extraction)

3Measurement precision

If conventional deep neural networks are used, then semantic modeling capability is improved, but adaptability to changing trends in input data over time deteriorates

Engineering Contradiction:
Improvesemantic modeling capabilityVSAvoidadaptability to changing trends
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic training mechanisms where neural network parameters are continuously adjusted based on incoming data trends. The system can adapt to changing patterns over time by updating network weights and structures, maintaining both semantic modeling capability and adaptability to evolving data characteristics.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent incorporates feedback loops where forecasting results and actual outcomes are used to continuously retrain and refine the neural networks. This feedback mechanism enables the system to adapt to changing trends while preserving semantic modeling capabilities through iterative improvement.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS10474950B2Training and operation of computational models
Publication Date: 2019.11.12 MICROSOFT TECHNOLOGY LICENSING LLC
  • US10474950B2 patent drawing
  • US10474950B2 patent drawing
  • US10474950B2 patent drawing

AI summary

A processing unit can acquire datasets from respective data sources, each having a respective unique data domain. The processing unit can determine values of a plurality of features based on the plurality of datasets. The processing unit can modify input-specific parameters or history parameters of a computational model based on the values of the features. In some examples, the processing unit can determine an estimated value of a target feature based at least in part on the modified computational model and values of one or more reference features. In some examples, the computational model can include neural networks for several input sets. An output layer of at least one of the neural networks can be connected to the respective hidden layer(s) of one or more other(s) of the neural networks. In some examples, the neural networks can be operated to provide transformed feature value(s) for respective times.