Dynamic Neural Network Connection Variable Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural network training methods require pre-defined model architectures and do not dynamically learn the connections between layers, limiting their ability to optimize the network structure for improved performance in image recognition tasks.

Innovation Solution

A method and system for updating connection variables between layers of a neural network model based on input and output data, allowing for dynamic adjustment of connection intensities and the decision to maintain or abandon non-adjacent layer connections, thereby optimizing the network structure for better accuracy and performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If pre-defined model architecture is used, then device complexity is reduced and ease of manufacture is improved, but adaptability and performance optimization are worsened

Engineering Contradiction:

Improveease of neural network constructionVSAvoidadaptability of network structure

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic architecture search by allowing the neural network model to automatically learn and determine its own connection structure between layers during the training process, rather than using a fixed pre-defined architecture. This enables the network to adapt its structure dynamically based on the specific task requirements, resolving the contradiction between ease of construction and adaptability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system enables the neural network to self-determine its optimal architecture by automatically learning connection variables between layers without requiring manual pre-definition. The network serves itself by autonomously optimizing its structural configuration, eliminating the need for external architectural design while maintaining ease of implementation.

Inventive Principle:

Principle #25Self-service

2Device complexity

If pre-defined connection paths are used, then device complexity is reduced, but manufacturing precision and performance are worsened

Engineering Contradiction:

Improvecomplexity of network connectionsVSAvoidprecision of model architecture

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The patent transforms the static pre-defined connection paths into dynamic learned connections. The connection variables between layers are learned automatically during training, allowing the network to discover precise connection patterns specific to each task. This dynamic approach maintains simplicity while achieving high precision in architectural configuration.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the connection parameters from fixed pre-defined values to learned variables that can be optimized during training. By allowing connection variables to be dynamically adjusted based on performance feedback, the system achieves precise architectural configuration without increasing operational complexity.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If dynamic connection learning is implemented, then adaptability and performance are improved, but device complexity and training time are worsened

Engineering Contradiction:

Improveadaptability of network structureVSAvoidcomplexity of training process

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements a universal architecture search mechanism that can be applied to any neural network model regardless of its specific application. The same dynamic connection learning framework works across different tasks and network types, making the increased complexity worthwhile by providing universal adaptability without requiring task-specific architectural design.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system uses performance feedback from the training process to guide the learning of connection variables. By continuously monitoring training outcomes and adjusting connections based on this feedback, the system efficiently navigates the complex search space, making the dynamic approach manageable despite increased training complexity.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11620509B2Model constructing method of neural network model, model constructing system, and non-transitory computer readable storage medium

Publication Date: 2023.04.04 HTC CORP

AI summary

A model constructing method for a neural network model applicable for image recognition processing is disclosed. The model constructing method includes the following operation: updating, by a processor, a plurality of connection variables between a plurality of layers of the neural network model, according to a plurality of inputs and a plurality of outputs of the neural network model. The plurality of outputs represent a plurality of image recognition results. The plurality of connection variables represent a plurality of connection intensities between each two of the plurality of layers.