Deep Multi-Task Representation Learning for Efficient Classification

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

As the number of features in machine learning algorithms increases, so does the complexity, requiring more resources such as memory and computing power, making it difficult to maintain high attribute classification accuracy and train models effectively.

Innovation Solution

A deep multi-task representation learning model with a combined generative and discriminative component is used to learn shared representations from multi-task multimodal data, employing an iterative bottom-up/top-down approach to classify data and infer missing information across modalities, thereby reducing the computational resources needed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the number of features in machine learning algorithms is increased, then the classification accuracy is improved, but the complexity of the algorithm increases and requires more memory and computing power

Engineering Contradiction:
Improveclassification accuracyVSAvoidalgorithm complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the learning process into multiple tasks that share common features. Instead of treating all features uniformly, the algorithm divides them into task-specific subsets while maintaining shared representations, thereby reducing overall complexity while preserving classification accuracy.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements multi-task learning where a single model performs multiple classification tasks simultaneously. The shared layers of the neural network learn universal feature representations that can be applied across different tasks, reducing the need for separate models and thereby reducing computational resources while maintaining or improving accuracy.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Measurement precision

If the number of features in machine learning algorithms is increased, then the classification accuracy is improved, but the memory and computing power requirements increase

Engineering Contradiction:
Improveclassification accuracyVSAvoidcomputing power
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

By designing a multi-task learning model where shared neural network layers serve multiple classification tasks, the patent reduces redundant computation. The shared features are computed once and reused across tasks, significantly reducing the computing power and energy required compared to training separate models for each task.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent merges multiple classification tasks into a single unified model. By combining the training objectives and sharing parameters across tasks, the model achieves better generalization and reduces the total computational burden compared to training multiple independent models.

Inventive Principle:
Principle #5Merging (Combining)

3Measurement precision

If the number of features in machine learning algorithms is increased, then the classification accuracy is improved, but the memory requirements increase

Engineering Contradiction:
Improveclassification accuracyVSAvoidmemory
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The shared layers in the multi-task learning model store feature representations that are universally useful across multiple tasks. This eliminates the need to store separate feature sets for each task, significantly reducing memory requirements while maintaining the ability to achieve high classification accuracy across all tasks.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The model segments features into shared and task-specific components. The shared features are stored in common layers, while task-specific features are stored in separate output layers. This segmentation allows efficient memory utilization by avoiding redundant storage of identical feature representations.

Inventive Principle:
Principle #1Segmentation

4Measurement precision

If the complexity of the algorithm is increased, then the classification accuracy is improved, but the training time increases

Engineering Contradiction:
Improveclassification accuracyVSAvoidtraining time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

By training a single multi-task model that performs multiple classification functions simultaneously, the patent reduces the total training time compared to training separate models for each task. The shared features are learned once and benefit all tasks, improving efficiency while maintaining or improving overall classification accuracy.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent combines multiple training objectives into a unified loss function and trains all tasks together in a single model. This joint training approach leverages shared representations and reduces the cumulative training time that would result from sequentially or independently training multiple separate models.

Inventive Principle:
Principle #5Merging (Combining)

Data Source

PatentUS12073305B2Deep multi-task representation learning
Publication Date: 2024.08.27 SRI INTERNATIONAL
  • US12073305B2 patent drawing
  • US12073305B2 patent drawing
  • US12073305B2 patent drawing

AI summary

Technologies for analyzing multi-task multimodal data to detect multi-task multimodal events using a deep multi-task representation learning, are disclosed. A combined model with both generative and discriminative aspects is used to share information during both generative and discriminative processes. The technologies can be used to classify data and also to generate data from classification events. The data can then be used to morph data into a desired classification event.