Neural Network Parameter Differential Reporting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current wireless communication systems face inefficiencies in transmitting and updating large neural network models, as uploading the entire trained artificial intelligence module with each update is not feasible due to data size and resource constraints.

Innovation Solution

Implementing a differential model update mechanism where the user equipment (UE) trains and reports only the differences in neural network parameters based on a trigger, such as a loss function threshold or parameter change metric, rather than the entire neural network, to reduce over-the-air transmissions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the entire trained neural network model is uploaded with each update, then the model update completeness is improved, but the signaling overhead and resource consumption increase significantly

Engineering Contradiction:

Improvemodel update completenessVSAvoidsignaling overhead

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the differential portion (changes) from the complete neural network model for transmission. Instead of uploading the entire model, the system identifies and transmits only the modified parameters or weight changes, significantly reducing the data volume while maintaining update completeness at the receiver side through differential synthesis.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The model update process is segmented into identifying changes, encoding differentials, and transmitting only the variable portion. The update is divided into base model (already present at receiver) and differential updates (transmitted), allowing selective transmission of only necessary information.

Inventive Principle:

Principle #1Segmentation

2Quantity of substance

If differential model updates are transmitted, then the signaling overhead is reduced, but the complexity of tracking parameter changes increases

Engineering Contradiction:

Improvedata transmission volumeVSAvoidparameter change tracking

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The system implements feedback mechanisms where the receiver acknowledges received updates and the transmitter tracks which parameters have been updated. This feedback loop manages the complexity by providing structured state information about what has been transmitted and received, simplifying the differential tracking process.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary actions by pre-identifying which parameters will change before transmission begins. By pre-computing the differential set and preparing the update manifest, the system reduces the real-time complexity of tracking changes during the actual update process.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20220335294A1Reporting for machine learning model updates

Publication Date: 2022.10.20 QUALCOMM INC

AI summary

A receiver receives, from a transmitter, a reference neural network. The receiver trains the reference neural network to obtain updated neural network parameters for the reference neural network. The receiver reports to the transmitter in response to a trigger, a difference between the updated neural network parameters and previous neural network parameters for the reference neural network. The trigger may be based on a loss function, a magnitude of the difference between the updated neural network parameters and the previous neural network parameters, and/or a difference between performance of the reference neural network with the updated neural network parameters and performance of the reference neural network with the previous neural network parameters.