Neural Network Training with Auxiliary Loss to Reduce Discretization Bias
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks suffer from discretization bias, leading to low accuracy in trained models due to the inherent discretization of output results.
Innovation Solution
A method and device for data processing that involves inputting labeled training and verification data into a preset model, updating model parameters, determining verification and auxiliary loss values, and deciding whether to stop training based on these values to achieve a trained model that reduces discretization bias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If neural network output is discretized, then the model structure is simplified, but the accuracy of the trained model decreases due to discretization bias
Solution Approach 1:
The patent introduces an auxiliary loss function as an intermediary mechanism that bridges the discretized output structure and continuous learning requirements. This auxiliary loss function computes gradients based on the difference between discretized output and target values, enabling continuous parameter updates while maintaining discretized output structure, thus resolving the contradiction between simplified structure and high accuracy
Solution Approach 2:
The patent changes the parameter optimization approach by introducing auxiliary loss values that modify the gradient computation process. Instead of directly optimizing discrete outputs, the system optimizes continuous parameters through auxiliary loss functions that guide parameter changes to reduce discretization bias, thereby improving accuracy without increasing structural complexity
2Ease of manufacture
If traditional training methods are used, then the training process is simple, but the model accuracy remains low due to discretization bias
Solution Approach 1:
The patent implements a feedback mechanism where auxiliary loss values are computed from the difference between discretized output and target values, and this feedback is used to guide parameter updates. The auxiliary loss function provides continuous feedback signals that help the model learn to reduce discretization bias while maintaining the simplicity of the overall training framework
Solution Approach 2:
The patent performs preliminary computation of auxiliary loss values before the main parameter update step. By pre-computing the auxiliary loss and its gradients, the system prepares correction signals in advance that will be applied during parameter updates, improving accuracy without significantly complicating the training process
Data Source
AI summary
Provided are a method and device for data processing and a storage medium. The method includes: inputting labeled training data in a training set into a preset model to be trained and updating the model parameters of the preset model; inputting labeled verification data in a verification set into the preset model after the model parameters are updated to obtain a first prediction label; obtaining a verification loss value based on a difference between the first prediction label and a marked label of the labeled verification data; determining an auxiliary loss value based on current structural parameters of the preset model; determining whether to stop training the preset model based on the verification loss value and the auxiliary loss value; and classifying data to be classified based on a target network model constructed by a network structure included in the trained preset model to obtain a classification result.