Differentiable Jaccard Loss Approximation for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional loss functions used for training artificial neural networks (ANNs), such as per-pixel softmax, fail to effectively factor the overall accuracy of matches between predicted outputs and desired results, leading to slow convergence and increased resource and time requirements for training.
Innovation Solution
The introduction of a differentiable Jaccard Loss approximation, which determines a ratio of true positive pixels to the union of true and false positive pixels, allowing for improved accuracy and faster training by enabling backpropagation and gradient descent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional loss functions (e.g., per-pixel softmax) are used for training ANN, then the training process can be performed, but the convergence is slow and training time is increased
Solution Approach 1:
The patent changes the mathematical formulation of the loss function from traditional per-pixel softmax to a differentiable Jaccard Loss approximation. This parameter change in the loss function's mathematical structure enables better alignment with segmentation accuracy metrics, resulting in faster convergence and reduced training time while maintaining differentiability for backpropagation
Solution Approach 2:
The patent replaces the traditional mechanical computation of per-pixel softmax loss with a mathematical approximation that uses differentiable operations to compute Jaccard Index-like metrics. This substitution of the loss computation mechanism allows for more efficient gradient computation and faster training convergence
2Measurement precision
If traditional loss functions are used, then training can proceed, but the overall accuracy between predicted and desired results is not effectively factored
Solution Approach 1:
The differentiable Jaccard Loss approximation provides a feedback mechanism that directly measures segmentation accuracy using Jaccard Index principles (intersection over union of predicted and ground truth masks). This feedback is differentiable, allowing continuous optimization through backpropagation, thereby improving both accuracy measurement and training efficiency simultaneously
Data Source
AI summary
Systems and methods described herein may relate to training an artificial neural network (ANN) using a differentiable Jaccard Loss approximation. An example embodiment may involve obtaining a training image and a corresponding ground truth mask that represents a desired segmentation of the training image. The embodiment may further involve applying an ANN on the training image to generate an output segmentation of the training image that depends on a plurality of weights of the ANN and determining a differentiable Jaccard Loss approximation based on the output segmentation of the training image and the ground truth mask. The embodiment also involves modifying one or more weights of the ANN based on the differentiable Jaccard Loss approximation and providing a representation of the ANN as modified to a mobile computing device.