Multi-task learning method based on information entropy dynamic weighting

Abstract

The invention discloses a multi-task learning method based on information entropy dynamic weighting, and belongs to the technical field of machine learning. The method comprises the following steps: firstly, building an initial multi-task learning model M, carrying out model inference on an input image to obtain a plurality of task output graphs, and respectively carrying out normalization processing on the task output graphs to obtain corresponding normalized probability graphs; then, calculating a fixed weight multi-task loss function by utilizing each normalized probability graph, and carrying out preliminary training on the multi-task learning model M; and finally, on the basis of the preliminarily trained multi-task learning model M, constructing a final adaptive multi-task loss function through an information entropy dynamic weighting algorithm, performing iterative optimization training on the preliminarily trained multi-task learning model until the multi-task learning model achieves convergence, training is terminated, and obtaining an optimized multi-task learning model M1. The method can effectively cope with different types of tasks, self-adaptively balance the relative importance of each task, and is high in algorithm applicability, simple and efficient.

Description

technical field [0001] The invention belongs to the technical field of machine learning, and in particular relates to an adaptive balancing method for multi-task learning based on dynamic weighting of information entropy. Background technique [0002] Machine learning is one of the core technologies of artificial intelligence to improve the performance of computer algorithms through empirical knowledge to achieve intelligent and autonomous learning. However, machine learning techniques usually require a large number of learning samples, especially the recently popular deep learning models usually require a large number of labeled samples to train the network. However, in many applications, some task labels of training samples are difficult to collect or manual labeling is time-consuming and laborious. In this case, multi-task learning can be utilized to maximize the utilization of limited training samples in each task. [0003] Multi-task learning aims to jointly learn mul...

AI Technical Summary

Problems solved by technology

However, there are still several problems in deep multi-task learning: (1) the information exchange between different sub-tasks is not sufficient enough to fully utilize the advantages of multi-task learning; (2) the loss function of most existing MTL research is usually composed of sub-tasks The loss is linearly weighted, which relies on human experience and lacks adaptability

[0007] In the study of optimization strategies, most of the work related to multi-task learning simply sets the weights of each task to a fixed ratio, but this method relies heavily on human experience, and in some cases, inappropriate weights may lead to some subtasks not working

Images