Multi-task learning method based on information entropy dynamic weighting

A multi-task learning and information entropy technology, which is applied in the field of multi-task learning adaptive balance based on information entropy dynamic weighting, can solve problems such as dependence on human experience, lack of adaptability, and subtasks cannot work normally, so as to improve task performance , the effect of strong applicability of the algorithm

Pending Publication Date: 2021-10-22
BEIHANG UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, there are still several problems in deep multi-task learning: (1) the information exchange between different sub-tasks is not sufficient enough to fully utilize the advantages of multi-task learning; (2) the loss function of most existing MTL research is usually composed of sub-tasks The loss is linearly weighted, which

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-task learning method based on information entropy dynamic weighting
  • Multi-task learning method based on information entropy dynamic weighting
  • Multi-task learning method based on information entropy dynamic weighting

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0061] The specific implementation method of the present invention will be further described in detail below in conjunction with the accompanying drawings and taking a multi-task learning network that jointly realizes semantic segmentation, depth estimation and edge detection in computer vision as an example.

[0062] The present invention proposes a multi-task learning self-adaptive balance method based on information entropy dynamic weighting, which adopts phased training, first uses the multi-task loss function with fixed weight for pre-training, and then uses the adaptive multi-task loss function with dynamic weighting Do dynamic training. In the process of model training, the information entropy algorithm can effectively evaluate the prediction results of each task, and adjust the relative weight of the task through the dynamic weighting strategy, so that the multi-task prediction model pays more attention to and improves the tasks with relatively poor performance, thereby...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-task learning method based on information entropy dynamic weighting, and belongs to the technical field of machine learning. The method comprises the following steps: firstly, building an initial multi-task learning model M, carrying out model inference on an input image to obtain a plurality of task output graphs, and respectively carrying out normalization processing on the task output graphs to obtain corresponding normalized probability graphs; then, calculating a fixed weight multi-task loss function by utilizing each normalized probability graph, and carrying out preliminary training on the multi-task learning model M; and finally, on the basis of the preliminarily trained multi-task learning model M, constructing a final adaptive multi-task loss function through an information entropy dynamic weighting algorithm, performing iterative optimization training on the preliminarily trained multi-task learning model until the multi-task learning model achieves convergence, training is terminated, and obtaining an optimized multi-task learning model M1. The method can effectively cope with different types of tasks, self-adaptively balance the relative importance of each task, and is high in algorithm applicability, simple and efficient.

Description

technical field [0001] The invention belongs to the technical field of machine learning, and in particular relates to an adaptive balancing method for multi-task learning based on dynamic weighting of information entropy. Background technique [0002] Machine learning is one of the core technologies of artificial intelligence to improve the performance of computer algorithms through empirical knowledge to achieve intelligent and autonomous learning. However, machine learning techniques usually require a large number of learning samples, especially the recently popular deep learning models usually require a large number of labeled samples to train the network. However, in many applications, some task labels of training samples are difficult to collect or manual labeling is time-consuming and laborious. In this case, multi-task learning can be utilized to maximize the utilization of limited training samples in each task. [0003] Multi-task learning aims to jointly learn mul...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06N3/08
CPCG06N3/08G06F18/2415
Inventor 王玉峰丁文锐肖京
Owner BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products