Natural language multi-task modeling and prediction method and system with dependencies

Abstract

The present invention provides a natural language multi-task modeling and prediction method and system with dependencies. The hierarchical encoder performs encoding representation of different levels of input word embedding for different levels of tasks; the label embedding layer converts the labels of different tasks Embedded in the same pull space; the label migrator transfers the embedded label; the predictor predicts the probability distribution of each task according to the encoding result and migration result of each task; the Gumbel sampling layer predicts for each task Gumbel sampling is performed on the probability distribution of the given probability, and the counterfactual value is obtained with the set probability, so as to perform counterfactual inference. If there is a causal relationship between tasks, the causal effect is obtained, and the multi-task model is jointly optimized. According to the causal correlation, the present invention can obtain returns from high-level tasks for low-level tasks, thereby making the prediction results of the optimized model more accurate for low-level tasks and improving the prediction accuracy of high-level tasks.

Description

technical field [0001] The present invention relates to a multi-task learning technology in the technical field of natural language processing, in particular to a natural language multi-task modeling and prediction method and system with dependencies. Background technique [0002] In the field of machine learning, multi-task learning is an important learning method because it allows the use of knowledge about related tasks to improve the effect of machine learning. In recent years, some studies have proposed a hierarchical multi-task model for tasks with dependencies. Since the potential dependencies between tasks can be utilized, its effect is generally better than that of a flat multi-task framework. However, these hierarchical multi-task models only consider the cascading of neural network encoders, but ignore the strong logical correlation between prediction results, resulting in inconsistent prediction results between various tasks, which limits the use of machine learn...

AI Technical Summary

Problems solved by technology

However, these hierarchical multi-task models only consider the cascading of neural network encoders, but ignore the strong logical correlation between prediction results, resulting in inconsistent prediction results between various tasks, which limits the use of machine learning models. Applications in actual scenarios, such as the application of judicial judgment prediction based on judgment documents, will cause inconsistencies between the predicted relevant laws and crimes

Images