Natural language multi-task modeling and prediction method and system with dependency relationship

Abstract

The invention provides a natural language multi-task modeling and predicting method and system with a dependency relationship. A hierarchical encoder embeds input words and performs encoding representation of different levels for tasks of different levels; a label embedding layer embeds labels of different tasks into the same pull-type space; a label migration device transfers the embedded label;a predictor predicts probability distribution of each task according to the encoding result and the migration result of each task; and a Gumbel sampling layer performs Gumbel sampling on the predictedprobability distribution of each task and performs counter-fact value taking according to a set probability so as to perform counter-fact deduction, and if causal association exists between the tasks, a causal effect of the tasks is obtained, and joint optimization is performed on the multi-task model. According to the causal association, the low-level task can obtain the return from the high-level task, so that the prediction result of the optimized model on the low-level task is more accurate, and the prediction precision of the high-level task is improved.

Description

technical field [0001] The present invention relates to a multi-task learning technology in the technical field of natural language processing, in particular to a natural language multi-task modeling and prediction method and system with dependencies. Background technique [0002] In the field of machine learning, multi-task learning is an important learning method because it allows the use of knowledge about related tasks to improve the effect of machine learning. In recent years, some studies have proposed a hierarchical multi-task model for tasks with dependencies. Since the potential dependencies between tasks can be utilized, its effect is generally better than that of a flat multi-task framework. However, these hierarchical multi-task models only consider the cascading of neural network encoders, but ignore the strong logical correlation between prediction results, resulting in inconsistent prediction results between various tasks, which limits the use of machine learn...

AI Technical Summary

Problems solved by technology

However, these hierarchical multi-task models only consider the cascading of neural network encoders, but ignore the strong logical correlation between prediction results, resulting in inconsistent prediction results between various tasks, which limits the use of machine learning models. Applications in actual scenarios, such as the application of judicial judgment prediction based on judgment documents, will cause inconsistencies between the predicted relevant laws and crimes

Images