Social noise text entity relationship extraction optimization method and system

Abstract

According to the social noise text entity relationship extraction optimization method and system, problems that in the social text field, the data labeling cost is high, the data updating speed is high, a data set has deviation, and an original model may fall into false correlation can be solved. According to the method, causal inference is introduced into the process of generating word vectors through natural language processing, advantages of intervention and anti-facts in causal inference are applied, data enhancement is achieved, and deviation caused by a data set is weakened. According to the method, the input cost of manual data labeling can be reduced, non-normalization and innovativeness of the text in an actual scene can be effectively simulated, and robustness of the model for social noise text entity relation extraction is improved; due to the fact that the method processes the word vectors, the method has good adaptability and application possibility for various existing models.

Description

technical field [0001] The invention belongs to the technical field of entity relationship extraction optimization, in particular to a method and system for entity relationship extraction optimization of social noise text. Background technique [0002] Entity relationship extraction technology has become a key part of big data analysis and knowledge graph construction. The goal of this technology is to output all (subject, relationship type, object) triples in the sentence as the target. As the field continues to develop, new methods are constantly being proposed. The earliest pipeline model divides entity extraction and relationship extraction into two successive steps, but this method is prone to cumulative errors. Subsequently, many researchers proposed a variety of entity-relationship joint extraction models to reduce the cumulative error. Existing joint entity-relationship extraction models can be broadly classified into two categories: encoder-based models and task-d...

AI Technical Summary

Problems solved by technology

If training is performed directly on such a data set without modification, for the former, a certain object and relationship may obtain a spurious correlation (SpuriousCorrelation) due to model fitting, that is, the object and relationship itself are logically inconsistent. There is no relevant significance, but because the data set often appears together from the perspective of statistical probability, the model mistakenly believes that there is a correlation between the two; for the latter, it may be due to the false correlation mentioned above in the distribution of the subject and object. The situation where the object is in the front and the subject is in the back cannot be accurately predicted, and even the result may be that the subject and the object are reversed

Therefore, the previous methods are to study the correlation of entities, relations and sentences in a statistical sense, which cannot effectively reduce or avoid the occurrence of the above phenomena.

Images