Associated information fused program language recognition system and method

Abstract

The invention discloses a program language recognition system and method fusing associated information, and the system comprises a basic feature extraction module which is used for generating a word embedding vector as a part-of-speech feature by using an embedding layer in Torch, taking a feature vector trained by a GloVe word vector technology as a semantic feature, and generating a semantic feature; taking the part-of-speech features and the semantic features after late fusion as basic features of the model; the associated information extraction module is used for adopting mutual information between words and a dependency syntactic relationship of sentences as associated information for identifying program languages; and the label representing module is used for representing labels. According to the method, feature vectors are represented through a word embedding technology, associated information capable of representing program language features is fused, deeper semantic features are acquired by using a graph convolutional neural network, and finally, a dependency relationship among tags is considered, and a conditional random field model is used for tag decoding, so that the purpose of identifying program languages is achieved.

Description

technical field [0001] The invention relates to formula language recognition, in particular to a formula language recognition system and method for fusing associated information. Background technique [0002] Formulaic language is a multi-word combination with specific functions and semantics, and is generally recognized, stored and retrieved as a whole. Studies have shown that most of the expressions of human language consist of formulas in nature. Formulaic language recognition, also known as "multi-word expression recognition", is a basic task in natural language processing. It has a wide range of applications and has important theoretical and practical significance for computer-aided language teaching and machine translation. [0003] In recent years, the research on formula language at home and abroad is in the ascendant stage. Scholars have obtained a lot of research results on formula language with the help of corpus technology and computer application programs AntCo...

AI Technical Summary

Problems solved by technology

Recognition methods based on statistics and rules rely on pre-set standards and have poor portability. When faced with complex text, they cannot effectively identify various types of programming languages.

With the rise of machine learning in the field of natural language processing, some scholars have tried to use classifiers such as random forests and support vector machines to identify formulaic language through classification techniques. However, this method has high requirements for feature selection and needs to be selected to effectively reflect The feature set of programming language characteristics leads to its poor generalization ability

Images