Chinese semantic matching method based on pinyin and BERT embedding

Abstract

The invention provides a Chinese semantic matching method based on pinyin and BERT embedding. The method comprises the following steps: constructing a semantic matching model comprising a data preprocessing module, a BERT embedded layer module, a pooling layer module and a classifier module, and training the semantic matching model so as to carry out Chinese semantic matching on a to-be-matched statement by utilizing the trained semantic matching model; enabling the data preprocessing module to perform pinyin conversion and pinyin segmentation on each character in the two to-be-matched Chinesesentences to obtain a corresponding pinyin sequence; enabling the BERT embedded layer module to generate an embedded vector for each pinyin according to the context of the obtained pinyin sequence toobtain an embedded vector sequence; enabling the pooling layer module to aggregate the embedded vector sequence into a one-dimensional semantic representation vector for classification; and enablingthe classifier module to perform classification according to the one-dimensional semantic representation vector to obtain a prediction result corresponding to the semantic relationship between the twoChinese sentences. According to the method, the data volume required by pre-training can be greatly reduced, and a relatively good effect is ensured.

Description

technical field [0001] The present invention relates, in particular, to a Chinese semantic matching method based on pinyin and BERT embedding. Background technique [0002] The semantic matching task aims to model the semantics of two sentences and classify the relationship between them, which is the basis of many Natural Language Processing (NLP) tasks. For example, in natural language reasoning tasks, semantic matching is used to judge whether a certain hypothesis can be inferred from a certain premise; in question answering tasks and information retrieval tasks, semantic matching is used to calculate the correlation between the input sentence and each candidate answer And sort all candidate answers; in machine reading comprehension tasks, semantic matching is used to select the correct one among several options based on an article and corresponding questions. [0003] With the development of deep learning, the deep semantic matching model that automatically extracts feat...

AI Technical Summary

Problems solved by technology

In order to solve the problem that the traditional embedding vector generation method cannot directly represent the ambiguity of elements, Yang Piao et al. used the BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model to dynamically generate the embedding vectors corresponding to Chinese characters in the task of named entity recognition. And combined with two-way GRU and Conditional Random Fields (Conditional Random Fields, CRF) for named entity recognition

Images