Neural machine translation system-oriented sentence granularity metamorphic test method

Abstract

The invention discloses a sentence granularity metamorphic testing method oriented to a neural machine translation system, and belongs to the field of metamorphic testing, aiming at the translation robustness evaluation problem existing in the neural machine translation system. The method comprises the following steps: firstly, carrying out data preprocessing operation on two groups of texts which need to be subjected to similarity calculation to generate two groups of word lists; then, using a TF-IDF-based word bag model to construct text vectors for the two groups of word lists, and calculating an included angle cosine value of the two groups of text vectors; meanwhile, calculating a Jaccard similarity coefficient and an editing distance of the two groups of word lists; and finally, calculating the similarity between the sentences and the metamorphic relationship satisfaction rate according to the defined similarity calculation formula and the metamorphic relationship. According to the method, the problem that a single cosine similarity method is difficult to reflect semantic changes caused by different word sequences of the sentences is solved, the capability of distinguishing the semantic changes caused by the word sequence changes of the sentences is enhanced, and the sentence similarity calculation accuracy and the translation quality evaluation accuracy are improved.

Description

technical field [0001] The invention belongs to the field of metamorphosis testing, in particular to the problem of sentence granularity similarity calculation in the metamorphosis test process of a neural machine translation system, and proposes a sentence granularity metamorphosis test method oriented to a neural machine translation system. Background technique [0002] With the development of deep learning, machine translation based on neural network has made great progress. However, the neural network model adopted by the neural machine translation system lacks interpretability and understandability, which leads to the test prediction problem in the testing process of the neural machine translation system. The transformation test is a test method proposed to solve the test prediction problem. Therefore, the transformation test It is also one of the main methods for testing neural machine translation systems. During the metamorphosis test of the neural machine translatio...

AI Technical Summary

Problems solved by technology

[0003] Although the similarity calculation method based on cosine distance is well constructed, it ignores the original word order, grammar and semantics of the text

Images