Statement correction method and device based on bert model and ngram model

Abstract

The invention provides a statement correction method and device based on a bert model and an ngram model, and belongs to the technical field of data processing. The method comprises the following steps: determining a plurality of candidate target words used for replacing wrong words in statements, and forming a plurality of candidate statements; for each candidate statement, calculating a first confusion degree of the statement based on a preset bert model, and calculating a second confusion degree of the statement based on a preset ngram model; giving a first weighting coefficient of the first confusion degree, and giving a second weighting coefficient of the second confusion degree; determining the total perplexity of each candidate statement by weighting the first perplexity and the second perplexity; and determining a final statement based on the total perplexity. According to the method, the bert model and the ngram model are complementary, so that the accuracy of statement correction is improved.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to View More

AI Technical Summary

Benefits of technology

This invention relates to an improved way to smooth out documents with repetitive content without leaking important details or losing their original purpose. It combines two techniques: one involves comparing different terms from each other by calculating differences between them; another technique uses word/word pairs instead of just counting how many times they have been used beforehand. Overall this technology helps improve document quality and preventing data loss during communication sessions.

Problems solved by technology

This patented technical solution described in the patents involves computing the confidence value of each letter called ppi_intestation ratio (PTS). By comparing these values against predetermined thresholds, we may determine if any letters have higher chance than other ones. If they don’t get enough evidence during testing, indicating that some parts were missing while others had been added correctly. However, even after learning about how long things took beforehand, most phrases still contain errors. To address such issues, previous solutions involve either adding extra bits like zeros or repeating existing tokens over multiple times instead of just repeatedly applying new symbols until convergence occurs.

Images