Method and device for screening similar statements in a database

Abstract

The invention discloses a method and device for screening similar statements in a database. The method comprises the steps that semantic vectors and semantic vectors of multiple target extension statements of a target standard statement are utilized to expect to screen all the target extension statements to obtain extension statements to be subjected to quality inspection; Calculating the similarity between the semantic vector of each extended statement to be subjected to quality inspection and the semantic vectors of other extended statements in the database; Wherein the semantic vector is obtained through a semantic measurement model; And based on the similarity and a first preset screening condition, screening each extended statement to be subjected to quality inspection and each otherextended statement to obtain similar extended statements to be subjected to quality inspection and corresponding similar other extended statements. Therefore, only the target extension statement needing quality inspection is screened as the to-be-quality-inspected extension statement, and the number of the to-be-quality-inspected extension statements is reduced; Compared with a screening result obtained through manual screening, the automatic similar statement screening is more perfect and accurate, manpower and time are saved, and the screening efficiency is improved.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a method and device for screening similar sentences in a database. Background technique [0002] With the rapid development of artificial intelligence, the application of intelligent customer service system that best embodies artificial intelligence technology is becoming more and more extensive. The realization of intelligent customer service system depends on its core "brain" - knowledge base. The knowledge points in the knowledge base generally adopt the “question-answer” input-output form, in which the text input representing the knowledge point is called a standard sentence, and the text input derived from the expansion and deformation of the standard sentence is called an extended sentence, which has the same semantics as the standard sentence Same, slightly different text. Usually, the knowledge base includes multiple standard sentences, and each standar...

AI Technical Summary

Problems solved by technology

[0004] After research, the inventor found that in the prior art, the proportion of random selection by the staff is limited and only some sentence patterns can be extracted, which has a certain risk of missed detection; according to the staff's Experience manual screening requires a lot of manpower and time-consuming, and has high professional requirements for the staff. The screening results are highly correlated with the experience of the staff, which ultimately leads to low screening efficiency and insufficient screening results.

Images