Method and device for evaluating relevance of retrieval texts, server and storage medium

Abstract

The embodiment of the invention discloses a method and device for evaluating relevance of retrieval texts, a server and a storage medium. The method comprises the following steps: performing text features extraction on a plurality of sample pairs composed of query and retrieval texts, wherein the text features include original text features and structured text features; and taking the text features and correlation labels of the plurality of sample pairs as corpuses and training, and obtaining an evaluation model, wherein the evaluation mode is used for evaluating the relevance between the query and retrieval texts. According to the embodiment of the invention, when evaluating the relevance of the retrieval texts, the depth and automation of evaluation problems are taken into account, and the accuracy and generalization recognition capability of the decision logic are taken into account; besides, the comprehensiveness and the matching degree of retrieval recall can be improved through evaluation, and user experience can be improved; and meanwhile, the training and use of the evaluation model make the coverage of evaluated texts large and the cost of manual evaluation is reduced.

Description

technical field [0001] The embodiments of the present invention relate to Internet technologies, and in particular to an evaluation method, device, server and storage medium for retrieving text relevance. Background technique [0002] In the retrieval system, the user expresses all the required information through the query text, and the correlation between the retrieval results provided by the retrieval system and the query text determines the opportunity for the retrieval system to provide users with more in-depth services. The key factor to measure the quality of a retrieval system is whether it can accurately and efficiently evaluate the relevance of retrieved texts. [0003] In the prior art, methods for evaluating the relevance of retrieved texts include: manual sampling, sample selection by product evaluators, and manual comparison of multiple versions or products; direct comparison of text similarity through query text and recall text , calculate the length or propo...

AI Technical Summary

Problems solved by technology

[0004] However, the evaluation method of manual random evaluation involves excessive manpower investment and limited sample coverage. It only has a high accuracy rate for a single case, and its ability and timeliness to measure the overall status quo are poor, and it is impossible to intervene in batches of the system.

The evaluation method of directly comparing text similarity cannot cope with the diversity of semantic expressions, and the recognition granularity is relatively coarse. This kind of judgment is also covered in general retrieval systems, it is difficult to find in-depth problems, and the overall recognition accuracy is low

The automatic verification under the formulation of clear indicators is limited by the complexity of the strategy logic, and it cannot take into account the generalization processing ability and accuracy at the same time. The iteration cost of this type of decision model is high.

Images