Text similarity matching model compression method and system for knowledge distillation

Abstract

The invention provides a text similarity matching model compression method and system for knowledge distillation. The method comprises the following steps: obtaining training data; determining a firsttraining model by adopting a first deep text matching algorithm according to the training data; distilling the priori knowledge of the first training model into training data of a second training model, adopting a second deep text matching algorithm to determine the second training model, wherein the calculated amount of the first deep text matching algorithm is larger than that of the second deep text matching algorithm; and predicting a text similarity matching result by adopting the second training model. According to the method, a text matching method based on knowledge distillation is adopted, a calculation result of a large model is fused into a training process of a small model, when a second training model is adopted for online prediction, the operation speed is prevented from being reduced, meanwhile, a priori result of a first training large model is utilized, and the prediction accuracy can be improved.

Description

technical field [0001] The invention relates to the technical field of text matching, in particular to a text similarity matching model compression method and system for knowledge distillation. Background technique [0002] At present, in the open domain question answering in the human-computer dialogue system, the mainstream solution is the combination of the retrieval module and the matching module. The steps of the current mainstream text matching scheme are: Step 1: First, through the retrieval module, obtain a fixed number N (such as 20) of candidate results; Step 2: Through a deep text matching algorithm (such as a twin network based on long and short-term memory), get The score of each candidate result; Step 3: take out the candidate scores in step 2, and use the candidate with the highest score as the final matching result. [0003] At present, the mainstream scheme of matching module is the method of deep learning. In order to achieve better results, the deep lear...

AI Technical Summary

Problems solved by technology

In order to achieve better results, the deep learning model has become more and more complex, and the amount of calculation is increasing

Moreover, since the retrieval module obtains N results, N calculations are required. Therefore, when the depth matching algorithm actually lands products, in order to ensure speed, it still tends to use a simple model with a small amount of calculation, and the calculation speed has become the largest. bottleneck

Images