The invention discloses a large
language model evaluation method,
system and device, a medium and a program product, and the method comprises the steps: firstly, carrying out the
pollution detection of an obtained benchmark
test data set through a
data detection method, obtaining first suspected
pollution data, and then according to the type of the benchmark
test data set, obtaining second suspected
pollution data; data
rewriting is conducted on the first suspected pollution data to obtain second suspected pollution data,
data detection is conducted on the second suspected pollution data again through a
data detection method till a preset stop condition is met, a benchmark
test data set after pollution removal is obtained, and finally the data of the second suspected pollution data are detected on the basis of the benchmark test
data set after pollution removal. And evaluating the large
language model. Potential pollution of the benchmark test
data set can be reduced, performance expansion of the large
language model caused by remembering the leaked benchmark test is effectively relieved, the
evaluation result of the large language model is more credible, and the real ability of the model can be better reflected.