Intelligent evaluation method and device for large model and medium

Through intelligent evaluation methods, custom evaluation is performed using referee model and benchmarking model, which solves the problem that traditional evaluation technology is difficult to evaluate deep understanding and creative generation ability, achieves the fairness and reliability of evaluation results, and adapts to the rapid development of large-scale model technology.

CN120179794AActive Publication Date: 2025-06-20INSPUR GENERSOFT CO LTD
11 Cites 2 Cited by

Patent Information

Application Number
CN202510652924.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-21
Publication Date
2025-06-20
Estimated Expiration
2045-05-21

AI Technical Summary

Technical Problem

Traditional big model evaluation technology is difficult to effectively evaluate the evaluation tasks that require deep understanding, reasoning ability and creative generation ability, and is prone to problems such as "model cheating" and fixed evaluation indicators, which are difficult to adapt to the rapid development of big models and their application scenarios.

Method used

An intelligent evaluation method is proposed, by obtaining the measured model, referee model and benchmarking model, defining scoring criteria and weights, obtaining user-defined evaluation information, performing intelligent evaluation based on the preset evaluation set, and supporting custom evaluation dimensions and confrontation dialogues to improve the fairness and reliability of evaluation results.

Benefits of technology

It realizes the fairness and reliability of intelligent evaluation results, supports custom evaluation dimensions and scenarios, reduces evaluation complexity, improves evaluation efficiency and accuracy, and adapts to the rapid development of large-scale model technology.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses an intelligent evaluation method and device for a large model and a medium, and relates to the field of large models.The method comprises the steps that a tested model, a judgment model and a benchmarking model are obtained, score definition is conducted, and a first cue word is maintained; obtaining self-defined evaluation information defined by a user based on scores; according to the self-defined evaluation information, on the basis of evaluation questions in a preset evaluation set, respectively asking questions for the tested model and the benchmarking model, and respectively obtaining a corresponding first output answer and a corresponding second output answer; and performing intelligent evaluation on the tested model through the judgment model. According to the method, decision making is assisted through the judgment model, intelligent scoring is performed according to each evaluation problem, a scoring basis is provided, an intelligent evaluation result is generated, manual decision-making scoring is assisted, a complete evaluation standard is set, consistency and complexity reduction in the evaluation process are ensured, and fairness and reliability of the evaluation result are ensured at the same time.
Need to check novelty before this filing date? Find Prior Art