Big model thinking chain evaluation method and system for formalized theorem proving

By constructing and evaluating a large-scale model thought chain for formal theorem proofs, and optimizing training data and strategies, the problem of poor interpretability of large models in formal theorem proofs is solved, the efficiency and accuracy of proofs are improved, and the security of computer systems is ensured.

CN118709791BActive Publication Date: 2026-06-02THE QUARTERMASTER RES INST OF THE GENERAL LOGISTICS DEPT OF THE CPLA
View PDF 4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
THE QUARTERMASTER RES INST OF THE GENERAL LOGISTICS DEPT OF THE CPLA
Filing Date
2024-06-26
Publication Date
2026-06-02

AI Technical Summary

Technical Problem

Existing technologies for formal theorem proofs lack comprehensive methods for evaluating thought processes in large models, making it difficult to understand their internal reasoning processes and decision-making mechanisms. This makes it impossible to determine whether the large model truly understands the rules and logic of the proof, affecting the improvement of training data and strategies.

Method used

By constructing an original dataset containing a state change tree and a premise library, a baseline model is trained. By evaluating proof success rate, accuracy, completeness, attention weights, and thought chain cues, the dependent data of the large model during inference is determined, and the dataset and training method are modified to optimize the training strategy.

Benefits of technology

It provides a method for evaluating the thought chain of large models for formal theorem proofs, optimizes training data and strategies, improves the interpretability and proof efficiency of the model, and ensures the accuracy of computer system security checks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118709791B_ABST
    Figure CN118709791B_ABST
Patent Text Reader

Abstract

The application discloses a large model thinking chain evaluation method and system for formalized theorem proving, and the method comprises the following steps: constructing an original data set and training a large model to obtain a trained large model as a benchmark model; based on the original data set, a first data set and a second data set are constructed, benchmark intermediate data, first intermediate data and second intermediate data are generated, and the first performance of the benchmark model is determined; based on the comparison result of the benchmark attention data and third attention data, the second performance of the benchmark model is determined; based on the comparison result of the benchmark thinking chain prompt data and fourth benchmark thinking chain prompt data, the third performance of the benchmark model is determined; based on the first performance, the second performance and the third performance, the original data set is modified to obtain a modified training set and a training method; and the trained large model is trained again based on the modified training set to obtain a new trained large model.
Need to check novelty before this filing date? Find Prior Art