A tool-integrated reasoning reinforcement learning system and method based on multi-dimensional rewards

By employing multi-dimensional reward calculation and inter-group relative policy optimization algorithms, the problem of traditional reward functions being unable to provide refined evaluation in tool invocation tasks is solved, enabling efficient learning and accurate invocation of large language models in tool integration reasoning tasks.

CN121503575BActive Publication Date: 2026-11-10ZHEJIANG UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610040344.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-01-13
Publication Date
2026-11-10
Estimated Expiration
2046-01-13

AI Technical Summary

Technical Problem

Existing reinforcement learning reward function designs cannot perform fine-grained evaluation of key details such as tool name, parameter format, and parameter value in tool invocation tasks, resulting in low model learning efficiency and high evaluation costs for large models, making it difficult to simultaneously ensure the accuracy and efficiency of multi-dimensional evaluation.

Method used

A multi-dimensional reward calculation module is adopted, including format reward and multi-dimensional correctness reward. A comprehensive reward tensor is generated through linear combination. Combined with the inter-group relative policy optimization algorithm, token-level advantage value calculation and gradient update are realized, driving the integration of inference capabilities of large language model optimization tools.

Benefits of technology

It significantly improves the convergence speed and accuracy of the model in tool invocation tasks, enhances the signal strength and comprehensiveness of key tokens, and improves the accuracy and training efficiency of the model in tool integration inference tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121503575B_ABST
    Figure CN121503575B_ABST
Patent Text Reader

Abstract

This invention discloses a tool-integrated reasoning reinforcement learning system and method based on multi-dimensional rewards, belonging to the fields of natural language processing and reinforcement learning. The invention includes: a tool-integrated reasoning prompt word design module that provides predefined prompt words and tag sequences to train a large language model to autonomously generate reasoning trajectories, selecting to invoke tools or output a final response; a multi-dimensional reward calculation module that constructs multi-dimensional reward signals based on tool invocation or final response results, including format rewards and multi-dimensional correctness rewards; a reward fusion module that integrates reward signals from different dimensions to generate a comprehensive reward tensor; and a model training module that trains the large language model based on the comprehensive reward tensor and an inter-group relative policy optimization algorithm to achieve autonomous reasoning. This invention can significantly improve the accuracy and training efficiency of large language models in tool-integrated reasoning tasks. In complex tool invocation scenarios, the multi-dimensional reward mechanism guides the model to accurately generate tool names and parameters.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Method and device for optimizing large-scale language model calling tool, electronic equipment and computer readable medium

    CN120872444A