Model reasoning capability optimization method and system and computer readable storage medium

Abstract

The invention provides a model reasoning ability optimization method and system and a computer readable storage medium, in the method, a mixed training data set is generated based on multiple historical dialogue logs, fine-tuning training is performed on partial parameters of recall knowledge data in the mixed training data set, a loss function of a recall knowledge part is not calculated, and the optimization efficiency of the model reasoning ability is improved. The model can focus on reasoning logic optimization instead of external knowledge memory, and has the ability of self-adaptively identifying problem difficulty. Besides, a dialogue data backflow mechanism in a preset period continuously optimizes the mixed training data set, so that the reasoning ability of the model can be optimized in real time, the adaptation accuracy of computing resources is further improved, efficient utilization of the reasoning resources and parallel improvement of the accuracy of reasoning results are finally realized, and the reasoning efficiency is improved. The problems of reasoning resource waste and poor reasoning accuracy caused by static resource control in related technologies are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a model reasoning capability optimization method and system and a computer readable storage medium. BACKGROUND

[0002] In the current inference optimization technology based on knowledge distillation, the powerful reasoning capability of a teacher model is migrated to a student model by applying the teacher model and high-quality training samples, and the reasoning quality is improved by controlling the thinking time of the student model and repeatedly correcting the output answer. Although this scheme can improve the reasoning capability of the student model to a certain extent, the thinking time control process of the student model is static control, which cannot dynamically adjust the allocation of computing resources according to the actual difficulty of the input question, resulting in either excessive thinking or falling into a dead loop due to forced increase of the reasoning process, causing serious waste of reasoning resources, or incomplete thinking due to forced interruption of the reasoning process, thereby causing reasoning result errors. SUMMARY

[0003] Based on the above problems, in order to solve the problems of reasoning resource waste and poor reasoning accuracy caused by static resource control in the related art, the embodiments of the present application provide a model reasoning capability optimization method, system and computer readable storage medium.

[0004] The embodiments of the present application disclose the following technical solutions:

[0005] In a first aspect, the embodiments of the present application provide a model reasoning capability optimization method, comprising:

[0006] According to the mixed training data set, the target model is partially parameter fine-tuning trained to obtain a fine-tuning model; wherein for the recall knowledge data in the mixed training data set, the partial parameter fine-tuning training does not calculate the loss function of the recall knowledge part; the mixed training data set is generated based on multiple historical dialogue logs;

[0007] Based on the fine-tuning model, the inquiry reply content is output according to the current inquiry information and the historical dialogue log;

[0008] Based on a preset model optimization period, the dialogue data associated with the inquiry reply content is backflowed to the mixed training data set to complete the reasoning optimization for the target model.

[0009] In a possible implementation manner, the construction manner of the mixed training data set comprises:

[0010] A plurality of target historical dialogue logs are determined from a plurality of historical dialogue logs; the user interaction stickiness of the target historical dialogue log is greater than a first threshold value;

[0011] perform data filtering processing on the plurality of target historical dialogue logs according to preset dialogue filtering rules, to obtain inquiry reply reference data; the preset dialogue filtering rules include a historical dialogue filtering rule and an agent reply filtering rule, the historical dialogue filtering rule is used to eliminate data with a product relevance lower than a second threshold, and the agent reply filtering rule is used to eliminate data containing user privacy information;

[0012] Based on the inquiry reply reference data, the mixed training data set is constructed.

[0013] In a possible implementation, the outputting of the inquiry reply content based on the fine-tuned model according to the current inquiry information and the historical dialogue logs comprises:

[0014] Based on the fine-tuned model, an inquiry reply strategy and an external tool calling requirement are generated according to the current inquiry information and the historical dialogue logs.

[0015] A knowledge calling tool corresponding to the external tool calling requirement is called, and an external knowledge fragment corresponding to the current inquiry information is determined according to the inquiry reply strategy.

[0016] Based on the fine-tuned model, the inquiry reply content is output according to the current inquiry information, the external knowledge fragment, and the historical dialogue logs.

[0017] In a possible implementation, the mixed training data set includes thinking process data and non-thinking process data, the data proportion of the thinking process data in the mixed training data set is greater than that of the non-thinking process data, the thinking process data includes thinking data for the recalled knowledge data, and the thinking process data includes a calling process for the knowledge calling tool.

[0018] In a possible implementation, the partial parameter fine-tuning training is implemented by using a lora partial parameter training model.

[0019] In a second aspect, the embodiments of the present application provide a model inference capability optimization system, which comprises:

[0020] A model fine-tuning module is configured to perform partial parameter fine-tuning training on a target model according to a mixed training data set, to obtain a fine-tuned model; wherein for recalled knowledge data in the mixed training data set, the partial parameter fine-tuning training does not calculate a loss function of the recalled knowledge part; and the mixed training data set is generated based on a plurality of historical dialogue logs.

[0021] A reply output module is configured to output inquiry reply content based on the fine-tuned model according to current inquiry information and the historical dialogue logs.

[0022] The reasoning optimization module is configured to, based on a preset model optimization period, flow the dialog data associated with the inquiry reply content back to the mixed training data set to complete reasoning optimization for the model.

[0023] In a possible implementation, the mixed training data set is constructed in the following manner:

[0024] determine a plurality of target historical dialog logs from the plurality of historical dialog logs; user interaction stickiness of the target historical dialog logs is greater than a first threshold value;

[0025] perform data filtering processing on the plurality of target historical dialog logs according to a preset dialog filtering rule to obtain inquiry reply reference data; the preset dialog filtering rule includes a historical dialog filtering rule and an agent reply filtering rule, the historical dialog filtering rule is configured to exclude data with a product relevance lower than a second threshold value, and the agent reply filtering rule is configured to exclude data containing user privacy information;

[0026] construct the mixed training data set based on the inquiry reply reference data.

[0027] In a possible implementation, the reply output module is specifically configured to:

[0028] generate an inquiry reply strategy and an external tool calling requirement based on the fine-tuned model and the current inquiry information and the historical dialog log;

[0029] call a knowledge calling tool corresponding to the external tool calling requirement, and determine an external knowledge fragment corresponding to the current inquiry information according to the inquiry reply strategy;

[0030] output the inquiry reply content based on the fine-tuned model, the current inquiry information, the external knowledge fragment, and the historical dialog log.

[0031] In a possible implementation, the mixed training data set includes thinking process data and non-thinking process data, a data proportion of the thinking process data in the mixed training data set is greater than that of the non-thinking process data, the thinking process data includes thinking data for the recalled knowledge data, and the thinking process data includes a calling process for the knowledge calling tool.

[0032] In a third aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement any possible model reasoning capability optimization method in the first aspect.

[0033] Compared to existing technologies, this application offers the following advantages: This application provides a method, system, and computer-readable storage medium for optimizing model reasoning capabilities. In this method, the hybrid training dataset is generated based on multiple historical dialogue logs, and fine-tuning is performed on some parameters of the recalled knowledge data within the hybrid training dataset. The loss function for the recalled knowledge portion is not calculated, enabling the model to focus on optimizing reasoning logic rather than external knowledge memorization, while also possessing the ability to adaptively identify question difficulty. During actual reasoning, the fine-tuned model can directly output the query response for simple questions based on the current query information and historical dialogue logs, avoiding overthinking or infinite loops caused by forcibly increasing the reasoning process, significantly reducing the waste of reasoning resources. For complex questions, it combines external tools to retrieve knowledge fragments before outputting the response, ensuring a complete and sufficient thinking process and effectively avoiding errors caused by forcibly interrupting reasoning. Furthermore, the dialogue data feedback mechanism within a preset period continuously optimizes the hybrid training dataset, enabling the model's reasoning capabilities to be optimized in real time, further improving the accuracy of computational resource adaptation. Ultimately, it achieves efficient utilization of reasoning resources and parallel improvement in the accuracy of reasoning results, effectively solving the problems of wasted reasoning resources and poor reasoning accuracy caused by static resource control in related technologies. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart illustrating a method for optimizing model inference capabilities provided in an embodiment of this application;

[0036] Figure 2 A flowchart illustrating a method for constructing a hybrid training dataset provided in an embodiment of this application;

[0037] Figure 3 A flowchart illustrating a method for outputting query response content as provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the structure of a model reasoning capability optimization system provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and accompanying drawings. It should be particularly noted that the embodiments described in this application are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0040] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0041] As described earlier, current reasoning optimization techniques based on knowledge distillation primarily rely on the application of teacher models and high-quality training samples to transfer the powerful reasoning capabilities of the teacher model to the student model. Reasoning quality is improved by controlling the student model's thinking time and repeatedly revising the output answers. While this approach can enhance the student model's reasoning ability to some extent, its control over the student model's thinking time is static. It cannot dynamically adjust the allocation of computational resources based on the actual difficulty of the input problem. This leads to either excessive thinking or getting stuck in a dead loop due to forced increases in the reasoning process, resulting in a serious waste of reasoning resources, or incomplete thinking due to forced interruptions in the reasoning process, leading to incorrect reasoning results.

[0042] Based on this, embodiments of this application provide a method, system, and computer-readable storage medium for optimizing model reasoning capabilities. In this method, the hybrid training dataset is generated based on multiple historical dialogue logs, and fine-tuning is performed on some parameters of the recalled knowledge data within the hybrid training dataset. The loss function for the recalled knowledge portion is not calculated, enabling the model to focus on optimizing reasoning logic rather than external knowledge memorization, while also possessing the ability to adaptively identify question difficulty. During actual reasoning, the fine-tuned model can directly output the query response for simple questions based on the current query information and historical dialogue logs, avoiding overthinking or infinite loops caused by forcibly increasing the reasoning process, significantly reducing the waste of reasoning resources. For complex questions, it combines external tools to retrieve knowledge fragments before outputting the response, ensuring a complete and sufficient thinking process and effectively avoiding errors caused by forcibly interrupting reasoning. Furthermore, a dialogue data feedback mechanism within a preset period continuously optimizes the hybrid training dataset, enabling the model's reasoning capabilities to be optimized in real time, further improving the accuracy of computational resource adaptation. Ultimately, this achieves efficient utilization of reasoning resources and parallel improvement in the accuracy of reasoning results, effectively solving the problems of wasted reasoning resources and poor reasoning accuracy caused by static resource control in related technologies.

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0044] See Figure 1 The figure is a flowchart illustrating a method for optimizing model inference capability according to an embodiment of this application, specifically including the following steps:

[0045] S101: Based on the hybrid training dataset, perform partial parameter fine-tuning training on the target model to obtain a fine-tuned model; wherein, for the recall knowledge data in the hybrid training dataset, the partial parameter fine-tuning training does not calculate the loss function of the recall knowledge part; the hybrid training dataset is generated based on multiple historical dialogue logs.

[0046] In this embodiment, the hybrid training dataset is generated primarily from multiple historical dialogue logs, providing data support for fine-tuning some parameters of the target model. The training process focuses on fine-tuning some parameters using this hybrid dataset, with the core objective of obtaining a performance-adapted fine-tuned model through targeted optimization. Notably, for the recalled knowledge data included in the hybrid training dataset, the loss function for this part is explicitly set not to be calculated during this partial parameter fine-tuning training.

[0047] Taking insurance product data as an example, the recalled knowledge data originates from objective information such as product terms and premium standards accessed by external tools. Its purpose is to provide accurate references for the model to generate responses, rather than for the model to learn and generate this type of knowledge itself. Therefore, excluding this part from loss calculation can prevent the model from deviating from the core training objective due to overfitting to the expression of recalled knowledge, ensuring that training focuses on generating personalized response strategies and scripts based on the user's actual inquiry content and corresponding historical question-and-answer data. At the same time, not calculating the loss function for the recalled knowledge part can effectively avoid the model's mislearning of objective knowledge, reduce response illusions caused by knowledge generation bias, and thus ensure the accuracy and authority of product information, rules, and other content (in the insurance scenario) involved in the response. In addition, setting the recalled knowledge part can also reduce ineffective training overhead, without requiring the model to consume computational resources on fixed objective knowledge, allowing parameter fine-tuning to more efficiently serve the improvement of key capabilities such as response logic optimization, ultimately enabling the fine-tuned model to flexibly use external knowledge to support question-and-answer communication.

[0048] On the other hand, in one possible implementation, partial parameter fine-tuning training is achieved through LoRA (Low-Rank Adaptation) partial parameter training. The core logic of LoRA is to freeze most of the original parameters of the base model, inserting only a small number of trainable low-rank matrix parameters into the key layers of the model. By training these newly added parameters, the model adapts to specific scenarios without requiring full fine-tuning of the entire base model. This approach effectively reduces computational resource consumption during training, decreases GPU memory usage and training time, and avoids the problems of model overfitting or degradation of original general capabilities that may result from full fine-tuning. In the training scenario of this embodiment, since the mixed training dataset includes both thinking and non-thinking process data, and the loss function is not calculated for the knowledge recall part, LoRA's partial parameter training mode can accurately focus on the core training objective, allowing the target model to learn dialogue response strategies, speech generation logic, and tool invocation processes, without being distracted by adjustments to irrelevant parameters. In addition, LoRA supports flexible parameter expansion and mounting. Based on the needs of different users or scenarios, exclusive training parameters can be quickly added or switched, providing convenience for personalized model adaptation. Ultimately, while ensuring training efficiency and the core capabilities of the model, it achieves efficient adaptation to specific business scenarios.

[0049] Next, the process of constructing the hybrid training dataset will be described in detail with reference to the accompanying drawings of specific process implementation examples.

[0050] See Figure 2 The figure is a flowchart illustrating a method for constructing a hybrid training dataset according to an embodiment of this application, specifically including the following steps:

[0051] S1011: Determine multiple target historical dialogue logs from the multiple historical dialogue logs; the user interaction stickiness of the target historical dialogue logs is greater than a first threshold.

[0052] User interaction stickiness serves as a key indicator for measuring the value of a dialogue, and the first threshold provides a quantifiable standard for the selection process of target historical dialogue logs. For example, in insurance marketing scenarios, user interaction stickiness is not only reflected in the duration or number of dialogue rounds, but also in the user's urgency regarding the product and the effectiveness of the agent's communication strategy. Logs with interaction stickiness exceeding the first threshold often indicate that the user has proactively engaged in in-depth consultations regarding their insurance needs, such as repeatedly asking core questions about premium standards, coverage, and underwriting requirements, or conducting multiple rounds of communication with the agent regarding key information such as product suitability and the purchase process. These logs contain the user's genuine needs and the efficient communication logic of high-quality agents, thus serving as core high-quality material for agent training. By using user interaction stickiness exceeding the first threshold as the selection criterion for target historical dialogue logs, low-quality and invalid data can be accurately eliminated, such as brief and irrelevant small talk, one-off consultations without follow-up, and discussions deviating from insurance product topics. This ensures that the target historical dialogue logs are highly targeted and valuable, laying the foundation for the subsequent refinement of personalized response strategies and model training.

[0053] S1012: Perform data filtering processing on multiple target historical dialogue logs according to preset dialogue filtering rules to obtain question and answer response reference data; the preset dialogue filtering rules include historical dialogue filtering rules and agent response filtering rules, the historical dialogue filtering rules are used to remove data whose relevance to the inquiry product is lower than a second threshold; the agent response filtering rules are used to remove data containing user privacy information.

[0054] After filtering based on user interaction engagement, the target historical dialogue logs already possess certain high-quality attributes. However, they may still contain content irrelevant to the core needs or that does not conform to data standards. To ensure the quality of data used for subsequent model training, further data filtering processing is required on multiple target historical dialogue logs according to preset dialogue filtering rules, ultimately obtaining accurate question-and-answer response reference data. Specifically, the preset dialogue filtering rules consist of historical dialogue filtering rules and agent response filtering rules. These two types of rules work together to strictly control the data from two dimensions: the relevance of dialogue content and the compliance of responses.

[0055] The historical dialogue filtering rules serve to eliminate data whose relevance to the inquired product falls below a second threshold, ensuring that the retained dialogue data focuses on the core inquiry scenario. In the target historical dialogue logs, some dialogue content may deviate from the user's core inquiry about the product. For example, there may be detailed discussions involving non-target products, ambiguous product references due to system breakpoints, or a very low proportion of core product-related information and a scattered topic. Including such data in subsequent use would distract the model's training focus, resulting in responses that cannot accurately match the user's inquiry about the target product. By setting a second threshold as a relevance judgment standard, highly relevant dialogue data can be quantitatively filtered out. When the frequency of mentioning the target product, the depth of discussion of core information, and the user's inquiry focus in the dialogue are all below this threshold, it is judged as insufficiently relevant and is removed. The application of historical dialogue filtering rules effectively filters out irrelevant and interfering data, ensuring that the final retained dialogue data always revolves around the user's inquiry about the target product, guaranteeing that the reference data for Q&A responses is targeted.

[0056] Furthermore, the agent response screening rules focus on data compliance, specifically designed to remove data containing user privacy information. During interactions between the agent and the user, some responses may contain user privacy information, such as personal identification information and contact details, due to communication scenarios or operational oversights. Retention of such information may pose a risk of user privacy leaks. The agent response screening rules use clear privacy information identification standards to screen agent responses. Once a response containing privacy information is detected, the corresponding dialogue data is directly removed. This screening action not only meets the stringent requirements of regulatory compliance but also effectively protects the legitimate rights and interests of users. It ensures that the final question-and-answer response reference data is valuable under the premise of safety and compliance, guaranteeing that subsequent response strategies and script templates built based on this data do not involve privacy information. This maintains service compliance and creates a safe communication environment for users.

[0057] S1013: Construct the hybrid training dataset based on the question-and-answer response reference data.

[0058] Based on the question-and-answer response reference data obtained after filtering according to preset rules, the construction of the hybrid training dataset will use this data as the foundation. Specifically, in the process of constructing the hybrid training dataset based on the question-and-answer response reference data, it is necessary to extract key information from the reference data, including historical dialogue context, core user queries, adapted response strategies, and compliant and standardized response scripts, ensuring that no core and effective information is omitted. Subsequently, in combination with the actual needs of model training, the extracted information is formatted and adapted. For example, according to the input-output training logic, "historical dialogue + current user message" is set as the input structure, and "response strategy + recalled knowledge + response script" is set as the output structure. At the same time, dynamic adjustments are made for extremely long data to ensure that the overall data length meets the constraints of model training. In addition, to improve the diversity and training adaptability of the dataset, probability sampling will be performed based on the dialogue turn distribution characteristics of the reference data to reasonably balance the data proportion of different scenarios and different interaction depths, ultimately forming a hybrid training dataset that is targeted, complete, and adaptable.

[0059] Furthermore, the hybrid training dataset also includes thinking process data and non-thinking process data, with a significantly higher proportion of thinking process data compared to non-thinking process data. This design aims to allow the model to more fully learn the reasoning logic and knowledge application capabilities in complex scenarios. The thinking process data, as a core component of the dataset, contains two key elements: firstly, thinking process data related to recalled knowledge, which records the complete logical trajectory of the model's selection, analysis, integration, and adaptation when faced with various externally recalled reference knowledge, reflecting the integration of knowledge with specific dialogue scenarios; secondly, the thinking process data also covers the process of calling knowledge-requesting tools, detailing how the model selects appropriate knowledge-requesting tools based on dialogue needs, triggers the tool call process, and obtains effective knowledge fragments when supplementing external knowledge. Thus, the thinking process data complements the non-thinking process data, ensuring efficient model response in simple scenarios while, through the higher proportion of thinking process data, allowing the model to fully learn the core logic of knowledge application and tool call in complex dialogues, providing solid data support for the subsequent model to achieve "read-as-you-go" intelligent response capabilities.

[0060] The above is a flowchart describing the method for constructing the hybrid training dataset in this application example. The following will continue to combine... Figure 1 The subsequent execution flow of the model inference capability optimization method in this example will be introduced.

[0061] S102: Based on the fine-tuning model, output the query response content according to the current query information and the historical dialogue log.

[0062] Based on a model with fine-tuned parameters, the output of query responses is guided by the core needs of the current query, while deeply integrating contextual information from historical dialogue logs to generate responses that fit the specific scenario. Leveraging the dialogue logic and knowledge application capabilities learned during training, the fine-tuned model can extract key information from historical dialogue logs, such as past user needs, communication preferences, and core concerns, clearly defining the connection between the current query and historical interactions, preventing responses from becoming detached from context. For the specific needs of the current query, the model invokes the thinking logic and knowledge retrieval capabilities solidified during training, integrating effective information and relevant reference knowledge from historical dialogues. Through rapid reasoning and adaptation, it generates query responses that address the core of the current query while fitting the past communication context, ensuring coherence and relevance, and fully meeting the user's actual consultation needs.

[0063] Next, with reference to the accompanying drawings of a specific process embodiment, the execution process of outputting the query response content through fine-tuning the model in step S102 will be described in detail. See also Figure 3 The figure is a flowchart illustrating a query response output method provided in an embodiment of this application, specifically including the following steps:

[0064] S1021: Based on the fine-tuning model, generate an inquiry response strategy and external tool call requirements according to the current inquiry information and the historical dialogue log.

[0065] When generating inquiry response strategies based on a model with fine-tuned parameters, the current inquiry information needs to be used as an anchor point to extract key information from historical dialogue logs. The model accurately extracts the user's past needs and communication preferences from historical dialogue logs, such as whether the user has repeatedly inquired about the details of a certain type of service, and what information they have shown significant interest in. This information is then combined with the specific needs, tone of voice, and context of the current inquiry to comprehensively determine the user's true intentions. Relying on the thinking logic and personalized adaptation capabilities learned during training, the fine-tuned model will formulate inquiry response strategies that are both targeted and coherent. For example, if historical dialogues show that the user values ​​efficient and concise information presentation, and the current inquiry is a clear query-type need, the strategy will focus on quickly reaching the core answer; if the user has previously expressed concerns about service security, and the current inquiry involves related operational procedures, the strategy will proactively incorporate security explanations while responding to the procedures, ensuring that the response addresses the current needs while also fitting the historical communication context.

[0066] The generation of external tool call requests also relies heavily on the fine-tuning model's comprehensive analysis of current inquiry information and historical dialogue logs. In generating these requests, the model first determines whether the current inquiry requires objective data, real-time information, or dedicated resources from external tools for an accurate response. This determination is based on information gaps in historical dialogues, such as specific parameter queries not mentioned in past conversations, real-time data requiring verification, or dedicated service interfaces that need to be invoked. When it's determined that external tool support is needed, the model, based on the tool call logic learned during training, breaks down the core requirements of the current inquiry, identifies the corresponding tool type, and generates specific external tool call requests. For example, if the current inquiry involves the specific price of a certain service, but historical dialogues do not contain this price information, the model will generate a call request for a premium query interface; if the inquiry involves confirming policy details, and historical dialogues lack complete information, it will trigger a call request for a product policy knowledge query interface.

[0067] S1022: Call the knowledge retrieval tool corresponding to the external tool retrieval requirement, and determine the external knowledge fragment corresponding to the current query information according to the query response strategy.

[0068] After generating a clear external tool call request, the system accurately invokes the corresponding knowledge-based tool. The entire process is guided by a pre-defined inquiry response strategy, ultimately identifying the external knowledge fragment that best matches the current inquiry. The selection of the knowledge-based tool corresponds to the type of external tool call request, ensuring that the tool's functionality matches the requirement. For example, if the request is to query the terms and conditions, the system calls the product terms knowledge query interface; if the request is to verify the price, it triggers the premium query interface. During the tool call process, the inquiry response strategy plays a crucial guiding role. It clarifies the core direction, key dimensions, and user concerns of the response. Based on this strategy, the tool performs targeted filtering, precise retrieval, and efficient extraction of massive amounts of external knowledge, eliminating information irrelevant, redundant, or illogical to the current inquiry, focusing on information that supports the implementation of the response strategy. The final determined external knowledge fragment is both objective and accurate, closely aligned with the current inquiry scenario and user needs, providing external knowledge support for subsequent information integration to generate high-quality inquiry responses.

[0069] S1023: Based on the fine-tuning model, output the query response content according to the current query information, the external knowledge fragments, and the historical dialogue logs.

[0070] Based on a model with fine-tuned parameters, the output of query responses comprehensively integrates three key elements: the current query information, external knowledge fragments, and historical dialogue logs. This ensures the accuracy, coherence, and professionalism of the responses. The model first analyzes the user's past communication history, needs, preferences, and potential concerns from historical dialogue logs to ensure the response aligns with the long-term communication context and avoids detachment from the surrounding text. Simultaneously, it focuses on the core demands of the current query, clarifying the core direction of the response. Then, it uses professional knowledge from external knowledge fragments to compensate for the model's internal knowledge limitations. Through reasoning logic and expression adaptability learned during training, the model integrates and connects these three types of information. This ensures the response accurately addresses the core needs of the current query, maintains communication coherence through historical dialogues, and enhances the accuracy and persuasiveness of the content by relying on external knowledge fragments. Ultimately, it generates high-quality query responses that meet the needs of the scenario and align with user expectations.

[0071] S103: Based on the preset model optimization cycle, the dialogue data associated with the query response content is fed back to the hybrid training dataset to complete the inference optimization for the model.

[0072] Finally, based on a pre-defined model optimization cycle, the dialogue data associated with the query and response content is fed back into the hybrid training dataset to achieve cyclical updates to the model. This associated dialogue data originates from real-world customer interaction scenarios, comprehensively recording users' latest queries, feedback on responses, and dynamic changes in needs during actual communication. This not only maintains the hybrid training dataset's adaptability to business scenarios but also supplements it with information on new scenarios and evolving needs not covered by the initial data, effectively addressing potential issues such as insufficient timeliness and incomplete scenario coverage in the initial dataset. Feeding this data back into the hybrid training dataset enables the cyclical use of data resources, ensuring that the training data remains synchronized with actual business dynamics, providing high-quality data support that best meets real-world needs for the model's final inference optimization.

[0073] This application provides a method for optimizing model reasoning capabilities. In this method, a hybrid training dataset is generated based on multiple historical dialogue logs, and fine-tuning is performed on some parameters of the recalled knowledge data within the hybrid training dataset. The loss function for the recalled knowledge portion is not calculated, allowing the model to focus on optimizing reasoning logic rather than external knowledge memorization, while also possessing the ability to adaptively identify question difficulty. During actual reasoning, the fine-tuned model can directly output the query response for simple questions based on the current query information and historical dialogue logs, avoiding overthinking or infinite loops caused by forcibly increasing the reasoning process, significantly reducing the waste of reasoning resources. For complex questions, it combines external tools to retrieve knowledge fragments before outputting the response, ensuring a complete and sufficient thought process and effectively avoiding errors caused by forcibly interrupting reasoning. Furthermore, a dialogue data feedback mechanism within a preset period continuously optimizes the hybrid training dataset, enabling real-time optimization of the model's reasoning capabilities, further improving the accuracy of computational resource adaptation. Ultimately, this achieves efficient utilization of reasoning resources and parallel improvement in the accuracy of reasoning results, effectively solving the problems of wasted reasoning resources and poor reasoning accuracy caused by static resource control in related technologies.

[0074] The following describes a model reasoning capability optimization system provided in the embodiments of this application. The model reasoning capability optimization system described below and the model reasoning capability optimization method described above can be referred to in correspondence.

[0075] See Figure 4 The figure is a schematic diagram of the structure of a model reasoning capability optimization system provided in an embodiment of this application, which specifically includes the following modules:

[0076] The model fine-tuning module 100 is used to perform partial parameter fine-tuning training on the target model based on the mixed training dataset to obtain a fine-tuned model; wherein, for the recalled knowledge data in the mixed training dataset, the partial parameter fine-tuning training does not calculate the loss function of the recalled knowledge part; the mixed training dataset is generated based on multiple historical dialogue logs;

[0077] The response output module 200 is used to output the query response content based on the fine-tuning model, the current query information, and the historical dialogue log.

[0078] The inference optimization module 300 is used to feed back the dialogue data associated with the query response content to the hybrid training dataset based on a preset model optimization cycle, so as to complete the inference optimization for the model.

[0079] In one possible implementation, the hybrid training dataset is constructed in the following ways:

[0080] Multiple target historical dialogue logs are determined from the multiple historical dialogue logs mentioned above; the user interaction stickiness of the target historical dialogue logs is greater than a first threshold.

[0081] Data filtering processing is performed on multiple target historical dialogue logs according to preset dialogue filtering rules to obtain question and answer response reference data; the preset dialogue filtering rules include historical dialogue filtering rules and agent response filtering rules, the historical dialogue filtering rules are used to remove data whose relevance to the inquiry product is lower than a second threshold; the agent response filtering rules are used to remove data containing user privacy information.

[0082] Based on the question-and-answer response reference data, the hybrid training dataset is constructed.

[0083] In one possible implementation, the response output module 200 is specifically used for:

[0084] Based on the fine-tuning model, an inquiry response strategy and external tool call requirements are generated according to the current inquiry information and the historical dialogue logs.

[0085] The external tool is invoked to access the knowledge access tool corresponding to the request, and the external knowledge fragment corresponding to the current query information is determined according to the query response strategy.

[0086] Based on the fine-tuning model, the query response content is output according to the current query information, the external knowledge fragments, and the historical dialogue logs.

[0087] In one possible implementation, the hybrid training dataset includes thinking process data and non-thinking process data, wherein the proportion of thinking process data in the hybrid training dataset is greater than that of non-thinking process data, and the thinking process data includes thinking data for the recalled knowledge data, and the thinking process data includes the invocation process for the knowledge retrieval tool.

[0088] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a computer-readable storage medium storing computer instructions for causing the computer to execute the model reasoning capability optimization method as described in any of the above embodiments.

[0089] The computer-readable media in this application embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0090] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the model inference capability optimization method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0091] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the system, method, and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The system, method, and medium embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0092] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for optimizing model reasoning ability, characterized in that, include: Based on the hybrid training dataset, the target model is partially fine-tuned to obtain a fine-tuned model; wherein, for the recalled knowledge data in the hybrid training dataset, the partial parameter fine-tuning training does not calculate the loss function of the recalled knowledge part; the hybrid training dataset is generated based on multiple historical dialogue logs; Based on the fine-tuning model, the query response content is output according to the current query information and the historical dialogue log; Based on a preset model optimization cycle, the dialogue data associated with the query response content is fed back to the hybrid training dataset to complete the inference optimization for the target model.

2. The method according to claim 1, characterized in that, The hybrid training dataset is constructed in the following ways: Multiple target historical dialogue logs are determined from the multiple historical dialogue logs mentioned above; the user interaction stickiness of the target historical dialogue logs is greater than a first threshold. Data filtering processing is performed on multiple target historical dialogue logs according to preset dialogue filtering rules to obtain question and answer response reference data; the preset dialogue filtering rules include historical dialogue filtering rules and agent response filtering rules, the historical dialogue filtering rules are used to remove data whose relevance to the inquiry product is lower than a second threshold; the agent response filtering rules are used to remove data containing user privacy information. Based on the question-and-answer response reference data, the hybrid training dataset is constructed.

3. The method according to claim 1, characterized in that, The step of outputting query response content based on the fine-tuned model, according to the current query information and the historical dialogue log, includes: Based on the fine-tuning model, an inquiry response strategy and external tool call requirements are generated according to the current inquiry information and the historical dialogue logs. The external tool is invoked to access the knowledge access tool corresponding to the request, and the external knowledge fragment corresponding to the current query information is determined according to the query response strategy. Based on the fine-tuning model, the query response content is output according to the current query information, the external knowledge fragments, and the historical dialogue logs.

4. The method according to claim 3, characterized in that, The hybrid training dataset includes thinking process data and non-thinking process data. The proportion of thinking process data in the hybrid training dataset is greater than that of non-thinking process data. The thinking process data includes thinking data related to the recalled knowledge data and the calling process related to the knowledge retrieval tool.

5. The method according to claim 1, characterized in that, The partial parameter fine-tuning training is implemented using the LoRa partial parameter training model.

6. A model reasoning capability optimization system, characterized in that, include: The model fine-tuning module is used to fine-tune some parameters of the target model based on the mixed training dataset to obtain a fine-tuned model; wherein, for the recalled knowledge data in the mixed training dataset, the partial parameter fine-tuning training does not calculate the loss function of the recalled knowledge part; the mixed training dataset is generated based on multiple historical dialogue logs; The response output module is used to output the response content based on the fine-tuning model, the current query information, and the historical dialogue log. The inference optimization module is used to feed back the dialogue data associated with the query response content to the hybrid training dataset based on a preset model optimization cycle, so as to complete the inference optimization for the model.

7. The system according to claim 6, characterized in that, The hybrid training dataset is constructed in the following ways: Multiple target historical dialogue logs are determined from the multiple historical dialogue logs mentioned above; the user interaction stickiness of the target historical dialogue logs is greater than a first threshold. Data filtering processing is performed on multiple target historical dialogue logs according to preset dialogue filtering rules to obtain question and answer response reference data; the preset dialogue filtering rules include historical dialogue filtering rules and agent response filtering rules, the historical dialogue filtering rules are used to remove data whose relevance to the inquiry product is lower than a second threshold; the agent response filtering rules are used to remove data containing user privacy information. Based on the question-and-answer response reference data, the hybrid training dataset is constructed.

8. The system according to claim 6, characterized in that, The response output module is specifically used for: Based on the fine-tuning model, an inquiry response strategy and external tool call requirements are generated according to the current inquiry information and the historical dialogue logs. The external tool is invoked to access the knowledge access tool corresponding to the request, and the external knowledge fragment corresponding to the current query information is determined according to the query response strategy. Based on the fine-tuning model, the query response content is output according to the current query information, the external knowledge fragments, and the historical dialogue logs.

9. The system according to claim 8, characterized in that, The hybrid training dataset includes thinking process data and non-thinking process data. The proportion of thinking process data in the hybrid training dataset is greater than that of non-thinking process data. The thinking process data includes thinking data related to the recalled knowledge data and the calling process related to the knowledge retrieval tool.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the model reasoning capability optimization method as described in any one of claims 1-5.

Images