Neural symbol collaborative distillation method for improving complex reasoning of small language model

Abstract

The invention provides a neural symbol collaborative distillation method for improving complex reasoning of a small language model, and the method comprises the steps: firstly, extracting general reasoning capability and general knowledge from a large language model, and distilling the general reasoning capability and general knowledge into parameterized SLMs; secondly, aiming at specific capabilities and rare knowledge required by a complex reasoning task, storing the knowledge in a symbol knowledge base by adopting a symbol knowledge distillation means; by decoupling this general capability from a specific capability, high performance can be achieved with a smaller model scale and cost. In addition, the invention further comprises a multi-task learning method, by introducing auxiliary tasks such as enhanced distillation, answer prediction and direct reasoning prompt, the robustness of the SLMs to possible noise knowledge is further improved, and specific knowledge is effectively utilized. According to the method, the capability of processing complex reasoning tasks by a small language model is enhanced, and the method is suitable for complex reasoning scenes with requirements for general cognitive competence and professional domain knowledge.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and large model reasoning, and specifically relates to a neural symbolic collaborative distillation method for improving complex reasoning of small language models. Background Art

[0002] In recent years, large language models (LLMs), such as ChatGPT, Llama3, etc., have demonstrated excellent performance on a variety of complex reasoning tasks. These models are trained with massive amounts of data and are able to capture the deep semantics and patterns in language, thus achieving remarkable achievements in mathematical reasoning, common sense reasoning, and symbolic reasoning. However, large language models are computationally expensive and their requirements for environmental resources limit their large-scale deployment and practical applications. In addition, their complexity and large number of parameters also pose challenges to the interpretability and maintainability of the models. To address these issues, researchers have begun to explore how to migrate the advanced reasoning capabilities of large language models to smaller and more efficient models.

[0003] As a model compression method, the core idea of ​​knowledge distillation (KD) technology is to transfer the knowledge of a large and complex model (teacher model) to a smaller and simpler model (student model). Traditional knowledge distillation methods mainly focus on transferring the output probability distribution of the teacher model to the student model, but this method has limitations in handling complex reasoning tasks, especially in scenarios that require specific domain knowledge and advanced reasoning skills.

[0004] In addition, existing research has also attempted to combine external knowledge bases (KB) with language models to enhance the model's reasoning capabilities. These knowledge-augmented language models (KALMs) improve performance by retrieving and integrating external knowledge. However, these methods usually face problems with the accuracy and efficiency of knowledge retrieval, as well as the challenge of how to effectively integrate the retrieved knowledge into the model's reasoning process. Summary of the invention

[0005] To solve the above technical problems, the present invention provides a Neural-Symbolic Collaborative Distillation (NesyCD) method for improving the complex reasoning of small language models, aiming to enhance the ability of small language models (SLMs) to handle complex reasoning tasks. This method is particularly suitable for complex reasoning scenarios that require both general cognitive abilities and professional domain knowledge. NesyCD is implemented through two main steps: first, general reasoning abilities and general knowledge are extracted from large language models (LLMs) and distilled into parameterized SLMs; second, symbolic knowledge distillation is used to store specific abilities and rare knowledge required for complex reasoning tasks in a symbolic knowledge base (KB). Through the decoupling of general abilities from specific abilities, NesyCD can achieve high performance at a smaller model scale and cost. In addition, the present invention also includes a multi-task learning method, which further improves the robustness of SLMs to noisy knowledge and effectively utilizes specific capabilities and rare knowledge by introducing auxiliary tasks such as augmented distillation (AD), answer prediction (AP) and direct chain-of-thought (DC).

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A neural symbolic co-distillation method for improving complex reasoning in small language models, the method comprising the steps of:

[0008] Step 1: General ability distillation: Use the teacher model to process the training data set to generate high-quality reasoning processes and answers; through supervised learning, train the initial student model to imitate the reasoning process and answers generated by the teacher model to obtain a primary student model to generate general reasoning capabilities and general knowledge;

[0009] Step 2, demonstration case collection: Use the primary student model to process the training data set, determine which knowledge can be learned and which cannot be learned, collect the learned knowledge as the correct case set, and the unlearned knowledge as the error case set;

[0010] Step 3: Symbolic knowledge distillation: Use the teacher model to analyze the set of error cases, extract specific capabilities and rare knowledge in specific fields, and store the extracted specific capabilities and rare knowledge in the symbolic knowledge base.

[0011] Step 4: Neural distillation enhanced by symbolic knowledge base: Using a multi-task learning strategy, the primary student model is trained to obtain an extended student model, which is used to generate reasoning processes and answers based on the questions and the retrieved specific capabilities and rare knowledge.

[0012] Furthermore, the multi-task learning strategy in step 4 includes at least one auxiliary task, and the auxiliary task is enhanced distillation, answer prediction or direct reasoning prompt.

[0013] Furthermore, the symbolic knowledge base in step 4 is generated by analyzing error cases of the primary student model on a specific task.

[0014] Furthermore, the step 4 includes that the extended student model dynamically retrieves specific capabilities and rare knowledge in the symbolic knowledge base according to confidence during the reasoning process.

[0015] Furthermore, step 4 includes that the performance of the extended student model in complex reasoning tasks is close to or exceeds that of a large language model.

[0016] The beneficial effects of the present invention are:

[0017] This paper proposes an innovative Neural Symbolic Co-Distillation (NesyCD) method, which can not only transfer the general reasoning capabilities of large language models to small models, but also effectively store and manage domain-specific expertise through symbolic knowledge bases, thereby significantly improving the performance of small language models in complex reasoning tasks. The NesyCD method provides a more cost-effective, easier to understand and maintain solution for small language models by decoupling general and specific capabilities and combining multi-task learning strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the comparison between the NesyCD method and the traditional CoT distillation method;

[0019] Figure 2 Logic diagram of the neural symbolic co-distillation method for improving complex reasoning of small language models in the present invention;

[0020] Figure 3 It is a comparison diagram of the implementation effect of the method of the present invention and the prior art;

[0021] Figure 4 The influence of the confidence threshold on the model performance according to the embodiment of the present invention;

[0022] Figure 5 This is a diagram of the ablation experiment results of the present invention. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0024] Existing methods often ignore the gap between knowledge modeling and complex reasoning capabilities of LLM and SLM. Due to the small number of model parameters, it is difficult for the student model to obtain the comprehensive knowledge required for complex reasoning tasks. Figure 1 As shown, the SLM trained with traditional CoT distillation cannot solve the problem of the teacher model. Neural knowledge distillation faces challenges in representing sparse knowledge and expertise due to its limited parameter space. In complex reasoning tasks, especially in difficult problems, expertise may contradict general knowledge (for example, velocity superposition in relative motion). This sparse expertise is difficult to model in a small-scale model, which seriously affects the performance of SLM on challenging problems and reduces its ability to handle unknown tasks. Therefore, the present invention extracts expertise using symbolic distillation technology and stores it in a knowledge base. Specific knowledge is dynamically retrieved according to the confidence threshold during testing.

[0025] like Figure 2 As shown, this is a logic diagram of the neural symbolic collaborative distillation method for improving the complex reasoning of a small language model in the present invention. First, a large language model (LLM) is selected as a teacher model, which has the ability to handle complex reasoning tasks, and a small language model (SLM) is selected, which will serve as the initial student model to receive knowledge transfer.

[0026] The method comprises four steps, specifically comprising:

[0027] Step 1: General capability distillation: Use the teacher model to process a set of training data sets to generate high-quality reasoning processes (rationales) and answers. Through supervised learning, the initial student model is trained to imitate the reasoning process and answers generated by the teacher model to obtain the primary student model (Primary Learning Model, SP). By learning the reasoning process of the large model, the small model can model relatively simpler and more general reasoning capabilities and general knowledge.

[0028] Step 2: Demonstration case collection: Use the primary student model (SP) to process the training data set, test the training data set, determine which knowledge can be learned and which cannot be learned, classify the knowledge that cannot be learned (i.e., errors) as relatively difficult specific knowledge, and collect correct cases (i.e., learned knowledge) and error cases (i.e., knowledge that cannot be learned). The error cases are used for the next step of symbolic knowledge extraction and storage.

[0029] Step 3, symbolic knowledge distillation: Use the teacher model to analyze error cases and extract domain-specific expertise, including learning summaries and supplementary knowledge points. Store the extracted specific capabilities and rare knowledge in the symbolic knowledge base (KB).

[0030] Step 4, Neural distillation enhanced by symbolic knowledge base: Using multi-task learning methods, further train the primary student model to obtain the enhanced student model (SE), enabling it to generate reasoning processes and answers based on the questions and retrieved specific capabilities and rare knowledge. Implement a retrieval mechanism to retrieve the most relevant knowledge points from the symbolic knowledge base based on the relevance of the question.

[0031] The multi-task learning method designs and implements auxiliary tasks, including: Augmented Distillation (AD): directly using professional knowledge to assist reasoning. Answer Prediction (AP): training the model to directly answer questions that do not require a reasoning process. Direct Chain-of-Thought (DC): using the model's internal knowledge to solve simpler problems. In the reasoning stage, for a question, the confidence of the prediction of the first 16 tokens is calculated. If it is higher than the threshold, it means that it is confident enough and no additional retrieval is required; if it is lower than the threshold, it is necessary to retrieve the specific knowledge related to the closest question from a specific knowledge base.

[0032] Symbolic distillation refers to a technique in machine learning that extracts knowledge from complex, often opaque models (such as deep neural networks) and converts it into a symbolic, more easily understood format. This approach combines the traditional principle of knowledge distillation with the principle of symbolization, using symbolic distillation technology to prompt the teacher model (big model) to generate learning summaries and supplementary knowledge for problems and errors that can help solve such problems. At the same time, it is also necessary to prompt the big model to generate reasoning processes (rationale) for each data point in the training set. Neural distillation is reflected in updating the reasoning process knowledge of the big model to the neural parameters of the model through fine-tuning.

[0033] The division between professional and general knowledge is not an accurate enough division. The present invention is more intended to illustrate that general represents relatively easy abilities and knowledge that small models can learn and model, while professional represents relatively more difficult and sparse abilities and knowledge that small models cannot learn and summarize.

[0034] like Figure 3As shown in the figure, in order to illustrate the effectiveness of the method, the method of the present invention uses open source SLMs such as TinyLLaMA and LLaMA2-7B to conduct experiments on multiple public multi-task datasets, including in-domain (ID) and out-of-domain (OOD) datasets, covering mathematical reasoning, common sense reasoning, symbolic reasoning and other fields: BBH (BIG-Bench Hard), GSM8K, AGIEval, ARC, MetaMath, etc.

[0035] NesyCD significantly improves the complex reasoning capabilities of small language models (SLMs) on multiple benchmark test sets. Specifically, improvements were achieved on the test sets of mathematical reasoning (GSM8k), common sense reasoning (BBH), general reasoning (AGIEval), and factual knowledge (ARC). NesyCD was compared with other methods (including Zero-shot, Fine-tuning, CoT distillation, etc.). Experimental results show that NesyCD achieves better performance on multiple datasets.

[0036] like Figure 4 As shown in the figure, the impact of the confidence threshold (Δthreshold) on the model performance is given, and it is found that too high or too low thresholds may lead to performance degradation. In extreme cases, Δthreshold = 0 means that no retrieval is performed, while Δthreshold = 100 means that retrieval is performed for all cases. Both extremes lead to significant performance degradation, emphasizing the necessity of adaptive retrieval. For simple tasks, the model may produce hallucinations and incorrect answers, while for complex tasks that lack external guidance, the model cannot rely solely on internal parameters. Despite these challenges, the method of the present invention consistently outperforms other baselines, even in extreme cases.

[0037] The present invention also analyzes the efficiency of NesyCD in terms of training data and model size, and experiments are performed using Qwen2-0.5B, 1.5B and 7B. It can be found that as the training data decreases, the performance gap between NesyCD and other baselines becomes larger and larger, which shows that NesyCD has superior robustness and sample efficiency. This shows that NesyCD performs better with fewer samples, and its effectiveness can be further enhanced by increasing the training data, thereby achieving more effective distillation. Then considering the change in model size, the results highlight the significant practical advantages of NesyCD in resource-constrained environments, because it reduces the computational cost required for SLM while achieving performance levels that exceed large LLMs. This further illustrates that SLM (0.5B) cannot fully utilize the CoT reasoning generated by LLM, so it is necessary to implement the proposed NesyCD.

[0038] like Figure 5 As shown in Figure 2, LLaMA2-7B was used to perform ablation experiments to evaluate the contribution of each part of the NesyCD model. Through ablation studies, the effectiveness of each component in NesyCD, such as learning summary, supplementary knowledge, enhanced distillation, and multi-task learning, was verified. It can be observed that excluding any single component will degrade the performance, emphasizing the importance of each element. In addition, expertise is orthogonal and universal, which enhances Zero-shot-CoT and other CoT distillation methods to confirm the importance of refining symbolic knowledge.

[0039] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A neural symbolic co-distillation method for improving complex reasoning in small language models, characterized by: The method comprises the steps of: Step 1: General ability distillation: Use the teacher model to process the training data set to generate high-quality reasoning processes and answers; through supervised learning, train the initial student model to imitate the reasoning process and answers generated by the teacher model to obtain a primary student model to generate general reasoning ability and general knowledge; Step 2, demonstration case collection: Use the primary student model to process the training data set, determine which knowledge can be learned and which cannot be learned, collect the learned knowledge as the correct case set, and the unlearned knowledge as the error case set; Step 3: Symbolic knowledge distillation: Use the teacher model to analyze the error case collection, extract specific capabilities and rare knowledge in a specific field, and store the extracted specific capabilities and rare knowledge in the symbolic knowledge base; Step 4: Neural distillation enhanced with symbolic knowledge base: Using a multi-task learning strategy, the primary student model is trained to obtain an extended student model, which is used to generate reasoning processes and answers based on the question and the retrieved specific capabilities and rare knowledge.

2. The neural symbolic collaborative distillation method for improving complex reasoning in small language models according to claim 1, characterized in that: The multi-task learning strategy in step 4 includes at least one auxiliary task, and the auxiliary task is enhanced distillation, answer prediction or direct reasoning prompt.

3. The neural symbolic collaborative distillation method for improving complex reasoning of small language models according to claim 1, characterized in that: The symbolic knowledge base in step 4 is generated by analyzing error cases of the primary student model on a specific task.

4. The neural symbolic collaborative distillation method for improving complex reasoning in small language models according to claim 1, characterized in that: The step 4 includes that the extended student model dynamically retrieves specific capabilities and rare knowledge in the symbolic knowledge base according to confidence during the reasoning process.

5. The neural symbolic collaborative distillation method for improving complex reasoning of small language models according to claim 1, characterized in that: The step 4 includes that the performance of the extended student model in complex reasoning tasks is close to or exceeds that of a large language model.

Images