Model training and reasoning method and device, equipment, storage medium and program product

Through deep dynamic prefix technology and thinking graph construction method, the logical reasoning ability of large language models is improved, the problem of excessive resource occupation is solved, and the efficient improvement of logical reasoning ability and resource conservation is achieved. At the same time, the model reasoning process is provided. Explanatory.

CN119026659APending Publication Date: 2024-11-26CHINA MOBILE COMM LTD RES INST +1

Patent Information

Application Number
CN202411173562.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The implementation of large language models in terms of logical reasoning capabilities is difficult, which leads to excessive resource use of model training and operation. How to efficiently improve logical reasoning capabilities to save resources is a technical problem that needs to be solved.

Method used

By obtaining the training data set, including logical inference problems and their step-by-step solutions, using deep dynamic prefix parameters to build an initial large language model, and adjusting the model through training data, building a thinking graph to optimize the loss function, thereby improving the logical inference of the model ability.

Benefits of technology

It realizes the logical reasoning process of large language models while reducing the amount of parameters, saves computing resources, and assists humans in understanding and trace the model through thinking graph technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026659A_ABST
    Figure CN119026659A_ABST
Patent Text Reader

Abstract

The invention discloses a model training and reasoning method and device, equipment, a storage medium and a program product, relates to the technical field of communication, and aims to save resources. The method comprises the steps that a training data set is obtained, the training data set comprises a plurality of training texts, each training text sequentially comprises logical reasoning questions, step-by-step answers of the logical reasoning questions and text segmented words marking the ending of the texts, the step-by-step answers comprise prompt information, and the prompt information comprises the logical reasoning questions and the text segmented words marking the ending of the texts; the prompt information is used for extracting a final answer of the logical reasoning question; obtaining a deep dynamic prefix parameter; constructing an initial large language model by utilizing the deep dynamic prefix parameters; the initial large language model is trained through the training data set, a thinking map is constructed, and a loss function is obtained and used for adjusting the deep dynamic prefix parameters. The embodiment of the invention can save resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a model training, reasoning method, device, equipment, storage medium and program product. Background Art

[0002] Large Language Model (LLM) refers to a deep learning model with a large number of parameters that can handle a variety of natural language tasks (such as text classification, question-answering, and conversation). The most commonly used large language models in academia and industry are generally based on the Transformer architecture, and its core technology is the self-attention mechanism. In addition to improving model performance, the advantage of the self-attention mechanism is that it can assist people in explaining the operating mechanism of large language models by showing how the model distributes weights between text tokens (i.e., the individual segments that the model divides the text into).

[0003] Applying large language models to logical reasoning is one of the key research areas that scholars are currently focusing on. Logical reasoning ability is one of the manifestations of high-level intelligence of large language models. However, this ability is difficult to implement and often requires the support of large language models with huge parameters (>50B). This will result in a large amount of computing and industrial operation resources being occupied during model training or operation.

[0004] Therefore, how to efficiently improve the logical reasoning capabilities of large language models in order to save resources is a technical problem that needs to be solved. Summary of the invention

[0005] The embodiments of the present application provide a model training, reasoning method, device, equipment, storage medium and program product to save resources.

[0006] In a first aspect, an embodiment of the present application provides a model training method, comprising:

[0007] Acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation indicating the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logic reasoning question;

[0008] Get deep dynamic prefix parameters;

[0009] Building an initial large language model using the deep dynamic prefix parameters;

[0010] The initial large language model is trained using the training data set to construct a mind map and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

[0011] Optionally, the obtaining of deep dynamic prefix parameters includes:

[0012] Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector;

[0013] Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem;

[0014] The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

[0015] Optionally, the method further includes:

[0016] The gating parameters are obtained by:

[0017] The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

[0018] Optionally, obtaining a dynamic value vector according to the value vector and the gating parameter includes:

[0019] The product of the value vector and the gating parameter is used as the dynamic value vector.

[0020] Optionally, the constructing an initial large language model using the deep dynamic prefix parameter includes:

[0021] Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model;

[0022] The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

[0023] Optionally, the using the training data set to train the initial large language model, constructing a mind map, and obtaining a loss function includes:

[0024] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0025] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0026] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0027] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0028] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0029] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0030] S4. Calculate the loss function using the obtained probability;

[0031] The target symbols include: line break, colon, period, question mark or semicolon.

[0032] Optionally, determining a value of a self-attention strength measurement indicator between the second reasoning process and the executed reasoning process includes:

[0033] The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0034] Optionally, calculating the loss function using the obtained probability includes:

[0035] The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

[0036] In a second aspect, the embodiment of the present application further provides a model reasoning method, including:

[0037] Get the logical reasoning problems to be processed;

[0038] Using the logical reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the method according to any one of claims 1 to 8;

[0039] Based on the output text of the large language model, a final answer to the logical reasoning problem to be processed is obtained.

[0040] Optionally, the output text includes a plurality of text segmentations output in sequence, and the output text based on the large language model to obtain a final answer to the logical reasoning problem to be processed includes:

[0041] Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0042] The answer indicated by the prompt information is used as the final answer.

[0043] Optionally, the output text includes a plurality of text segmentations output in sequence, and the method further includes:

[0044] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0045] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0046] In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0047] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0048] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0049] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0050] The target symbols include: line break, colon, period, question mark or semicolon.

[0051] In a third aspect, the embodiment of the present application further provides a model training device, comprising:

[0052] A first acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logical reasoning question, a step-by-step answer to the logical reasoning question, and a text segmentation marking the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0053] The second acquisition module is used to obtain the deep dynamic prefix parameter;

[0054] A first processing module, configured to construct an initial large language model using the deep dynamic prefix parameters;

[0055] The second processing module is used to train the initial large language model using the training data set, construct a mind map, and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameters.

[0056] Optionally, the second acquisition module is further used to:

[0057] Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector;

[0058] Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem;

[0059] The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

[0060] Optionally, the device may further include:

[0061] The third acquisition module is used to acquire the gating parameter in the following manner:

[0062] The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

[0063] Optionally, the second acquisition module is further used to:

[0064] The product of the value vector and the gating parameter is used as the dynamic value vector.

[0065] Optionally, the first processing module is further used to:

[0066] Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model;

[0067] The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

[0068] Optionally, the second processing module is further used for:

[0069] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0070] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0071] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0072] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0073] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0074] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0075] S4. Calculate the loss function using the obtained probability;

[0076] The target symbols include: line break, colon, period, question mark or semicolon.

[0077] Optionally, the second processing module is further used for:

[0078] The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0079] Optionally, the second processing module is further used for:

[0080] The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

[0081] In a fourth aspect, the embodiment of the present application further provides a model reasoning device, including:

[0082] A first acquisition module is used to acquire the logical reasoning problem to be processed;

[0083] A first processing module is used to use the logic reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the aforementioned training method;

[0084] The second acquisition module is used to obtain the final answer to the logical reasoning problem to be processed based on the output text of the large language model.

[0085] Optionally, the second acquisition module is further used to:

[0086] Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0087] The answer indicated by the prompt information is used as the final answer.

[0088] Optionally, the device may further include:

[0089] The second processing module is used for:

[0090] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0091] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0092] In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0093] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0094] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0095] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0096] The target symbols include: line break, colon, period, question mark or semicolon.

[0097] In a fifth aspect, an embodiment of the present application further provides a model training device, comprising: a processor and a transceiver; wherein the processor is used to:

[0098] Acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation indicating the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logic reasoning question;

[0099] Get deep dynamic prefix parameters;

[0100] Building an initial large language model using the deep dynamic prefix parameters;

[0101] The initial large language model is trained using the training data set to construct a mind map and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

[0102] Optionally, the processor is further configured to:

[0103] Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector;

[0104] Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem;

[0105] The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

[0106] Optionally, the processor is further configured to:

[0107] The gating parameters are obtained by:

[0108] The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

[0109] Optionally, the processor is further configured to:

[0110] The product of the value vector and the gating parameter is used as the dynamic value vector.

[0111] Optionally, the processor is further configured to:

[0112] Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model;

[0113] The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

[0114] Optionally, the processor is further configured to:

[0115] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0116] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0117] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0118] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0119] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0120] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0121] S4. Calculate the loss function using the obtained probability;

[0122] The target symbols include: line break, colon, period, question mark or semicolon.

[0123] Optionally, the processor is further configured to:

[0124] The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0125] Optionally, the processor is further configured to:

[0126] The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

[0127] In a sixth aspect, an embodiment of the present application further provides a model reasoning device, comprising: a processor and a transceiver; wherein the processor is used to:

[0128] Wherein, the processor is used for:

[0129] Get the logical reasoning problems to be processed;

[0130] Using the logic reasoning problem to be processed as the input of the large language model to obtain the output text of the large language model, wherein the large language model is trained using the aforementioned training method;

[0131] Based on the output text of the large language model, a final answer to the logical reasoning problem to be processed is obtained.

[0132] Optionally, the processor is further configured to:

[0133] Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0134] The answer indicated by the prompt information is used as the final answer.

[0135] Optionally, the processor is further configured to:

[0136] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0137] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0138] In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0139] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0140] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0141] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0142] The target symbols include: line break, colon, period, question mark or semicolon.

[0143] In the seventh aspect, an embodiment of the present application also provides a communication device, including: a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the model training method or the model inference method as described above are implemented.

[0144] In an eighth aspect, an embodiment of the present application further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps in the model training method or the model inference method as described above are implemented.

[0145] In a ninth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the model training method or model reasoning method as described above.

[0146] In an embodiment of the present application, a deep dynamic prefix technology is used to allow the model to flexibly adjust the prefix for different input data, thereby reducing the amount of parameters that need to be fine-tuned during the model fine-tuning process and saving computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0147] Figure 1 This is one of the flow charts of the model training method provided in the embodiment of the present application;

[0148] Figure 2 It is a schematic diagram of adding prefixes in a large model of the prior art;

[0149] Figure 3 It is a schematic diagram of adding prefixes in a large model of an embodiment of the present application;

[0150] Figure 4 is a schematic diagram of the gating mechanism in the large model of the embodiment of the present application;

[0151] Figure 5 This is the second flowchart of the model reasoning method provided in the embodiment of the present application;

[0152] Figure 6 is a flowchart of a processing method provided in an embodiment of the present application;

[0153] Figure 7 is a schematic diagram of a model training method provided in an embodiment of the present application;

[0154] Figure 8 is a schematic diagram of the first node of the mind map of an embodiment of the present application;

[0155] Fig. 9 It is a schematic diagram of a mind map of an embodiment of the present application;

[0156] Fig.10 It is a schematic diagram of the next node of the mind map of the embodiment of the present application;

[0157] Fig.11 Schematic diagram of the calculation process of the self-attention intensity measurement index value of the embodiment of the present application;

[0158] Fig.12 This is an example of the application of the self-attention mechanism in a large language model of the decoder type;

[0159] Fig.13 It is a schematic diagram of a mind map of an embodiment of the present application;

[0160] Fig.14 It is a schematic diagram of the processing process of an embodiment of the present application;

[0161] Fig.15 is a schematic diagram of the model reasoning process of an embodiment of the present application;

[0162] Fig.16 It is one of the structural diagrams of the model training device provided in the embodiment of the present application;

[0163] Fig.17 It is one of the structural diagrams of the model reasoning device provided in the embodiment of the present application;

[0164] Fig.18 This is the second structural diagram of the model training device provided in the embodiment of the present application;

[0165] Fig.19 This is the second structural diagram of the model reasoning device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0166] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0167] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0168] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0169] See also Figure 1 , Figure 1 is a flow chart of the model training method provided in the embodiment of the present application, such as Figure 1 As shown, the following steps are included:

[0170] Step 101: Obtain a training data set, wherein the training data set includes multiple training texts, each training text includes a logical reasoning question, a step-by-step answer to the logical reasoning question, and a text segmentation marking the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract the final answer to the logical reasoning question.

[0171] Here, the training data set refers to the data used for model training, including the text input into the large language model during training. Each training text is, in sequence, a certain logic reasoning problem extracted from the logic reasoning problem data set, a step-by-step answer to the logic reasoning problem, and a text segmentation marking the end of the text. The text segmentation marking the end of the text may vary depending on the large language model used, including but not limited to <|endoftext|>. The data used for model training here does not need to add additional prompt words (such as instruction text or additional input and output examples) because deep dynamic prefixes are used in subsequent steps to replace artificially designed additional prompt words, avoiding the limitations of human understanding. The prompt information can be, for example, the three words "the answer is".

[0172] The logical reasoning problems and their step-by-step solutions can come from existing public data sets, such as mathematical word problem data sets. The step-by-step solutions can be obtained by manual annotation. The step-by-step solutions are used as part of the model training data here in the hope of guiding the large language model to output the solutions to the logical reasoning problems in a similar format after fine-tuning using these training data and the subsequent deep dynamic prefix technology, when the large language model is applied based on the deep dynamic prefix mechanism, that is, the step-by-step reasoning process is included in the solution.

[0173] In addition, in order to facilitate the automatic extraction of answers to logical reasoning questions output by the large language model in subsequent steps, a paragraph of text can be added at the end of the step-by-step answer example, and the paragraph of text is composed of connecting text (such as the three words "the answer is") and the final answer in the step-by-step answer. If the paragraph of text already exists at the end of the step-by-step answer, there is no need to add it. Inputting the step-by-step answer example of the logical reasoning problem with this paragraph of text into the large language model can guide the large language model to output the answer in a similar format when answering the logical reasoning problem. Therefore, the automatic extraction in the embodiment of the present application refers to using the text after the connecting text (such as "the answer is") in the output text of the large language model as the final answer of the large language model to the logical reasoning problem; if such text does not exist, it is considered that the large language model cannot answer the logical reasoning problem.

[0174] Step 102: Obtain deep dynamic prefix parameters.

[0175] The deep dynamic prefix parameter refers to the prefix parameter applied to the deep Transformer layer of the large language model. The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value. The preset value can be set as needed. For example, the deep Transformer layer is part or all of the Transformer layers in the first part of the Transformer layers close to the output layer of the basic large language model, and the number of the first part of the Transformer layers accounts for half of the total number of all Transformer layers of the basic large language model.

[0176] In this step, the first neural network is used to obtain initial prefix parameters, wherein the initial prefix parameters include a key vector and a value vector. Afterwards, a dynamic value vector is obtained based on the value vector and the gating parameter, wherein the gating parameter is related to the logical reasoning problem. Finally, the key vector and the dynamic value vector are used as the deep dynamic prefix parameters. The gating parameter is obtained by taking the logical reasoning problem as the input of the second neural network and running the second neural network; the product of the value vector and the gating parameter is used as the dynamic value vector.

[0177] The first neural network may be a feed-forward neural network, and the second neural network may also be a feed-forward neural network. The meaning of the dynamic value vector is that the value vector can be adjusted according to different input data.

[0178] Existing prefix tuning techniques add a small amount of prefixes (such as Figure 2 The prefix-related parameters include the key vector (Key) and the value vector (Value) corresponding to the prefix, which are calculated by an additional neural network. The neural network can be a feed-forward neural network. The input data is a fixed value (for example, a vector [0, 1, 2, 3]). The number of parameters is much smaller than that of the large language model, and can be trained during the prefix tuning process.

[0179] In the embodiments of the present application, Figure 3As shown in the figure, during the model training process, the deep dynamic prefix technology is used to insert prefixes only into the deep Transformer layer of the large language model, freeze the original parameters of the large language model, and dynamically adjust the inserted prefixes based on the input text using the gating mechanism.

[0180] The deep dynamic prefix technology of the embodiment of the present application only adds prefixes to the deep Transformer layer of the large language model, and the value vector corresponding to the prefix can be dynamically adjusted according to the input text based on the gating mechanism. The deep Transformer layer may refer to all or part of the Transformer layer close to the output layer of the model (i.e., the last half layer). Compared to traditional prefix tuning, the deep dynamic prefix technology only adds prefixes to the deep Transformer layer of the large language model, so that some unreasonable self-attention intensity distribution trends in the deep layer of the model can be corrected. Therefore, the embodiment of the present application can reduce the amount of parameters that need to be fine-tuned, while achieving an effect similar to prefix tuning.

[0181] Specifically, in each Transformer layer of the large language model, each input text segmentation will be mapped to a query vector (Query), a key vector (Key) and a value vector (Value). The output vector corresponding to each input text segmentation of the Transformer layer will be calculated as the weighted sum of the value vectors in the layer. Among them, the weight of each value vector is called the self-attention strength. For example, in the decoder-type large language model, the self-attention strength A between the text segmentation x corresponding to a certain output vector of each Transformer layer and any input text segmentation y in the Transformer layer (that is, the weight of the value vector corresponding to y when calculating the output vector corresponding to x) can be calculated using the following formula:

[0182]

[0183] Where softmax(.) represents the Softmax function, Q represents the query vector corresponding to x, K represents the key vector corresponding to y, and d represents the length of the query vector or the key vector (the two are equal in length).

[0184] In the deep dynamic prefix technology, it is necessary to obtain the key vector and value vector corresponding to the prefix in the deep Transformer layer, which are used to calculate the output vector corresponding to the text segmentation of the model input in the corresponding layer; and since there is no need to calculate the output vector corresponding to the prefix, there is no need for the query vector corresponding to the prefix. Because the position of the prefix is ​​before all the text segmentations input by each layer of Transformer, its corresponding key vector can directly use the self-attention mechanism to correct the self-attention intensity distribution of subsequent text segmentations (that is, through the calculation formula of A above). In addition, the value vector corresponding to the prefix can also affect the output corresponding to the subsequent text segmentation after multiplying with A, thereby indirectly correcting the distribution of self-attention intensity in the subsequent Transformer layer. Therefore, in this way, by correcting the self-attention distribution, it can assist the subsequent steps to more effectively construct a mind map based on self-attention intensity.

[0185] like Figure 4 As shown, the embodiment of the present application adds a gating mechanism to the deep dynamic prefix technology, allowing the value vector corresponding to the prefix to be dynamically adjusted according to the input data. Specifically, the dynamic value vector (Vdynamic) corresponding to the dynamic prefix can be calculated based on the following formula:

[0186] Vdynamic=gV

[0187] Wherein, V is the value vector corresponding to the prefix in the traditional prefix fine-tuning technology, and g is the gating parameter. The gating parameter can be a scalar, and the gating parameter used by each Transformer layer that adds a prefix can be different. The gating parameter can be calculated by a small trainable neural network, the input of which is a logical reasoning problem input by a large language model (excluding its step-by-step solution), and the output is the value of the gating parameter. The embodiment of the present application does not specifically limit its network structure.

[0188] Step 103: construct an initial large language model using the deep dynamic prefix parameters.

[0189] Here, the deep dynamic prefix parameter is applied to the deep Transformer layer of the basic large language model to obtain the initial large language model; wherein the deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value. The basic large language model can be any existing large language model. Compared with traditional prefix tuning, the deep dynamic prefix technology only adds prefixes to the deep Transformer layer of the large language model, which can achieve similar correction effects and model training results. Therefore, the embodiment of the present application can reduce the amount of parameters that need to be fine-tuned, while achieving effects similar to prefix tuning.

[0190] Step 104: Use the training data set to train the initial large language model, construct a mind map, and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameters.

[0191] The process may include the following:

[0192] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0193] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0194] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0195] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0196] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0197] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0198] S4. Calculate the loss function using the obtained probability;

[0199] The target symbols include: line break, colon, period, question mark or semicolon.

[0200] Here, the maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0201] Many large language models use a multi-head self-attention mechanism, allowing the model to use multiple self-attention heads in a single-layer Transformer. Each self-attention head can use a set of parameter mappings to obtain a set of query vectors, key vectors, and value vectors, and calculate a set of self-attention strengths and an output vector for each text segmentation. The parameters used by different self-attention heads can be different. Finally, the multiple initial output vectors obtained by multiple self-attention heads are used together to generate the final output vector corresponding to the segmentation. The method of generating the final output vector here includes but is not limited to concatenating multiple initial output vectors and multiplying them with a parameter matrix.

[0202] Therefore, the specific definition of the self-attention strength measure between two reasoning steps can include:

[0203] Assume that there are two reasoning steps M and N, where the text position of N is after M. The self-attention strength measure α(M,N) between two reasoning steps is defined as the maximum value of the self-attention strength generated between the query vector of the text segmentation in step N and the key vector of the text segmentation in step M:

[0204]

[0205] Among them, m and n are the text segmentation in sub-steps M and N respectively. The self-attention strength is calculated for the query vector of text segmentation n and the key vector of text segmentation m at the hth self-attention head of the lth layer Transformer. The maximum value is used in this formula instead of the average value because some inference steps are long and there may be some irrelevant information that pulls down the average value.

[0206] The logarithms of the probabilities are summed, and the sum is multiplied by negative one to obtain the loss function. Specifically, when using the deep dynamic prefix technology to train a large language model, the following loss function L can be used:

[0207]

[0208] Here, x is used to refer to the text segmentation contained in the step-by-step solution text in the training data, as well as the text segmentation marking the end of the text; x t represents the tth text segmentation in x in the order of text; h <t Including dynamic prefixes, and the order of text in each training data in x t The previous text segmentation; P(x t |h <t ) indicates that the large language model is given h<t Under the premise of generating x t During the model training process, the value of this loss function can be reduced by adjusting the model parameters, thereby increasing the probability of the model generating the corresponding text segmentation in the step-by-step solution under given conditions.

[0209] The threshold value ranges from 0 to 1 and can be determined by experiments. The experiment requires a data set containing multiple logical reasoning questions and their correct answers. You can first use a smaller value γ (for example, 0 to 0.5) as the threshold, and calculate the accuracy of answering the logical reasoning questions in the data set using the large language model when using this threshold. Then, according to the accuracy, γ can be appropriately increased or decreased until a higher accuracy is reached, and the γ at this time is used as the final threshold.

[0210] In an embodiment of the present application, a deep dynamic prefix technology is used to allow the model to flexibly adjust the prefix for different input data, thereby reducing the amount of parameters that need to be fine-tuned during the model fine-tuning process and saving computing resources.

[0211] See also Figure 5 , Figure 5 is a flow chart of the model reasoning method provided in the embodiment of the present application, such as Figure 5 As shown, the following steps are included:

[0212] Step 501: Obtain a logical reasoning problem to be processed;

[0213] Step 502: using the logic reasoning problem to be processed as input of a large language model to obtain output text of the large language model, wherein the large language model is trained using the aforementioned model training method;

[0214] Step 503: Based on the output text of the large language model, obtain the final answer to the logical reasoning problem to be processed.

[0215] In this step, the target text segmentation including the prompt information is determined, the target text segmentation is the text segmentation including the prompt information, the prompt information is used to extract the final answer to the logical reasoning question. After that, the answer indicated by the prompt information is used as the final answer.

[0216] Optionally, the output text includes a plurality of text segmentations output in sequence, and the method further includes:

[0217] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model without executing subsequent steps; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0218] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0219] In the case where any of the first text segmentations is the text segmentation marking the end of the text, after the execution of S1 is completed, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model, without executing subsequent steps; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0220] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0221] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0222] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0223] The target symbols include: line break, colon, period, question mark or semicolon.

[0224] In the embodiment of the present application, the deep dynamic prefix technology allows the model to flexibly adjust the prefix for different input data, thereby reducing the number of parameters that need to be fine-tuned during the model fine-tuning process and saving computing resources. At the same time, the mind chain technology can be combined with the graph structure to develop a mind map technology with strong interpretability to characterize the relationship between reasoning steps and assist humans in understanding and tracing the logical reasoning process of large language models.

[0225] The implementation process of the embodiments of the present application is described in detail below in conjunction with specific embodiments.

[0226] In order to save computing resources, fully characterize the relationship between reasoning steps, and facilitate the logical reasoning capabilities of large language models that are easy for humans to understand and trace, the implementation of the embodiments of this application is mainly based on the following goals:

[0227] 1) Efficiently improve the logical reasoning ability of large language models and reduce the number of parameters required for large language models.

[0228] 2) Combine thought chain technology with graph structure to develop a thought graph technology with strong interpretability to characterize the relationship between reasoning steps and assist humans in understanding and tracing the logical reasoning process of large language models.

[0229] 3) Further improve the thought chain technology to avoid the limitations of its reliance on artificially designed prompt words, while enhancing the logical reasoning ability of the large language model that the thought chain technology can stimulate.

[0230] 4) Further improvements are made to the prefix tuning technology to allow the prefix to be dynamically adjusted in real time based on the input data, and on this basis, the number of related parameters that need to be trained is reduced as much as possible.

[0231] To this end, an embodiment of the present application proposes a method for improving the logical reasoning ability of a large language model based on deep dynamic prefixes and mind maps. By only adding trainable prefix-related parameters to the deep Transformer layer of the large language model, and designing a gating mechanism to allow flexible prefix adjustment for different input data, the logical reasoning ability of the large language model after fine-tuning is improved and the required computing resources are reduced. In addition, while the large language model is outputting text (rather than after all text output is completed), a mind map is constructed by correcting the self-attention intensity, thereby reducing the mutual interference between steps with low correlation in the reasoning process of the large language model, improving the correctness of the logical reasoning of the large language model, and assisting humans in understanding and tracing the correlation between the reasoning steps of the large language model. Figure 6 As shown, the process may include:

[0232] Step 601: Obtain a training data set.

[0233] Specifically, the training data set includes a plurality of training texts, each of which includes a logic reasoning problem, a step-by-step solution to the logic reasoning problem, and a text segmentation marking the end of the training text.

[0234] Step 602: During the model training process, the deep dynamic prefix technology is used to insert prefixes only into the deep Transformer layer of the large language model, freeze the original parameters of the large language model, and dynamically adjust the inserted deep dynamic prefixes based on the input text using a gating mechanism.

[0235] Step 603: During the model training process of step 602, a mind map is constructed by calculating a self-attention strength measurement index, and the self-attention strength is corrected accordingly.

[0236] like Figure 7 As shown, the process may include:

[0237] Step 6031: During the model training process, before calculating the loss function value corresponding to each training data, the text segmentation x contained in the step-by-step solution in the data is calculated. t , check from front to back in text order, and calculate the corresponding term logP(x t |h <t ), when the checked text segmentation contains a line break / colon / period / question mark / semicolon or is a text segmentation marking the end of the text, the text segmentation is concatenated with the text segmentation before the text segmentation in the step-by-step answer in the original order and regarded as the first reasoning step; this reasoning step corresponds to the first node in the mind map.

[0238] like Figure 8As shown, an example of generating node 1 by step-by-step solution is shown. The checking can be implemented in an automated manner by running a code script. The formats of the colon / period / question mark / semicolon can include full-width format and half-width format. Fig. 9 As shown, a schematic diagram of a mind map is shown. The mind map is a conceptual graph structure that describes whether the self-attention strength measurement index between the reasoning steps of the large language model is zero. Each node represents an reasoning step. Two nodes connected by a line indicate that the self-attention measurement index between the corresponding two reasoning steps is non-zero, and two nodes not connected by a line indicate that the self-attention measurement index between the corresponding two reasoning steps is zero. The direction of the connection is from the graph node corresponding to the reasoning step that is earlier in the text order to the graph node corresponding to the reasoning step that is later in the text order.

[0239] Step 6032: Continue to check the text segmentation x contained in the step-by-step solution from front to back in the text order. t , and calculate the corresponding term logP(x t |h <t ), when a text segmentation contains a line break / colon / period / question mark / semicolon or is a text segmentation that marks the end of the text, the calculation of the corresponding item in the loss function for the subsequent text segmentation is suspended, and the text segmentation and the text segmentation that precedes the text segmentation in the step-by-step solution and does not belong to the existing reasoning steps are spliced ​​in the original order and regarded as a new reasoning step. This reasoning step corresponds to a newly generated node in the mind map, and the node is connected to all previously generated nodes in the mind map by directed lines, and the direction of the line is from the generated node to the node.

[0240] like Fig.10 As shown, an example of generating the next node by a step-by-step solution is shown. The check can be implemented in the same way as the check in step 6031. The formats of the colon / period / question mark / semicolon can include full-width format and half-width format. It should be noted that the connection between the nodes in step 6032 only represents that when this step is executed, the self-attention strength index between the two reasoning steps corresponding to the corresponding two nodes is non-zero. The connection between the nodes may be deleted in step 6033.

[0241] Step 6033: Fig.11 As shown, the value of the self-attention strength measurement index between the newly generated reasoning step and the already generated reasoning step in step 6032 is calculated. When the value of the index does not exceed a certain threshold, the self-attention strength generated in the large language model between the text segmentations that do not belong to the same reasoning step in the corresponding two reasoning steps is corrected to zero; at the same time, the connection between the corresponding nodes in the mind map will be deleted.

[0242] The self-attention strength measurement index between the reasoning steps can be calculated based on the self-attention strength generated between text segmentations in two reasoning steps. Fig.12 An example of the application of the self-attention mechanism of a decoder-like large language model is shown in Figure 1: each square in the figure represents a text segmentation; the upper row is the text segmentation that produces the query vector, and the lower row is the text segmentation that produces the key vector; there is a non-zero self-attention strength only between two squares connected by a line, and the direction of the arrow is from the text segmentation that produces the query vector to the text segmentation that produces the key vector.

[0243] Based on the above definition, in step 7302, when the value of the indicator does not exceed a certain threshold, the self-attention strength generated in the large language model between the text segmentations that do not belong to the same reasoning step in the corresponding two reasoning steps is corrected to zero. Therefore, the value of the indicator will also be corrected to zero at the same time, and the connection between the corresponding nodes in the mind map will be deleted, and finally form, for example Fig.13 The mind map shown in .

[0244] Step 6034: Repeat steps 6032 to 6033 until the text segmentation that marks the end of the text is detected, and use all logP(x t |h <t ) Calculate the loss function L corresponding to the corresponding training data.

[0245] like Fig.14 As shown, it is an exemplary schematic diagram of a mind map finally formed through the processing of steps 6032-6034.

[0246] In the above process, if, when executing step 6031 or step 6032, a text segmentation indicating the end of the text is detected before a text segmentation containing a line break, colon, period, question mark, or semicolon is detected, then there is no need to execute subsequent steps, and all logP(x t |h <t ) Calculate the loss function L corresponding to the corresponding training data. The text segmentation marking the end of the text may vary depending on the large language model used, including but not limited to <|endoftext|>.

[0247] Step 604: When the model is applied, the relevant parameters obtained through training in step 602 are used to dynamically calculate the exclusive deep prefix for each logical reasoning problem to be answered by the input model, and guide the large language model to generate a step-by-step answer; and in a manner corresponding to step 603, when the large language model performs text segmentation, the self-attention intensity used is corrected in real time and a mind map is constructed; finally, the final answer to the logical reasoning problem is automatically extracted from the model output.

[0248] This step is to use the relevant parameters obtained through training in step 602 when the model is applied, dynamically calculate the exclusive deep prefix for each logical reasoning problem to be answered by the input model, and guide the large language model to generate step-by-step answers; and in a manner corresponding to the aforementioned training process, when the large language model performs text segmentation, the self-attention intensity used is corrected in real time and a mind map is constructed; finally, the final answer to the logical reasoning problem is automatically extracted from the model output.

[0249] like Fig.15 As shown, this step can be divided into step 6041 to step 6045. The detailed description of step 6042 to step 6045 can refer to the detailed description of step 601 to step 604.

[0250] Step 6041: Input the logical reasoning problem to be answered into the large language model.

[0251] Step 6042: Perform real-time check when the large language model outputs each text segmentation. When the output text segmentation contains a line break / colon / period / question mark / semicolon, pause the output of the new text segmentation, and concatenate the text segmentation with the output text segmentation in the output order, which is regarded as the first reasoning step; this reasoning step corresponds to the first node in the mind map.

[0252] Step 6043: allow the large language model to continue to output text segmentation, and continue to check the text segmentation output by the large language model in real time. When the output text segmentation contains a line break / colon / period / question mark / semicolon, pause the output of new text segmentation, and concatenate the text segmentation with the output text segmentation that does not belong to the existing reasoning steps in the output order, and regard it as a new reasoning step; this reasoning step corresponds to a newly generated node in the mind map, and the node is connected to all previously generated nodes in the mind map by directed lines, and the direction of the line is from the generated node to the node.

[0253] Step 6044: Calculate the value of the self-attention strength measurement index between the newly generated reasoning step in step 6043 and the already generated reasoning step (the self-attention strength measurement index used here must be consistent with step 603). When the value of the index does not exceed the threshold used in step 603, the self-attention strength generated in the large language model between the text segmentations that do not belong to the same reasoning step in the corresponding two reasoning steps is corrected to zero; at the same time, the connection between the corresponding nodes in the mind map will be deleted.

[0254] Step 6045: Repeat steps 6043 to 6044 until the large language model generates a text segmentation that marks the end of the text, and then automatically extract the final answer to the large language model logical reasoning question. For example, the answer after the three words "the answer is" can be extracted as the final answer.

[0255] Through the above description, the embodiments of the present application have the following advantages:

[0256] Compared with traditional prefix tuning technology, the embodiments of the present application develop deep dynamic prefix technology to allow the model to flexibly adjust the prefix for different input data, and on this basis reduce the number of Transformer layers that need to insert prefixes, thereby minimizing the amount of parameters that need to be fine-tuned during the model fine-tuning process, saving computing resources while achieving similar model fine-tuning effects.

[0257] Compared with the traditional thought chain technology, the method proposed in the embodiment of the present application does not require researchers to manually design prompt words in the form of text to guide the large language model to answer logical reasoning problems step by step, avoiding the limitations of human understanding ability on the design of related prompt words. In addition, the method proposed in the embodiment of the present application fine-tunes the parameters of the large language model, further enhancing the potential of the thought chain technology to enhance the logical reasoning ability of the large language model.

[0258] The mind map technology based on the self-attention mechanism used in the embodiment of the present application can correct the self-attention strength between reasoning steps with weak correlation to zero, reducing the interference of the original non-zero self-attention strength on the subsequent output of the model.

[0259] In addition, compared with some recent studies that combine thought chains with graph structures, the embodiments of the present application have the following advantages:

[0260] First, the technology proposed in the embodiment of the present application can better assist the large language model to effectively represent the logic between the solution steps through the graph structure.

[0261] Second, the technology proposed in the embodiment of the present application provides a specific implementation method for each step of constructing a mind map based on self-attention when a large language model performs logical reasoning, which makes up for the defect that the existing theoretical framework and knowledge graph-related technologies (such as MindMap, ToG) cannot fully realize the answers to various logical reasoning problems.

[0262] Finally, the embodiment of the present application realizes the construction of a mind map while the large language model outputs a single reasoning path, and corrects the self-attention strength values ​​between the reasoning steps with low correlation to zero, avoiding the inefficiency of some previous graph structure-based methods (such as RGV) that require outputting multiple reasoning paths in full and then constructing the relevant graph structure, as well as the problem that some reasoning steps with weak correlation will still interfere with each other through the self-attention mechanism.

[0263] See also Fig.16 , Fig.16 is a structural diagram of the model training device provided in the embodiment of the present application. Fig.16 As shown, the information processing device includes:

[0264] The first acquisition module 1601 is used to acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation marking the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract the final answer to the logic reasoning question;

[0265] The second acquisition module 1602 is used to acquire a deep dynamic prefix parameter;

[0266] A first processing module 1603 is used to construct an initial large language model using the deep dynamic prefix parameters;

[0267] The second processing module 1604 is used to train the initial large language model using the training data set, construct a mind map, and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

[0268] Optionally, the second acquisition module is further used to:

[0269] Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector;

[0270] Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem;

[0271] The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

[0272] Optionally, the device may further include:

[0273] The third acquisition module is used to acquire the gating parameter in the following manner:

[0274] The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

[0275] Optionally, the second acquisition module is further used to:

[0276] The product of the value vector and the gating parameter is used as the dynamic value vector.

[0277] Optionally, the first processing module is further used to:

[0278] Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model;

[0279] The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

[0280] Optionally, the second processing module is further used for:

[0281] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0282] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0283] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0284] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0285] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0286] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0287] S4. Calculate the loss function using the obtained probability;

[0288] The target symbols include: line break, colon, period, question mark or semicolon.

[0289] Optionally, the second processing module is further used for:

[0290] The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0291] Optionally, the second processing module is further used for:

[0292] The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

[0293] The device provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0294] See also Fig.17 , Fig.17 is a structural diagram of the model reasoning device provided in the embodiment of the present application. Fig.17 As shown, the model reasoning device includes:

[0295] The first acquisition module 1701 is used to acquire the logical reasoning problem to be processed;

[0296] A first processing module 1702 is used to use the logic reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the aforementioned model training method;

[0297] The second acquisition module 1703 is used to obtain the final answer to the logical reasoning problem to be processed based on the output text of the large language model.

[0298] Optionally, the second acquisition module is further used to:

[0299] Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0300] The answer indicated by the prompt information is used as the final answer.

[0301] Optionally, the device may further include:

[0302] The second processing module is used for:

[0303] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0304] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0305] In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0306] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0307] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0308] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0309] The target symbols include: line break, colon, period, question mark or semicolon.

[0310] The device provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0311] See also Fig.18 , Fig.18 is a structural diagram of the model training device provided in the embodiment of the present application. Fig.18 As shown, the information processing device includes: a processor 1801 and a transceiver 1802; wherein the processor 1801 is used to:

[0312] Acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation indicating the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logic reasoning question;

[0313] Get deep dynamic prefix parameters;

[0314] Building an initial large language model using the deep dynamic prefix parameters;

[0315] The initial large language model is trained using the training data set to construct a mind map and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

[0316] Optionally, the processor 1801 is further configured to:

[0317] Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector;

[0318] Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem;

[0319] The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

[0320] Optionally, the processor 1801 is further configured to:

[0321] The gating parameters are obtained by:

[0322] The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

[0323] Optionally, the processor 1801 is further configured to:

[0324] The product of the value vector and the gating parameter is used as the dynamic value vector.

[0325] Optionally, the processor is further configured to:

[0326] Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model;

[0327] The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

[0328] Optionally, the processor 1801 is further configured to:

[0329] S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing a target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as a text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation:

[0330] Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer;

[0331] In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer;

[0332] S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0333] Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution;

[0334] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0335] S4. Calculate the loss function using the obtained probability;

[0336] The target symbols include: line break, colon, period, question mark or semicolon.

[0337] Optionally, the processor 1801 is further configured to:

[0338] The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

[0339] Optionally, the processor 1801 is further configured to:

[0340] The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

[0341] The device provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0342] See also Fig.19 , Fig.19 is a structural diagram of the model reasoning device provided in the embodiment of the present application. Fig.19 As shown, the information processing device includes: a processor 1901 and a transceiver 1902; wherein the processor 1901 is used to:

[0343] Get the logical reasoning problems to be processed;

[0344] Using the logic reasoning problem to be processed as the input of the large language model to obtain the output text of the large language model, wherein the large language model is trained using the aforementioned training method;

[0345] Based on the output text of the large language model, a final answer to the logical reasoning problem to be processed is obtained.

[0346] Optionally, the processor 1901 is further configured to:

[0347] Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question;

[0348] The answer indicated by the prompt information is used as the final answer.

[0349] Optionally, the processor 1901 is further configured to:

[0350] S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation:

[0351] When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, concatenating the text segmentation with the text segmentation output by the large language model in an output order;

[0352] In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model;

[0353] S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes:

[0354] splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order;

[0355] S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map;

[0356] The target symbols include: line break, colon, period, question mark or semicolon.

[0357] The device provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0358] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0359] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0360] An embodiment of the present application provides a communication device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the model training method or model inference method as described above.

[0361] The embodiment of the present application also provides a readable storage medium, on which a program is stored, and when the program is executed by the processor, each process of the above-mentioned model training method or model reasoning method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0362] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned model training method or model reasoning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0363] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0364] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0365] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation indicating the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logic reasoning question; Get deep dynamic prefix parameters; Building an initial large language model using the deep dynamic prefix parameters; The initial large language model is trained using the training data set to construct a mind map and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

2. The method according to claim 1, characterized in that The obtaining of deep dynamic prefix parameters includes: Obtaining initial prefix parameters using a first neural network, wherein the initial prefix parameters include a key vector and a value vector; Obtaining a dynamic value vector according to the value vector and a gating parameter, wherein the gating parameter is related to the logical reasoning problem; The key vector and the dynamic value vector are used as the deep dynamic prefix parameters.

3. The method according to claim 2, characterized in that The method further comprises: The gating parameters are obtained by: The logical reasoning problem is used as an input of a second neural network, and the second neural network is run to obtain the gating parameters.

4. The method according to claim 2, characterized in that: The step of obtaining a dynamic value vector according to the value vector and the gating parameter includes: The product of the value vector and the gating parameter is used as the dynamic value vector.

5. The method according to claim 1, characterized in that The step of constructing an initial large language model using the deep dynamic prefix parameters includes: Applying the deep dynamic prefix parameter to the deep Transformer layer of the basic large language model to obtain the initial large language model; The deep Transformer layer is part or all of the Transformer layers in the target Transformer layer among all the Transformer layers of the basic large language model, and the number of Transformer layers between the target Transformer layer and the output layer of the basic large language model is less than or equal to a preset value.

6. The method according to claim 1, characterized in that The using the training data set to train the initial large language model, constructing a mind map, and obtaining a loss function includes: S1. Perform a first reasoning process based on the first text segmentation of the step-by-step answer, and determine a first node in the mind map based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing a target symbol in the step-by-step answer or any text segmentation before the first text segmentation containing the target symbol in the step-by-step answer; if there is no text segmentation containing the target symbol in the step-by-step answer, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation in the step-by-step answer; the first reasoning process includes performing the following processes on all text segmentations that meet the definition of the first text segmentation: Based on the deep dynamic prefix parameter and the text segmentation located before the first text segmentation in the training text, a probability of generating the first text segmentation is obtained; when the first text segmentation includes the target symbol or the first text segmentation is a text segmentation marking the end of the text, the first text segmentation and the text segmentation located before the first text segmentation in the step-by-step answer are spliced ​​according to the order of the text segmentations in the step-by-step answer; In the case where any of the first text segmentations is a text segmentation marking the end of the text, step S4 is directly executed after step S1 is executed; otherwise, after step S1 is executed, steps S2 and S3 are repeatedly executed until the second text segmentation is a text segmentation marking the end of the text, and step S4 is executed, wherein the second text segmentation is a text segmentation after the first text segmentation in the step-by-step answer; S2, performing a second reasoning process based on the second text segmentation, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes: Based on the deep dynamic prefix parameter and the text segmentation located before the second text segmentation in the training text, a probability of generating the second text segmentation is obtained; and, when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, the second text segmentation and the text segmentations located before the second text segmentation in the step-by-step solution and not belonging to the executed reasoning process are spliced ​​according to the order of each text segmentation in the step-by-step solution; S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map; S4. Calculate the loss function using the obtained probability; The target symbols include: line break, colon, period, question mark or semicolon.

7. The method according to claim 6, characterized in that The determining of a value of a self-attention strength measure index between the second reasoning process and the executed reasoning process includes: The maximum value of the self-attention strength generated between the query vector of the text segmentation in the second reasoning process and the key vector of the text segmentation in the executed reasoning process is used as the self-attention strength measurement indicator value.

8. The method according to claim 6, characterized in that The calculating the loss function using the obtained probability includes: The loss function is obtained by summing the obtained logarithms of the probabilities and multiplying the sum by negative one.

9. A model reasoning method, characterized in that: include: Get the logical reasoning problems to be processed; Using the logical reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the method according to any one of claims 1 to 8; Based on the output text of the large language model, a final answer to the logical reasoning problem to be processed is obtained.

10. The method according to claim 9, characterized in that The output text includes a plurality of text segmentations output in sequence, and the output text based on the large language model obtains the final answer to the logical reasoning problem to be processed, including: Determine a target text segmentation including prompt information, wherein the target text segmentation is a text segmentation including prompt information, and the prompt information is used to extract a final answer to the logical reasoning question; The answer indicated by the prompt information is used as the final answer.

11. The method according to claim 9, characterized in that The output text includes a plurality of text segmentations output in sequence, and the method further includes: S1. While the large language model outputs the first text segmentation, a first reasoning process is performed, and a first node in the mind map is determined based on the first reasoning process, wherein the first text segmentation is defined as the first text segmentation containing the target symbol output by the large language model before the text segmentation marking the end of the text or any text segmentation before the first text segmentation containing the target symbol; if the large language model does not output the text segmentation containing the target symbol before outputting the text segmentation marking the end of the text, then the first text segmentation is defined as the text segmentation marking the end of the text or any text segmentation before the text segmentation marking the end of the text in the step-by-step answer; if the large language model does not output the text segmentation marking the end of the text or the text segmentation containing the target symbol until the large language model stops outputting the text segmentation, then the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model; the first reasoning process includes performing the following processing on all text segmentations that meet the definition of the first text segmentation: When the first text segmentation includes the target symbol or the text segmentation marking the end of the text in the step-by-step solution, the text segmentation is concatenated with the text segmentations output by the large language model in an output order; In the case where any of the first text segmentations is the text segmentation marking the end of the text, the final answer to the logical reasoning problem to be processed is automatically extracted directly based on the output text of the large language model after the execution of S1 is completed; otherwise, after the execution of S1 is completed, the following steps S2 and S3 are repeatedly executed until the second text segmentation is the text segmentation marking the end of the text, and the final mind map is obtained based on the result of S3, wherein the second text segmentation is a text segmentation whose output order is after the first text segmentation in the text segmentations output by the large language model; S2, allowing the large language model to continue to output the second text segmentation, while checking each second text segmentation, and when the second text segmentation includes the target symbol or the second text segmentation is a text segmentation marking the end of the text, pausing the text segmentation output of the large language model, performing a second reasoning process, and determining a node to be processed in the mind map based on the second reasoning process, connecting the node to be processed with other existing nodes in the mind map using a directed line, and the direction of the directed line is from the existing node to the node to be processed; the second reasoning process includes: splicing the text segmentations that are not part of the executed reasoning process among the text segmentations output by the large language model in the output order; S3, determining a value of a self-attention strength measurement index between the second reasoning process and the executed reasoning process, and when the value of the self-attention strength measurement index is less than or equal to a threshold, correcting the self-attention strengths between the second reasoning process and the text segmentations that do not belong to the same reasoning process in the executed reasoning process to zero, and deleting the connection between the second reasoning process and the executed reasoning process in the mind map; The target symbols include: line break, colon, period, question mark or semicolon.

12. A model training device, characterized in that: include: A first acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logical reasoning question, a step-by-step answer to the logical reasoning question, and a text segmentation marking the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logical reasoning question; The second acquisition module is used to obtain the deep dynamic prefix parameter; A first processing module, configured to construct an initial large language model using the deep dynamic prefix parameters; The second processing module is used to train the initial large language model using the training data set, construct a mind map, and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameters.

13. A model reasoning device, characterized in that: include: A first acquisition module is used to acquire the logical reasoning problem to be processed; A first processing module, configured to use the logic reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the method according to any one of claims 1 to 8; The second acquisition module is used to obtain the final answer to the logical reasoning problem to be processed based on the output text of the large language model.

14. A model training device, characterized in that: include: processors and transceivers; Wherein, the processor is used for: Acquire a training data set, wherein the training data set includes a plurality of training texts, each of which includes a logic reasoning question, a step-by-step answer to the logic reasoning question, and a text segmentation indicating the end of the text, wherein the step-by-step answer includes prompt information, and the prompt information is used to extract a final answer to the logic reasoning question; Get deep dynamic prefix parameters; Building an initial large language model using the deep dynamic prefix parameters; The initial large language model is trained using the training data set to construct a mind map and obtain a loss function, wherein the loss function is used to adjust the deep dynamic prefix parameter.

15. A model reasoning device, characterized in that: include: processors and transceivers; Wherein, the processor is used for: Get the logical reasoning problems to be processed; Using the logical reasoning problem to be processed as an input of a large language model to obtain an output text of the large language model, wherein the large language model is trained using the method according to any one of claims 1 to 8; Based on the output text of the large language model, a final answer to the logical reasoning problem to be processed is obtained.

16. A communication device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is used to read the program in the memory to implement the steps in the model training method as described in any one of claims 1 to 8; or to implement the steps in the model reasoning method as described in any one of claims 9 to 11.

17. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the model training method described in any one of claims 1 to 8 are implemented; or the steps of the model inference method described in any one of claims 9 to 11 are implemented.

18. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps in the model training method described in any one of claims 1 to 8; or implement the steps in the model reasoning method described in any one of claims 9 to 11.

Citation Information

Patent Citations

  • Mathematical problem answering model training method and device

    CN116595159A

  • Large language model distillation method based on thinking chain

    CN116739110A

  • Method and system for generating questions and answers based on knowledge graph

    KR102697127B1

Cited By

  • Method and system for enhancing logical reasoning ability of large language model

    CN120806171A