Large model reasoning method, device, related equipment and computer program product

Abstract

The present application discloses a large-model reasoning method, apparatus, related equipment, and computer program product, relating to the field of artificial intelligence. The present application first obtains a problem statement of a problem to be processed, and then, in the process of chain reasoning based on the problem statement in the large model, determines whether the current node is a key node based on the input token of the current reasoning node. If not, the original output token of the previous node is compressed, and the subsequent node is inferred based on the compression processing result. The present application first determines whether the reasoning node is key or not, and then compresses the output token of non-key reasoning nodes when performing subsequent reasoning, thereby reducing the amount of input data required to be processed in the subsequent reasoning process, speeding up the subsequent processing speed, and helping to optimize the computational efficiency of chain reasoning, especially long chain reasoning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a large-scale model reasoning method, apparatus, related equipment, and computer program product. Background Art

[0002] In scenarios such as mathematical reasoning, code generation, and medical diagnosis, where large models rely on logic for reasoning, implicit reasoning flaws prevent them from producing good results. The Chain-of-Thought (CoT) approach enables large models to perform explicit reasoning. Specifically, large models can break down complex problems into multiple subproblems and perform multi-step reasoning until they reach the final result, thereby optimizing the model's reasoning capabilities.

[0003] Based on this, combining the explicit reasoning advantages of chain thinking with the capabilities of large-scale models can better obtain inference results for complex reasoning tasks. However, when large-scale artificial intelligence models apply chain thinking methods to handle complex reasoning tasks, they usually need to perform large-scale, long chains of reasoning calculations. As the reasoning chain lengthens, the amount of calculation increases significantly and the reasoning speed deteriorates significantly. The computational efficiency of large models when handling complex reasoning tasks is becoming increasingly prominent. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a large model reasoning method, apparatus, related equipment and computer program product to improve the speed of large model reasoning. The specific solution is as follows:

[0005] In a first aspect, a large model reasoning method is provided, comprising:

[0006] Get the problem statement of the problem to be addressed;

[0007] During the chain reasoning process of the large model based on the problem statement, determining whether the current reasoning node is a key node based on the input word token of the current reasoning node;

[0008] When the current inference node is not a key node, the original output token of the current inference node is compressed, and subsequent nodes are inferred based on the compression processing result; wherein, the original output token is the token generated by the large model during the inference process corresponding to the current inference node.

[0009] In a second aspect, a large model reasoning device is provided, comprising:

[0010] A problem acquisition unit, used to acquire a problem statement of a problem to be processed;

[0011] A key node determination unit is used to determine whether a current inference node is a key node according to an input word token of the current inference node during the chain inference process of the large model based on the problem statement;

[0012] The output word unit configuration unit is used to compress the original output token of the current inference node when the current inference node is not a key node. The original output token is the token generated by the large model during the inference process corresponding to the current inference node; and the compression processing result is determined as the output token for inferring subsequent nodes.

[0013] In a third aspect, an electronic device is provided, comprising: a memory and a processor;

[0014] The memory is used to store programs;

[0015] The processor is used to execute the program to implement the various steps of the large model reasoning method described in the first aspect of this application.

[0016] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the large model reasoning method described in the first aspect of this application are implemented.

[0017] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the various steps of the large model reasoning method described in the first aspect of this application.

[0018] By using the above technical solution, the present application first obtains the problem statement of the problem to be processed, and then, during the large-model chain reasoning process, determines whether the current node is a key node based on the input token of the current reasoning node. If not, the original output token of the previous node is compressed, and the subsequent nodes are inferred based on the compression processing result. The present application first determines whether the reasoning node is key or not, and then compresses the tokens of non-key reasoning nodes when performing subsequent reasoning, reducing the amount of input data required to be processed in the subsequent reasoning process, speeding up the subsequent processing speed, and helping to optimize the computational efficiency of chain reasoning, especially long chain reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are only intended to illustrate the preferred embodiment and are not to be considered as limiting the present application. The same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1A schematic diagram of an implementation system architecture of the large model reasoning method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a large model reasoning method provided in an embodiment of the present application;

[0022] Figure 3 This example shows the schematic diagram of the processing process when a large model performs chain reasoning;

[0023] Figure 4 A schematic diagram of the structure of a large model reasoning device provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] The applicants in this case discovered through research that chain reasoning generates a large number of intermediate steps and explanatory text to simulate the human thought process. As the chain reasoning process progresses and the reasoning chain lengthens, the model needs to process an increasing number of input tokens (tokens, words, representing the smallest unit or basic element of text processing) and the computational speed slows down. Furthermore, CoT-based reasoning schemes often generate unnecessary steps and text. In other words, the tokens generated by large models often contain a certain number of redundant tokens. For example, mathematical benchmark test results of large models (such as the 100-billion-parameter model GPT-4o and DeepSeek-R1) on the basic mathematical dataset GSM8K show that traditional chain reasoning generates an average of 215±38 redundant tokens, of which approximately 32.7% have a direct impact on the final conclusion.

[0027] Based on this, the unnecessary redundant tokens generated by large models have an adverse impact on computing speed, leading to inference efficiency issues. To address the above issues, this application provides a large model inference method, apparatus, related equipment, and computer program product to reduce the number of tokens that large models need to process, thereby optimizing the inference efficiency of large models.

[0028] This application provides a large model reasoning method that can be applied to Figure 1The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 (This description is made using one server as an example).

[0029] The terminal 100 or the server 200 can be used alone to execute the large model reasoning method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used together to execute the large model reasoning method provided in the embodiment of the present application.

[0030] Next describe Figure 1 The product form of the mid-terminal 100;

[0031] The terminal 100 in the embodiment of the present application may be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), etc., and the embodiment of the present application does not limit this.

[0032] The embodiment of the present application provides a large model reasoning method, which is illustrated by applying the method to a computer device. The computer device can be Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 2 The large model reasoning method provided in the embodiment of the present application specifically includes the following steps:

[0033] Step S101: Obtain a problem statement of a problem to be processed.

[0034] The aforementioned problem statement may refer to text input by the user regarding the problem to be solved. When the large model performs chain reasoning based on the problem statement, the initial token input to the large model is generated based on the aforementioned problem statement.

[0035] Step S102: During the chain reasoning process of the large model based on the problem statement, determine whether the current reasoning node is a key node based on the input token of the current reasoning node.

[0036] It should be noted that in the existing chained reasoning process corresponding to any inference node, the large model generates a corresponding token based on the input token of this node, obtains the inference result of this node, and concatenates this token with the input token of this node to obtain the input token of the next node for subsequent reasoning, until the final inference result is obtained. The token initially input to the large model is the aforementioned initial token. In the large model reasoning process, the current inference node refers to the inference node being processed.

[0037] To address the issue of computational speed degradation caused by excessively long input tokens, this application determines whether the current inference node is critical during chain reasoning based on the problem statement performed by the large model. This includes determining whether the token generated during the reasoning process corresponding to the current inference node is critical to the final reasoning result, i.e., whether the token generated during the reasoning process corresponding to the current inference node is critical to the final reasoning result, thereby providing a basis for subsequent token compression. To avoid unnecessary repetitive operations, the aforementioned current inference node may not include the last inference node, i.e., the logical node that obtains the final reasoning result.

[0038] Step S103: If the current inference node is not a key node, the original output token of the current inference node is compressed, and subsequent nodes are inferred based on the compression processing result.

[0039] Among them, the original output token is the token generated by the large model during the reasoning process corresponding to the current reasoning node, that is, the token that is spliced ​​with the input token of the current reasoning node in the existing chain reasoning scheme for use by the next reasoning node. Compared with the original output token, the compressed original output token (that is, the compression processing result) has a smaller data volume. By splicing the compression processing result with the input token of the current reasoning node, a shorter input token for the next reasoning node can be obtained. In other words, this embodiment compresses the subsequent input of the large model to a certain extent through compression processing. In addition, since the compressed processing is performed on the output token of non-critical nodes, the degree of interference of the inference results of non-critical nodes on the inference results of critical nodes in the subsequent reasoning process can be reduced, and the reasoning anomaly problem caused by the flooding of critical tokens can be alleviated to a certain extent, providing a basis for ensuring the quality of reasoning.

[0040] The above scheme first determines whether the inference node is critical or not, and then compresses the tokens of non-critical inference nodes during subsequent inference. This reduces the amount of input data required for subsequent inference processing, speeds up subsequent processing, and helps optimize chain reasoning, especially the computational efficiency of long chain reasoning.

[0041] In one or more embodiments provided in this application, the process of chain reasoning based on the problem statement of the large model may also include:

[0042] Step S104: When the current inference node is a key node, infer the subsequent node based on the original output token of the current inference node.

[0043] The above solution only compresses the original output tokens of non-critical reasoning nodes. By retaining the original output tokens of critical reasoning nodes, the critical reasoning information is retained. This can reduce the amount of subsequent input data while avoiding reasoning anomalies caused by the lack of critical reasoning information.

[0044] In another possible implementation, when the current inference node is a key node, the original output token of the current inference node may also be compressed, and the degree of compression is lower than the degree of compression in step S103.

[0045] Based on this, by compressing the original output tokens of the inference nodes, the amount of subsequent input data of the model is further reduced; and by compressing the original output tokens of key inference nodes and non-key inference nodes to varying degrees, key inference information can be retained to a certain extent, providing a basis for highlighting key inference logic.

[0046] The following is an exemplary description of the process of determining key nodes.

[0047] In one or more embodiments provided herein, step S102, determining whether the current inference node is a key node based on the input token of the current inference node, may include the following steps S201-S202:

[0048] Step S201: Obtain the attention matrix calculated by the large model based on the input token of the current inference node.

[0049] For example, the attention matrix can be the attention matrix calculated by the hidden layer of the large model (such as the Transformer module). Assuming that the Transformer module consists of m layers of Transformers and the Transformer module adopts a multi-head attention mechanism, in the reasoning process of a single node, the input token sequence length of this node is n, and the attention matrix calculated by the l-th layer Transformer at the h-th attention head can be expressed as ,in, , where R represents a real number, then the matrix Parameters in It represents the attention value of the i-th token calculated by the l-th layer Transformer on the j-th token in the attention head h.

[0050] Step S202: Determine, based on the attention matrix, whether the uncertainty of the attention distribution of each token constituting the input token is within a preset limit corresponding to the input token.

[0051] Specifically, if the attention distribution uncertainty of at least a preset number of tokens in the input tokens is within a preset limit, the current inference node is a key node, and the processing steps corresponding to the key node are executed, such as step S104. Otherwise, the current inference node is not a key node, and the processing steps corresponding to the non-key node are executed, such as step S103. Optionally, the aforementioned preset number can be half of the total number of tokens in the input tokens.

[0052] This embodiment determines the uncertainty of attention distribution of each token input into the large model based on the attention matrix, which helps to determine the key logical nodes that are more important for generating reasoning conclusions.

[0053] In one or more embodiments provided herein, step S201 of obtaining the attention matrix calculated by the large model based on the input token of the current inference node may include:

[0054] Get the attention matrix calculated based on the input token of the current inference node for some pre-set hidden layers in the large model.

[0055] Optionally, the aforementioned hidden layer may refer to a transformer layer, or other hidden layers in other large model structures, which is not limited in this application.

[0056] The above scheme uses the attention matrix of some hidden layers to judge the key nodes, which reduces the pressure of numerical calculation and improves the efficiency of key node judgment; when the pre-set number of hidden layers is greater than 1, this embodiment provides rich attention information for key node judgment by fusing the attention of different layers, thereby improving the quality of key node judgment.

[0057] In a possible implementation, the preset n hidden layers may include at least one of hidden layers for extracting low-level features, mid-level features, and high-level features.

[0058] For example, in a 12-layer Transformer, the 3rd, 6th, and 9th layers can be selected as pre-set partial hidden layers.

[0059] In one or more embodiments provided in the present application, for each token in the input token, the attention distribution uncertainty of the current token is represented by the attention entropy fusion value of the current token, wherein the attention entropy fusion value of the current token is the fusion value of the attention entropy values ​​of the current token in each attention head determined based on the obtained attention matrices.

[0060] Taking the attention matrix calculated at the lth layer as an example, the attention entropy value of the i-th token in the input token at the attention head h can be expressed as: , where represents the logarithm with natural logarithm (ie e) as the base, that is for The natural logarithm of .

[0061] Optionally, the aforementioned attention entropy fusion value may be a weighted fusion value of attention entropy values.

[0062] For example, the attention entropy fusion value of the i-th token in the input token can be expressed as: , where H represents the number of attention heads, N represents the set of layer identifiers of the pre-specified n hidden layers, Represents the attention entropy weight of the h-th attention head in the l-th layer, which is adaptively distributed through the importance of the attention head. Specifically, , is the global entropy mean, which is optimized by back propagation.

[0063] This embodiment comprehensively considers the attention of multiple layers and multiple attention heads when determining key nodes, which helps to improve the accuracy of key node determination.

[0064] In one or more embodiments provided in this application, the preset limit is positively correlated with the overall attention distribution uncertainty of the input token and the complexity of the problem statement.

[0065] In other words, the preset limit changes dynamically with the input token. The higher the overall attention distribution uncertainty of the input token, the higher the acceptable attention distribution uncertainty limit. Moreover, the preset limit is related to the problem situation that the large model is dealing with. The more complex the problem statement, the more information contained in the input token (such as the initial token), and the higher the uncertainty of the attention value of different tokens. In order to reduce the possibility of misjudgment of key nodes, the acceptable attention distribution uncertainty limit is also higher.

[0066] By dynamically presetting limits that are adapted to the current inference node, a basis is provided for achieving key node determination tasks in different scenarios and improving the accuracy of key node determination.

[0067] In one possible implementation, the uncertainty of the overall attention distribution of the input token can be represented by the statistical value of the attention entropy fusion value of the input token.

[0068] For example, the preset limit corresponding to the current inference node can be expressed as θ=μ H +βσ H , where μ Hrepresents the mean of the attention entropy fusion values ​​of each token of the input token of the current reasoning node, β represents the standard deviation coefficient positively correlated with the complexity of the problem statement, σ H Indicates the standard deviation of the attention entropy fusion value of each token of the input token of the current inference node.

[0069] Optionally, the standard deviation coefficient β satisfies: β∈[0.3,0.7]. In one possible implementation, the standard deviation coefficient can be dynamically adjusted according to the following formula. Specifically, , where L represents the complexity of the problem statement, such as syntactic complexity, and tanh represents the hyperbolic tangent function.

[0070] Based on the above content, if the attention entropy fusion values ​​of a certain number of tokens in the input token deviate too much from the mean fusion entropy value, it can be indicated that the attention distribution uncertainty of the input token of the current inference node is high or relatively scattered. The possibility of generating a token that is more critical to the inference result based on the input token of this node is low. Therefore, it is judged as a non-critical node and the original output token is compressed.

[0071] Next, the compression processing process of the output token of non-critical nodes is illustrated.

[0072] In one or more embodiments provided in this application, the aforementioned step S103, compressing the original output token of the current inference node, may include the following steps:

[0073] Step S301: Tokenize the original output token to obtain a tokenization result.

[0074] The symbolization process described above involves converting a specific token or type of token within the original output token into a symbol specified by the rule, based on pre-defined conversion rules. By using symbols to represent tokens, we can preserve some token information while reducing the amount of data.

[0075] Step S302: Encode the symbolization processing result to obtain an encoding result as the compression processing result.

[0076] The encoding process may refer to converting the symbolic processing result into a specific type of encoding, such as binary encoding, to further compress the token data volume.

[0077] The above scheme is based on a double-layer compression mechanism, which compresses the output tokens of non-critical nodes in two dimensions, reducing the amount of data carrying the reasoning information of non-critical nodes; based on this, the model can perform subsequent reasoning based on a smaller amount of input data, thereby improving the subsequent processing efficiency of the model to a certain extent. In addition, the present application solves the reasoning anomaly problem caused by the flooding of critical tokens to a certain extent by retaining critical tokens and compressing non-critical tokens, which helps to ensure the quality of reasoning. In addition, the compression processing result can retain non-critical node information to a certain extent, that is, the chain reasoning scheme provided by the present application retains the interpretable intermediate state. Therefore, the present application reduces token consumption while ensuring the reliability of reasoning, which helps to achieve high-efficiency and high-quality reasoning tasks.

[0078] In one or more embodiments provided herein, step S301, symbolizing the original output token to obtain a symbolized processing result, may include the following steps:

[0079] Step S401: Determine the token category of the original output token.

[0080] In one possible implementation, the original output tokens can be parsed using a syntax tree to obtain a syntax tree for the original output tokens. For example, a dependency parser based on Stanford CoreNLP can be used to generate a syntax tree. The generated syntax tree can include entity annotations and can be used to determine logical relationships between entities, identify verb tenses, and so on. Token categories are determined based on the syntax tree.

[0081] It should be noted that the token categories described in this application can include entity classes, verb classes, and numerical classes. In addition, other tokens such as prepositions, conjunctions, and adjectives can be divided into logical connective classes and modifying vocabulary classes according to their grammatical roles.

[0082] Step S402: Extract tokens of a preset token category from the original output tokens in order based on the determination result.

[0083] The preset token category can be at least one of the aforementioned token categories. The more token categories a preset token category includes, the lower the degree of symbolic compression, and the richer the reasoning information about the current node retained in the symbolic processing result. Based on this, the preset token category can be set based on the weight of the token's influence on the reasoning logic.

[0084] Step S403: Perform symbol conversion on the extracted token according to the symbol generation rule corresponding to the preset token category to generate a symbolized processing result.

[0085] Optionally, when generating the symbolic processing result, the compressed symbols of each extracted token can be directly concatenated. In another possible implementation, the compressed symbols of each extracted token can also be concatenated using a preset symbol. This application does not limit the specific method of generating the symbolic processing result from the compressed symbols of the token.

[0086] In one or more embodiments provided in the present application, the preset token category includes at least one of a numerical class, a verb class, and an entity class.

[0087] The following is an illustrative description of the symbol generation rules corresponding to the preset token categories. The symbol generation rules corresponding to the numerical class may include: converting the numerical class token into a first preset symbol and associating the numerical dimension metadata. Assuming that the first preset symbol is #, the numerical class token can be converted to: #[number]. The symbol generation rules corresponding to the verb class may include: converting the verb class token into a second preset symbol and carrying the timing code of the current token. Assuming that the second preset symbol is V$, the verb class token can be converted to: V$[timing code]. Optionally, 1 can be used to represent the past tense, 2 can be used to represent the present tense, and 3 can be used to represent the future tense. The symbol generation rules corresponding to the entity class may include: converting the entity class token into a third preset symbol and carrying the entity identifier of the current token. Optionally, the symbol generation rules corresponding to the entity class may also include: carrying the domain code of the current token while carrying the entity identifier of the current token. Assuming that the third preset symbol is E@, the entity class token can be expressed as E@[domain code]_[entity identifier]. Based on the above, the symbolic representations of the tokens of the preset token category can be connected by a fourth preset symbol in the order of the tokens in the original output token to obtain the compressed symbol of the original output token, that is, the aforementioned symbolic processing result. Optionally, the aforementioned fourth preset symbol can be |.

[0088] The applicants in this case have experimentally discovered that, compared to logical connective and modifier word classes, numerical, verb, and entity tokens are more important to reasoning. For example, retaining numerical, verb, and entity tokens as pre-defined token categories during symbolic compression can ensure over 92% inference fidelity in subsequent reasoning. Based on this, to improve compression, at least one of the numerical, verb, and entity tokens can be used as a pre-defined token category.

[0089] Based on the above content, for example, assuming that the original output token is: has been diagnosed with pneumonia for 3 days, the symbolic processing result obtained according to the aforementioned symbol generation rules can be expressed as: V$1|E@MED_J18.9|#3, where V$1 represents a verb in the past tense, MED represents the domain code "Medicine (MED)" of the entity class token pneumonia, and J18.9 represents the entity identifier of the entity class token pneumonia.

[0090] In addition, the preset token category may also include logically connected word classes, and the symbol generation rules corresponding to the logically connected word classes may include: converting the tokens of the logically connected word classes into corresponding logical symbols, such as converting "because" into "@cause". The preset token category may also include modifying word classes, and the symbol generation rules corresponding to the modifying word classes may include: converting the tokens of the modifying word classes into symbols of the corresponding part of speech, such as converting adjectives into the ADJ symbol. In another possible implementation, the tokens of the modifying word classes may also be retained, such as retaining the original word of the adjective token, pending subsequent encoding compression.

[0091] In one or more embodiments provided in this application, the aforementioned step S302, encoding the symbolization result, may include:

[0092] The symbolic processing result is encoded based on a pre-built compression dictionary.

[0093] The compression dictionary is a symbol-to-code mapping table that matches the domain of the problem to be processed, i.e., a pre-constructed domain-adaptive dictionary. In one possible implementation, the aforementioned encoding may refer to Huffman coding, and the specific process of the encoding and compression processing may include: constructing a Huffman tree, constructing a compression dictionary based on the Huffman tree, and performing Huffman encoding on the symbolized processing result based on the compression dictionary to obtain the encoded compression result. The Huffman tree construction scheme and compression dictionary generation scheme can refer to existing schemes and are not further described here.

[0094] Based on the above, the dual-layer semantic compression scheme of symbolic compression and coding compression provided by this application can be implemented based on the following code. Specifically:

[0095] class SymbolicCompressor:

[0096] def __init__(self):

[0097] self.abstract_map = {

[0098] 'numerical': '#',

[0099] 'verb': 'V$', # Add tense marker / timing code (V$1 indicates past tense)

[0100] 'entity': 'E@{}' # retain entity type (E@disease)

[0101] }

[0102] / / Compression def compress(self, text, K):

[0103] # Add entity type inheritance mechanism

[0104] compressed = []

[0105] for token in tag(text): # Use Stanford CoreNLP's depparse and ner modules for grammatical annotation

[0106] if token in K:

[0107] compressed.append(token)

[0108] else:

[0109] sym_type = self._get_symbol_type(token)

[0110] compressed.append(self.abstract_map[sym_type].format(token.entity_type))

[0111] return '|'.join(compressed)

[0112] class SymbolicCompressor:

[0113] def _huffman_encode(self, symbol_sequence):

[0114] # Encoding based on domain-adaptive compression dictionary

[0115] freq = Counter(symbol_sequence)

[0116] heap = [[weight, [sym, ""]] for sym, weight in freq.items()]

[0117] heapq.heapify(heap)

[0118] # ...Huffman tree construction process...

[0119] return encoded_sequence

[0120] The above scheme performs context-aware compression on token sequences. Specifically, while retaining the complete tokens corresponding to key nodes, it achieves compression at the grammatical and semantic layers through syntax tree parsing, token category determination, symbol conversion, and encoding processing, providing a basis for improving subsequent reasoning efficiency.

[0121] In one or more embodiments provided in this application, after inferring subsequent nodes based on the compression processing result, the following steps may also be included:

[0122] Step S501: Determine the reasoning path to be verified and calculate the path confidence.

[0123] The reasoning path to be verified includes a parallel reasoning node, which is a parallel node of a non-critical reasoning node and uses the compression processing result as an output token.

[0124] In one possible implementation, after determining the output token of the current inference node (i.e., the original output token or the compressed result of the original output token), the current inference path graph can be constructed. The constructed inference path graph can be expressed as: G = (V, E), V = {v1, v2, ..., v z}, E={(v i ,v j )|logical dependency}, where V represents the set of inference nodes, including a total of z inference nodes, including: an explicit inference node generated based on the original CoT with the original output token as the output token, and an implicit potential inference node triggered by the compression process with the compression result of the original output token as the output token; and the explicit inference node and the implicit potential inference node corresponding to the same original output token are parallel nodes, forming a parallel inference branch; E represents the edge set, which includes the edge formed by two inference nodes with a logical dependency relationship. For example, if v is determined by data flow analysis j The input dependency v iThe output of , then there is an edge e ij ∈ E. Then, based on the current reasoning path graph, the reasoning path to be verified is determined, such as the reasoning path with the parallel reasoning node as the end of the path, so as to realize timely verification backtracking of the compression processing.

[0125] Step S502: When the path confidence is lower than a preset confidence threshold, compression and restoration are performed on the compression processing result, and subsequent nodes are re-inferred based on the compression and restoration result.

[0126] The subsequent reasoning node refers to the subsequent reasoning node of the possible reasoning node with the compression processing result as the output head. When the path confidence is lower than the preset threshold, it can be indicated that the possibility of obtaining an accurate reasoning result based on the current reasoning path is low, which may be due to the loss of key reasoning information. Based on this, compression and restoration are performed, and local reasoning is re-performed, which provides a basis for ensuring the quality of reasoning. Based on the above scheme, when the path confidence is low, the aforementioned parallel reasoning nodes with the compression processing result as the output token can be removed, that is, the low-confidence reasoning path is removed and the high-confidence reasoning path is retained.

[0127] Optionally, the aforementioned symbolization process may further include: establishing an index table of symbols and original tokens.

[0128] Exemplarily, the index table can be an inverted index table to facilitate symbol restoration; the index table can be stored in a memory cache for quick access. Furthermore, the index table can be used to locate the semantic unit corresponding to the symbol during error propagation analysis. Based on this, the aforementioned compression and restoration process can be performed based on the index table. In another possible implementation, the original output token corresponding to the compression processing result can be obtained as the compression and restoration result.

[0129] In one or more embodiments provided in this application, calculating the path confidence may include:

[0130] Calculate the edge confidence of each edge that constitutes the reasoning path to be verified, use the depth attenuation factor to correct the calculated edge confidence of each edge, and superimpose the corrected edge confidences to obtain the path confidence.

[0131] Each edge consists of two inference nodes with a dependency relationship. When correcting the edge confidence of any edge, the depth attenuation factor used is a quantity that is negatively correlated with the inference depth of the edge. For example, the path confidence calculation formula can be expressed as: ; In the formula, p represents the reasoning path to be verified, e represents the edge on the reasoning path to be verified, and depth(e) represents the reasoning depth of edge e. ij For example, the edge confidence C(e ij )=softmax(H(vi )×W×H(v j )), where W represents the edge e ij The weight of H(V i ) represents the inference node V i The probability distribution or hidden state, H(V j ) represents the inference node V j The probability distribution or hidden state of .

[0132] It should be noted that in chain reasoning scenarios, especially long chain reasoning scenarios, a large number of redundant tokens may also cause tokens that are more important to the conclusion to be drowned out, increasing the risk of reasoning errors. For example, the drowning of key logical nodes will cause the error propagation rate to increase exponentially with the increase in the number of reasoning steps. This embodiment uses a depth attenuation factor to reduce the error propagation rate of long chain reasoning, especially long chain reasoning with more than 20 steps, to a certain extent alleviating the error propagation accumulation problem of traditional chain reasoning.

[0133] Optionally, when applying this solution, Monte Carlo Tree Search (MCTS) can be used to explore paths during chained reasoning based on the problem statement in a large model, retaining the top-K high-confidence branches. This allows for a real-time dynamic pruning mechanism to promptly prune redundant branches during reasoning, improving reasoning efficiency. Furthermore, K can dynamically change with the number of reasoning steps.

[0134] Figure 3 Taking a single inference node as an example, the following diagram shows the processing process of the input token based on a single inference node when a large model performs chain inference. Figure 3As shown, the present application first performs key node judgment based on the input token of the inference node (also known as the logic node) to extract the key logic. Specifically, the application first obtains the attention matrix calculated based on the multi-head attention mechanism, and then performs entropy calculation based on this to obtain the attention entropy fusion value of each token that constitutes the input token. Based on this, the preset limit that changes dynamically with the inference node is determined and the key node judgment is performed. Then, compression processing is performed based on the key node judgment result. Specifically, on the basis of retaining the original output token of the corresponding key node, the original output token of the non-key node is parsed to determine the token category. Then, symbol mapping is performed according to the symbol generation rule corresponding to the preset token category to obtain a symbolized sequence. Finally, the symbolized sequence is compressed through encoding processing to obtain the corresponding compression processing result. On this basis, the token compression is dynamically back-tested. Specifically, the reasoning path to be verified containing the implicit reasoning node (the reasoning node triggered by the compression processing and with the compression processing result as the output token) is determined, and the path confidence is calculated. If the path confidence does not meet the standard, the compression processing result is restored, and local reasoning is re-performed based on the restored processing result. If the path confidence meets the standard, it indicates that the subsequent reasoning process based on the compression processing result is highly reliable, and subsequent reasoning is performed accordingly to accelerate reasoning. This application improves the computational efficiency and reasoning reliability of large models by establishing a hierarchical distillation mechanism of key logic extraction-compression processing-dynamic verification, and has strong cross-scenario versatility.

[0135] The following describes the large model training process involved in this application.

[0136] In one or more embodiments provided in the present application, the loss functions used in the large model training stage include: a first loss function, a second loss function, and a third loss function.

[0137] Among them, the first loss function L task The degree of difference between the problem reasoning result output by the large model and the standard reasoning result can also be called task loss. It is used to measure the correctness of the representation answer and can be determined by cross entropy.

[0138] The second loss function L node The degree of difference between the key nodes determined during the reasoning process and the actual key nodes can also be called node loss. For example, the F1 value of the key nodes determined by two-way contrastive learning can be used to comprehensively measure precision and recall.

[0139] The third loss function L compressThe degree of difference between the compressed token sequence generated by the large model and the actual compressed token sequence can also be called compression loss. For example, it can be represented by the symbolic sequence BLEU (Bilingual Evaluation Understudy, BLEU) value.

[0140] Based on the above content, when training large models, sample data can be collected from fields such as mathematical reasoning, code generation, and medical diagnosis, and manually annotated with three levels of semantics, including: the first level, the complete reasoning process; the second level, the set of key nodes; the third level, the symbolic compression results, and the training data is configured accordingly for subsequent training.

[0141] In one or more embodiments provided in the present application, the large model can be trained using a progressive training method; the training process of the large model includes a first training stage, a second training stage, and a third training stage performed in sequence.

[0142] The first training phase takes minimizing the value of the second loss function as the training objective, focusing on optimizing the judgment accuracy of key nodes; optionally, the training data used in the first training phase can contain a certain proportion of pure mathematical data, such as 70% of pure mathematical data. In addition, the loss function used can be expressed as L = λ2 × L node , where the value of λ2 can be set according to demand.

[0143] The second training phase takes minimizing the value of the joint loss function composed of the second loss function and the third loss function as the training objective, focusing on optimizing the judgment accuracy of key nodes and compression loss; optionally, the training data used in the second training phase can contain a certain proportion of mixed domain data, such as 50% mixed domain data. In addition, the loss function used can be expressed as L = λ2 × L node +λ3×L compress , where the values ​​of λ2 and λ3 can be set according to requirements.

[0144] The third training stage takes minimizing the value of the joint loss function composed of the first loss function, the second loss function and the third loss function as the training objective. Optionally, the training data used in the third training stage may contain a certain proportion of real scene data, such as 30% of real scene data. In addition, the loss function used can be expressed as L = λ1 × L task +λ2×L node +λ3×L compress , where the values ​​of λ1, λ2 and λ3 can be set according to requirements.

[0145] The large model reasoning device provided in the embodiment of the present application is described below. The large model reasoning device described below and the large model reasoning method described above can be referenced to each other.

[0146] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a large model reasoning device disclosed in the embodiment of this application. Figure 4 As shown, the device may include:

[0147] A problem obtaining unit 11 is used to obtain a problem statement of a problem to be processed;

[0148] A key node determination unit 12 is configured to determine whether a current inference node is a key node according to an input word-meta token of the current inference node during the chain inference process of the large model based on the problem statement;

[0149] The output word unit configuration unit 13 is used to compress the original output token of the current inference node when the current inference node is not a key node. The original output token is the token generated by the large model in the inference process corresponding to the current inference node; and the compression processing result is determined as the output token for inferring subsequent nodes.

[0150] In one or more embodiments provided in this application, the process of the output word element configuration unit 13 compressing the original output token of the current inference node may include:

[0151] Performing symbolic processing on the original output token to obtain a symbolic processing result;

[0152] The symbolization processing result is encoded to obtain an encoding result as the compression processing result.

[0153] In one or more embodiments provided in the present application, the process of the output word unit configuration unit 13 performing symbolic processing on the original output token to obtain the symbolic processing result may include:

[0154] Performing token category determination on the original output token;

[0155] Extracting tokens of a preset token category from the original output tokens in order based on the determination result;

[0156] According to the symbol generation rule corresponding to the preset token category, the extracted token is symbolically converted to generate the symbolized processing result.

[0157] In one or more embodiments provided in this application, the preset token category includes: at least one of a numerical class, a verb class, and an entity class;

[0158] The symbol generation rule corresponding to the numerical class includes: converting the numerical class token into a first preset symbol and associating the numerical dimension metadata;

[0159] The symbol generation rule corresponding to the verb class includes: converting the verb class token into a second preset symbol and carrying the timing code of the current token;

[0160] The symbol generation rule corresponding to the entity class includes: converting the entity class token into a third preset symbol and carrying the entity identifier of the current token.

[0161] In one or more embodiments provided in the present application, the process of encoding the symbolization result by the output word element configuration unit 13 may include:

[0162] The symbolic processing result is encoded based on a pre-built compression dictionary; the compression dictionary is a symbol-code mapping table that matches the field to which the problem to be processed belongs.

[0163] In one or more embodiments provided herein, the process of the key node determination unit 12 determining whether the current inference node is a key node based on the input token of the current inference node may include:

[0164] Obtain the attention matrix calculated by the large model based on the input token;

[0165] Determining, based on the attention matrix, whether the attention distribution uncertainty of each token constituting the input token is within a preset limit corresponding to the input token;

[0166] If the attention distribution uncertainty of at least a preset number of tokens in the input tokens is within a preset limit, the current reasoning node is a key node; otherwise, the current reasoning node is not a key node.

[0167] In one or more embodiments provided herein, the process in which the key node determination unit 12 obtains the attention matrix calculated by the large model based on the input token may include:

[0168] Obtain the attention matrix calculated based on the input token for some pre-set hidden layers in the large model.

[0169] In one or more embodiments provided in this application, the preset limit is positively correlated with the overall attention distribution uncertainty of the input token and the complexity of the problem statement.

[0170] In one or more embodiments provided in the present application, for each token in the input token, the attention distribution uncertainty of the current token is represented by the attention entropy fusion value of the current token, wherein the attention entropy fusion value of the current token is the fusion value of the attention entropy values ​​of the current token in each attention head determined based on the obtained attention matrices.

[0171] In one or more embodiments provided in the present application, the output word unit configuration unit 13 may also be configured to determine the original output token as the output token for reasoning subsequent nodes when the current inference node is a key node.

[0172] In one or more embodiments provided in this application, the apparatus may further include: a retrospective verification unit, configured to:

[0173] Determine a reasoning path to be verified and calculate the path confidence; the reasoning path to be verified includes a parallel reasoning node, the parallel reasoning node is a parallel node of a non-critical reasoning node and the parallel reasoning node uses the compression processing result as an output token;

[0174] In the case that the path confidence is lower than a preset confidence threshold, compression and restoration processing is performed on the compression processing result, and subsequent nodes are re-inferred based on the compression and restoration result.

[0175] In one or more embodiments provided in this application, the process of calculating the path confidence by the backtracking verification unit may include:

[0176] Calculate the edge confidence of each edge constituting the reasoning path to be verified, use the depth attenuation factor to correct the calculated edge confidence of each edge, and superimpose the corrected edge confidences to obtain the path confidence; wherein, each edge is composed of two reasoning nodes with a dependency relationship.

[0177] In one or more embodiments provided in this application, the loss functions used in the large model training stage include: a first loss function, a second loss function, and a third loss function;

[0178] Among them, the first loss function represents the degree of difference between the problem reasoning result output by the large model and the standard reasoning result; the second loss function represents the degree of difference between the key nodes determined during the reasoning process and the actual key nodes; the third loss function represents the degree of difference between the compressed token sequence generated by the large model and the actual compressed token sequence.

[0179] In one or more embodiments provided in the present application, the large model is trained using a progressive training method; the training process of the large model includes a first training stage, a second training stage, and a third training stage performed sequentially;

[0180] Among them, the first training stage takes minimizing the value of the second loss function as the training goal, the second training stage takes minimizing the value of the joint loss function composed of the second loss function and the third loss function as the training goal, and the third training stage takes minimizing the value of the joint loss function composed of the first loss function, the second loss function and the third loss function as the training goal.

[0181] It should be noted that each unit in the large model inference device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above units can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above units.

[0182] An electronic device is also provided in an embodiment of the present application. Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, computers, etc. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0183] like Figure 5 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 2 or the program loaded from the storage device 8 to the random access memory (RAM) 3 to implement the large model reasoning method of the aforementioned embodiment of the present application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing device 1, ROM 2 and RAM 3 are connected to each other via a bus 4. The input / output (I / O) interface 5 is also connected to the bus 4.

[0184] Typically, the following devices may be connected to the I / O interface 5: an input device 6 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 7 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 8 including, for example, a memory card, a hard disk, etc.; and a communication device 9. The communication device 9 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0185] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any large model reasoning method provided in the embodiment of the present application.

[0186] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any large model reasoning method provided in the embodiment of the present application.

[0187] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0188] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0189] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0190] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0191] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

Claims

1. A large model reasoning method, characterized in that: include: Get a problem statement for the problem to be addressed; During the chain reasoning process of the large model based on the problem statement, determining whether the current reasoning node is a key node based on the input word token of the current reasoning node; If the current inference node is not a key node, compress the original output token of the current inference node and infer subsequent nodes based on the compression processing result; wherein the original output token is a token generated by the large model during the inference process corresponding to the current inference node; The step of determining whether the current inference node is a key node based on the input word token of the current inference node includes: Obtain the attention matrix calculated by the large model based on the input token of the current inference node; Determining, based on the attention matrix, whether the attention distribution uncertainty of each token constituting the input token is within a preset limit corresponding to the input token; If the attention distribution uncertainties of at least a preset number of tokens in the input tokens are within a preset limit, the current inference node is a key node; otherwise, the current inference node is not a key node; Among them, for each token in the input token, the attention distribution uncertainty of the current token is represented by the attention entropy fusion value of the current token, wherein the attention entropy fusion value of the current token is the fusion value of the attention entropy values ​​of the current token in each attention head determined based on the obtained attention matrices.

2. The large model reasoning method according to claim 1, characterized in that: The original output token of the current inference node is compressed, including: Performing symbolic processing on the original output token to obtain a symbolic processing result; The symbolization processing result is encoded to obtain an encoding result as the compression processing result.

3. The large model reasoning method according to claim 2, characterized in that: The original output token is symbolized to obtain a symbolized processing result, including: Performing token category determination on the original output token; Extracting tokens of a preset token category from the original output tokens in order based on the determination result; According to the symbol generation rule corresponding to the preset token category, the extracted token is symbolically converted to generate the symbolized processing result.

4. The large model inference method according to claim 3, characterized in that: The preset token category includes: at least one of a numerical value category, a verb category, and an entity category; The symbol generation rule corresponding to the numerical class includes: converting the numerical class token into a first preset symbol and associating the numerical dimension metadata; The symbol generation rule corresponding to the verb class includes: converting the verb class token into a second preset symbol and carrying the timing code of the current token; The symbol generation rule corresponding to the entity class includes: converting the entity class token into a third preset symbol and carrying the entity identifier of the current token.

5. The large model reasoning method according to claim 2, characterized in that: The encoding process is performed on the symbolization result, including: The symbolic processing result is encoded based on a pre-built compression dictionary; the compression dictionary is a symbol-code mapping table that matches the field to which the problem to be processed belongs.

6. The large model reasoning method according to claim 1, characterized in that: Obtain the attention matrix calculated by the large model based on the input token of the current inference node, including: Obtain the attention matrix calculated based on the input token for some pre-set hidden layers in the large model.

7. The large model reasoning method according to claim 1, characterized in that: The preset limit is positively correlated with the overall attention distribution uncertainty of the input token and the complexity of the problem statement.

8. The large model inference method according to any one of claims 1 to 7, characterized in that: The process of chain reasoning based on the problem statement in the large model also includes: In the case that the current inference node is a key node, a subsequent node is inferred based on the original output token.

9. The large model inference method according to any one of claims 1 to 7, characterized in that: After inferring subsequent nodes based on the compression processing results, it also includes: Determine a reasoning path to be verified and calculate the path confidence; the reasoning path to be verified includes a parallel reasoning node, the parallel reasoning node is a parallel node of a non-critical reasoning node and the parallel reasoning node uses the compression processing result as an output token; In the case that the path confidence is lower than a preset confidence threshold, compression and restoration processing is performed on the compression processing result, and subsequent nodes are re-inferred based on the compression and restoration result.

10. The large model reasoning method according to claim 9, characterized in that: The calculating path confidence comprises: Calculate the edge confidence of each edge constituting the reasoning path to be verified, use the depth attenuation factor to correct the calculated edge confidence of each edge, and superimpose the corrected edge confidences to obtain the path confidence; wherein, each edge is composed of two reasoning nodes with a dependency relationship.

11. The large model reasoning method according to any one of claims 1 to 7, characterized in that: The loss functions used in the large model training phase include: a first loss function, a second loss function, and a third loss function; Among them, the first loss function represents the degree of difference between the problem reasoning result output by the large model and the standard reasoning result; the second loss function represents the degree of difference between the key nodes determined during the reasoning process and the actual key nodes; the third loss function represents the degree of difference between the compressed token sequence generated by the large model and the actual compressed token sequence.

12. The large model reasoning method according to claim 11, characterized in that: The large model is trained by a progressive training method; the training process of the large model includes a first training stage, a second training stage and a third training stage performed in sequence; Among them, the first training stage takes minimizing the value of the second loss function as the training goal, the second training stage takes minimizing the value of the joint loss function composed of the second loss function and the third loss function as the training goal, and the third training stage takes minimizing the value of the joint loss function composed of the first loss function, the second loss function and the third loss function as the training goal.

13. A large model reasoning device, characterized in that: include: A problem acquisition unit, used to acquire a problem statement of a problem to be processed; A key node determination unit is used to determine whether the current inference node is a key node based on the input word meta token of the current inference node during the chain reasoning process of the large model based on the problem statement, wherein the determination of whether the current inference node is a key node based on the input word meta token of the current inference node includes: obtaining an attention matrix calculated by the large model based on the input token of the current inference node; determining whether the attention distribution uncertainty of each token constituting the input token is within a preset limit corresponding to the input token based on the attention matrix; if the attention distribution uncertainty of at least a preset number of tokens in the input token is within the preset limit, then the current inference node is a key node, otherwise the current inference node is not a key node; wherein, for each token in the input token, the attention distribution uncertainty of the current token is represented by the attention entropy fusion value of the current token, wherein the attention entropy fusion value of the current token is a fusion value of the attention entropy values ​​of the current token at each attention head determined based on each acquired attention matrix; The output word unit configuration unit is used to compress the original output token of the current inference node when the current inference node is not a key node. The original output token is the token generated by the large model during the inference process corresponding to the current inference node; and the compression processing result is determined as the output token for inferring subsequent nodes.

14. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the large model reasoning method as described in any one of claims 1-12.

15. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the large model reasoning method as described in any one of claims 1 to 12 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the large model reasoning method as described in any one of claims 1 to 12 is implemented.

Images