Model reasoning verification method, device, electronic device and storage medium

Abstract

The present application discloses a model reasoning verification method, device, electronic device, and storage medium. The method includes: obtaining text prompt words; using a large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences; establishing a corresponding candidate sequence combination tree based on each candidate text sequence; using the large model to perform tree decoding on the candidate sequence combination tree to determine the tree decoding output sequence; based on the tree decoding output sequence, performing consistency verification on each candidate text sequence to determine the optimal text sequence from the multiple candidate text sequences. The present application can realize model reasoning verification, introduce dynamic pruning reasoning, reduce the number of model verifications, improve the efficiency of model reasoning verification, enhance the model reasoning effect, and increase the model reasoning speed, and can be widely used in the field of large model application technology.

Description

Technical Field

[0001] The present application relates to the field of large model application technology, and in particular to a model reasoning verification method, device, electronic device and storage medium. Background Art

[0002] Currently, decoding using large-scale model inference is essentially one-way decoding (decode-only), meaning the current decoding depends on the result of the previous decoding. This creates a serial structure, significantly reducing decoding efficiency. This is especially true when inferring long texts, where a single request often requires generating articles of hundreds or thousands of words. Traditional serial decoding is no longer able to meet this requirement.

[0003] When it comes to large-model reasoning, methods such as quantization and distillation are often used to make the model smaller, thereby increasing the model reasoning speed. However, this in turn leads to a decrease in the model reasoning effect. For example, methods such as flash attention that use fusion operators to alleviate IO achieve model reasoning verification by sacrificing part of the softmax accuracy. Methods such as kv cache and page attention strike a balance between cache and reading speed, alleviating redundancy in the reasoning process by sacrificing cache, thereby accelerating the model's reasoning speed. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a model reasoning verification method, device, electronic device and storage medium, which can realize model reasoning verification, improve model reasoning speed, improve model reasoning verification efficiency, and improve model reasoning effect.

[0005] On the one hand, an embodiment of the present application proposes a model reasoning verification method, the method comprising the following steps:

[0006] Get text prompt words;

[0007] Using a large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences;

[0008] According to each of the candidate text sequences, a corresponding candidate sequence combination tree is established;

[0009] Performing tree decoding on the candidate sequence combination tree using the large model to determine a tree decoding output sequence;

[0010] According to the tree decoding output sequence, consistency verification is performed on each of the candidate text sequences, and an optimal text sequence is determined from the multiple candidate text sequences.

[0011] In some embodiments, the use of a large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences specifically includes:

[0012] Dividing the text prompt word into text characters, determining the position of each text character and the unit text corresponding to each text character position;

[0013] Inputting the unit text corresponding to each of the text character positions into the multi-head attention mechanism network included in the large model, and outputting multiple inference texts corresponding to each of the unit texts and the attention accumulation probability corresponding to each of the inference texts;

[0014] Dynamically determine a sampling threshold corresponding to each unit text according to the attention accumulation probability corresponding to each inference text corresponding to each unit text, and determine a plurality of candidate inference texts corresponding to the sampling threshold from the plurality of inference texts;

[0015] According to the multiple candidate inference texts and the text character positions corresponding to each of the unit texts, text arrangement and combination are performed to generate multiple candidate text sequences.

[0016] In some embodiments, establishing a corresponding candidate sequence combination tree based on each candidate text sequence specifically includes:

[0017] Constructing the candidate sequence combination tree;

[0018] Determining the node level of each unit text in the candidate sequence combination tree according to the text character position corresponding to each unit text;

[0019] Determine, according to the node level corresponding to each unit text, the node position of each candidate inference text corresponding to each unit text in the candidate sequence combination tree, and fill each candidate inference text into the corresponding node position;

[0020] According to each of the candidate text sequences, a text association relationship between each of the candidate inference texts is determined, and according to the text association relationship, an association mark is established in the candidate sequence combination tree.

[0021] In some embodiments, performing tree decoding on the candidate sequence combination tree using the large model to determine the tree decoding output sequence specifically includes:

[0022] Input the candidate sequence combination tree into the tree-like attention mechanism network in the large model for decoding, and output multiple target inference texts and the sequence position corresponding to each target inference text;

[0023] According to the sequence position of each target inference text, each target inference text is spliced ​​to generate the tree decoding output sequence.

[0024] In some embodiments, performing consistency verification on each of the candidate text sequences according to the tree decoding output sequence and determining the optimal text sequence from the plurality of candidate text sequences specifically includes:

[0025] For each candidate text sequence, determining a consistent text sequence that matches the tree-decoding output sequence from the candidate text sequences, and determining a text length corresponding to the consistent text sequence, wherein the consistent text sequence is a sequence corresponding to a text overlap portion between the candidate text sequence and the tree-decoding output sequence;

[0026] Determining a target consistent text sequence from the consistent text sequences according to the text lengths corresponding to the consistent text sequences, wherein the target consistent text sequence has the longest text length;

[0027] The candidate text sequence to which the target consistent text sequence belongs is determined as the optimal text sequence.

[0028] In some embodiments, dynamically determining a sampling threshold corresponding to each unit text based on the cumulative attention probability corresponding to each inference text corresponding to each unit text, and determining multiple candidate inference texts corresponding to the sampling threshold from multiple inference texts specifically includes:

[0029] Obtaining a preset attention accumulation probability threshold;

[0030] Determining a plurality of target texts from the plurality of inference texts according to the cumulative attention probabilities corresponding to the inference texts corresponding to the unit texts, wherein the cumulative attention probabilities corresponding to the target texts are greater than the cumulative attention probability threshold;

[0031] Counting the number of target texts, and dynamically setting the sampling threshold according to the number of target texts, wherein the sampling threshold is less than or equal to the number of target texts;

[0032] According to the attention cumulative probability corresponding to each target text, the attention cumulative probability is sorted, and a number of the candidate inference texts are determined from the multiple target texts, wherein the number of the candidate inference texts corresponds to the sampling threshold.

[0033] In some embodiments, performing text permutation and combination based on the plurality of candidate inference texts and the text character positions corresponding to each of the unit texts to generate the plurality of candidate text sequences specifically includes:

[0034] Determining the candidate text sequence position corresponding to each unit text according to the text character position corresponding to each unit text;

[0035] The candidate inference texts corresponding to the unit texts are sequentially filled into the candidate text sequence positions corresponding to the unit texts in the form of permutations and combinations to generate the candidate text sequences.

[0036] On the other hand, an embodiment of the present application provides a model reasoning verification device, the device comprising:

[0037] The first module is used to obtain text prompt words;

[0038] The second module is used to use the large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences;

[0039] The third module is used to establish a corresponding candidate sequence combination tree according to each candidate text sequence;

[0040] A fourth module is configured to perform tree decoding on the candidate sequence combination tree using the large model to determine a tree decoding output sequence;

[0041] The fifth module is used to perform consistency verification on each candidate text sequence according to the tree decoding output sequence, and determine the optimal text sequence from the multiple candidate text sequences.

[0042] On the other hand, an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the model reasoning verification method described above when executing the computer program.

[0043] On the other hand, an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the model reasoning verification method described above.

[0044] The embodiments of the present application include at least the following beneficial effects: the present application provides a model reasoning verification method, device, electronic device and storage medium, which obtains text prompt words, uses a large model to perform dynamic pruning reasoning on the text prompt words, generates multiple candidate text sequences, establishes a corresponding candidate sequence combination tree based on each candidate text sequence, uses the large model to perform tree decoding on the candidate sequence combination tree, determines the tree decoding output sequence, and performs consistency verification on each candidate text sequence based on the tree decoding output sequence, and determines the optimal text sequence from multiple candidate text sequences. The present application can realize model reasoning verification, introduce dynamic pruning reasoning, reduce the number of model verifications, improve the efficiency of model reasoning verification, enhance the model reasoning effect, and increase the speed of model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a model reasoning verification method provided by an embodiment of the present application;

[0046] Figure 2 This is a flowchart of step S102 in the embodiment of the present application;

[0047] Figure 3 Schematic diagram of dividing text prompt words into text characters in an embodiment of the present application;

[0048] Figure 4 This is a flowchart of step S203 in the embodiment of the present application;

[0049] Figure 5 This is a flowchart of step S204 in an embodiment of the present application;

[0050] Figure 6 This is a flowchart of step S103 in the embodiment of the present application;

[0051] Figure 7 This is a flowchart of step S104 in the embodiment of the present application;

[0052] Figure 8 This is a flowchart of step S105 in the embodiment of the present application;

[0053] Figure 9 This is a schematic diagram of the structure of a model reasoning verification device provided in an embodiment of the present application;

[0054] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0056] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0057] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0059] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0060] Before explaining the embodiments of the present application in detail, some of the nouns and terms involved in the embodiments of the present application are first explained. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0061] 1) Dynamic Pruning: During the decoding process of a large model, logits need to be sampled and decoded. If the number of tokens intercepted each time is fixed, it is static pruning. If it changes according to specific circumstances, it is dynamic pruning. This patent adjusts based on the cumulative probability distribution value, so it is dynamic pruning.

[0062] 2) Logit: Logit is an important concept in deep learning. It represents the raw numerical value output by the model, before it is processed by the softmax function. These raw numerical values ​​may be interpreted as the model's confidence scores for each possible output category, but they are not necessarily normalized or have probabilistic meaning. 23 During training, the model typically uses logits as input to the loss function, which is then converted into a probability distribution through the softmax function and ultimately used to calculate the loss. During testing or inference, the softmax function is often used to convert logits into a probability distribution so that predictions can be made based on the category with the highest score.

[0063] 3) Tokenization: This refers to breaking down input text into smaller units, such as words, letters, or characters. In natural language processing, breaking text into tokens helps the model understand semantic and grammatical structure. When a prompt (a text prompt word) is sent to the large model, it is broken down into multiple tokens, a process called tokenization. Generally, for an English word, four characters represent a token.

[0064] 4) Top-k Sampling: Top-k sampling is a decoding strategy commonly used in natural language generation tasks, designed to balance the diversity and accuracy of generated text. In each decoding step, the model selects the top k words with the highest likelihood, and then randomly selects one of these words as the next generated word.

[0065] 5) Prompt: refers to a text prompt word, which is a piece of text input to the model. It is intended to guide the model to generate output with specific content or following a specific context. It can help the AI ​​model generate articles, scripts, and other content.

[0066] 6) N-gram Model: The N-gram model is an algorithm based on a statistical language model. Its basic concept is to perform a sliding window operation of size N on the text content, forming a sequence of byte segments of length N. Each byte segment is called a gram. The frequency of occurrence of all grams is counted and filtered according to a set threshold to form a key gram list, which is the vector feature space of the text.

[0067] 7) Vocabulary size: This refers to the size of the training set vocabulary. In natural language processing (NLP), vocab size typically refers to the number of distinct words or tokens in the training set. During training, the model learns embedded representations of these words so that it can recognize and utilize them when processing new text.

[0068] 8) head_logits: In deep learning, this often refers to the raw scores or output layers of a model, especially in classification problems. These scores are the linear output of the model for each class, before being processed by activation functions like softmax or sigmoid. In deep learning frameworks, head_logits is typically represented as a tensor or array, where each element corresponds to the logit value for a class. For example, for a classification problem with 10 classes, the head_logits output of the network would be an array of 10 elements.

[0069] 9) Softmax: The softmax function is primarily used in the output layer of large models for multi-classification problems. It is essentially a normalized exponential function that converts an arbitrary set of real numbers into real numbers representing a probability distribution. The softmax function maps the output values ​​of the output nodes to the range [0, 1] and ensures that the sum of all output values ​​is 1, making it interpretable as a probability distribution.

[0070] Currently, decoding using large-scale model inference is essentially one-way decoding (decode-only), meaning the current decoding depends on the result of the previous decoding. This creates a serial structure, significantly reducing decoding efficiency. This is especially true when inferring long texts, where a single request often requires generating articles of hundreds or thousands of words. Traditional serial decoding is no longer able to meet this requirement.

[0071] When it comes to large-model reasoning, methods such as quantization and distillation are often used to make the model smaller, thereby increasing the model reasoning speed. However, this in turn leads to a decrease in the model reasoning effect. For example, methods such as flash attention that use fusion operators to alleviate IO achieve model reasoning verification by sacrificing part of the softmax accuracy. Methods such as kv cache and page attention strike a balance between cache and reading speed, alleviating redundancy in the reasoning process by sacrificing cache, thereby speeding up the model reasoning speed.

[0072] Based on this, the embodiments of the present application propose a model reasoning verification method, device, electronic device and storage medium, which adopts a multi-head attention mechanism network for parallel decoding, and uses a tree-like attention mechanism network for tree-like reasoning decoding to improve the reasoning speed of large models, introduce a dynamic pruning processing mechanism, reduce the number of result verifications of large models, improve the efficiency of model reasoning verification, enhance the model reasoning effect, and realize model reasoning verification.

[0073] Reference Figure 1 , Figure 1This is an optional flowchart of a model reasoning verification method provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S105:

[0074] Step S101, obtaining text prompt words;

[0075] Step S102: Using the large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences;

[0076] Step S103: establishing a corresponding candidate sequence combination tree based on each candidate text sequence;

[0077] Step S104, using the large model to perform tree decoding on the candidate sequence combination tree to determine the tree decoding output sequence;

[0078] Step S105 : performing consistency verification on each candidate text sequence according to the tree decoding output sequence, and determining the optimal text sequence from the multiple candidate text sequences.

[0079] In some embodiments, the text prompt word is Prompt, and Prompt can be input into the big model. The big model performs reasoning and verification based on Prompt to generate an output with specific content or following a specific context, that is, the above-mentioned optimal text sequence.

[0080] In some embodiments, reference Figure 2 , Figure 2 This is an optional flowchart of step S102 in the embodiment of the present application. Step S102 may include but is not limited to steps S201 to S204:

[0081] Step S201, dividing the text prompt word into text characters, determining the position of each text character and the unit text corresponding to each text character position;

[0082] Step S202: Input the unit text corresponding to each text character position into the multi-head attention mechanism network included in the large model, and output multiple inference texts corresponding to each unit text and the cumulative attention probability corresponding to each inference text;

[0083] Step S203, dynamically determining a sampling threshold corresponding to each unit text based on the cumulative attention probability corresponding to each inference text corresponding to each unit text, and determining a plurality of candidate inference texts corresponding to the sampling threshold from the plurality of inference texts;

[0084] Step S204 , performing text permutation and combination based on the multiple candidate inference texts and text character positions corresponding to each unit text, to generate multiple candidate text sequences.

[0085] In some embodiments, the prompt is divided into multiple tokens (such as the unit text above), each token has a fixed position in the prompt (such as the text character position above), refer to Figure 3 , Figure 3 This is an optional schematic diagram of dividing text prompt words into text characters in an embodiment of the present application, wherein Prompt is divided into multiple tokens, generating token1 to token5 and text character positions 1 to 5, where token1 corresponds to text character position 1, token2 corresponds to text character position 2, and so on.

[0086] Optionally, an N-gram model may be used to divide the prompt into a sequence of byte segments of length N, where each byte segment is a token.

[0087] In some embodiments, each unit text is processed in parallel through a multi-head attention mechanism network to improve the reasoning speed of the model. Optionally, the multi-head attention mechanism network includes multiple self-attention structures, and each attention structure corresponds one-to-one to each text character position, and is used to obtain the unit text at the corresponding text character position, perform inference and prediction on the unit text, and output the corresponding prediction result. The prediction result includes multiple inference texts and the cumulative attention probability corresponding to each inference text. Optionally, the tokens at different text character positions are input into the corresponding respective attention structures, and the respective attention structures are used to perform inference and prediction, and output multiple next tokens corresponding to each token (such as the above-mentioned inference texts) and the logit corresponding to each next token (such as the above-mentioned cumulative attention probability).

[0088] The multi-head attention mechanism network is fine-tuned through the loss function to improve the inference and prediction effect of the multi-head attention mechanism network. The loss function is:

[0089]

[0090] Among them, K is the number of self-attention structures, k is the kth self-attention structure, λ is the constraint weight, which is the constraint of the K value in the multi-head attention mechanism network. Referring to the regularization term constraint, it is a penalty for excessively large K values. t is the text character position corresponding to the unit text processed by the kth self-attention structure, and L is the cross-entropy loss of the probability p of the text estimated value y at position t+k+1 of the associated text character at position t when the kth self-attention structure is used to perform inference and prediction on the unit text at text character position t.

[0091] In some embodiments, reference Figure 4 , Figure 4This is an optional flowchart of step S203 in the embodiment of the present application. Step S203 may include but is not limited to steps S301 to S304:

[0092] Step S301, obtaining a preset attention accumulation probability threshold;

[0093] Step S302, determining a plurality of target texts from the plurality of inference texts based on the cumulative attention probabilities corresponding to the inference texts corresponding to the unit texts, wherein the cumulative attention probabilities corresponding to the target texts are greater than a cumulative attention probability threshold;

[0094] Step S303: Count the number of target texts and dynamically set a sampling threshold based on the number of target texts, wherein the sampling threshold is less than or equal to the number of target texts;

[0095] Step S304 , sorting the attention cumulative probabilities corresponding to the target texts, and determining a number of candidate inference texts from the multiple target texts, wherein the number of candidate inference texts corresponds to a sampling threshold.

[0096] In some embodiments, each unit text is inferred and predicted through a multi-head attention mechanism network to obtain multiple inference texts corresponding to each unit text. Top-k sampling is performed based on the attention cumulative probability corresponding to each inference text, that is, the Top k candidate inference texts are taken out from the multiple inference texts, wherein the k value is the above-mentioned sampling threshold. The k value can be set dynamically to realize dynamic pruning processing. Specifically, based on the attention cumulative probability corresponding to each inference text, multiple target texts with attention cumulative probability greater than the attention cumulative probability threshold are determined from the multiple inference texts. The k value is set dynamically based on the number of target texts. If the number of target texts is too large, the vocab size of the large model can be obtained. According to the vocab size, the k value is set, such as setting the k value to 60% or 50% of the vocab size. It can be set by yourself. If the number of target texts is small, the number of target texts can be directly taken as the k value. Then, based on the sampling threshold k corresponding to the unit text, multiple candidate inference texts corresponding to the sampling threshold are collected.

[0097] In some embodiments, reference Figure 5 , Figure 5 This is an optional flowchart of step S204 in the embodiment of the present application. Step S204 may include but is not limited to steps S401 to S402:

[0098] Step S401, determining the candidate text sequence position corresponding to each unit text according to the text character position corresponding to each unit text;

[0099] In step S402 , the candidate inference texts corresponding to the unit texts are sequentially filled into the candidate text sequence positions corresponding to the unit texts in the form of permutations and combinations to generate candidate text sequences.

[0100] In some embodiments, multiple candidate text sequences are generated based on multiple candidate inference texts of each unit text and the corresponding text character positions. The number of candidate text sequences is: k1*k2*k3*...*k n , where k n is the number of candidate inference texts corresponding to the unit text at text character position n. For example, assuming there are token1 to token3, the candidate inference texts corresponding to token1 include token11 and token12, the candidate inference texts corresponding to token2 include token21, and the candidate inference texts corresponding to token3 include token31 and token32. The generated candidate text sequence includes: token11token21 token31, token11 token21 token32, token12 token21 token31, and token12token21 token32.

[0101] In some embodiments, reference Figure 6 , Figure 6 This is an optional flowchart of step S103 in the embodiment of the present application. Step S103 may include but is not limited to steps S501 to S504:

[0102] Step S501, constructing a candidate sequence combination tree;

[0103] Step S502, determining the node level of each unit text in the candidate sequence combination tree according to the text character position corresponding to each unit text;

[0104] Step S503: determining the node position of each candidate inference text corresponding to each unit text in the candidate sequence combination tree according to the node level corresponding to each unit text, and filling each candidate inference text into the corresponding node position;

[0105] Step S504: determining the text association relationship between the candidate inference texts based on the candidate text sequences, and establishing association tags in the candidate sequence combination tree based on the text association relationship.

[0106] In some embodiments, a candidate sequence combination tree is established based on each candidate text sequence. According to the node level corresponding to each unit text, the unit text with the front text character position has a higher node level. The node level corresponding to the unit text is determined as the node level of each candidate inference text. At this node level, the node position of each candidate inference text is determined, and each candidate inference text is filled into the corresponding node position to realize the construction of the candidate sequence combination tree. The candidate sequence combination tree is used to realize the verification of multiple candidate text sequences at one time, thereby greatly improving the efficiency of model inference verification.

[0107] In some embodiments, reference Figure 7 , Figure 7 This is an optional flowchart of step S104 in the embodiment of the present application. Step S104 may include but is not limited to steps S601 to S602:

[0108] Step S601: Input the candidate sequence combination tree into the tree-like attention mechanism network in the large model for decoding, and output multiple target inference texts and the sequence position corresponding to each target inference text;

[0109] Step S602 : splicing the target inference texts according to their sequence positions to generate a tree-shaped decoding output sequence.

[0110] In some embodiments, a tree-like attention network is used to decode the candidate sequence combination tree. Tree attention is an attention mechanism that can efficiently capture information when processing tasks with complex data structures. Its core idea is to reduce computational complexity and improve efficiency by limiting the scope of attention of each element. Tree attention organizes sequence data into a tree structure, so that each element only pays attention to other elements within a certain range of its proximity. This mechanism makes tree attention perform well when processing long sequence data, effectively capturing local information while reducing unnecessary computation.

[0111] Tree attention jumps according to the tree decoding method. For example, assuming that there is a candidate text sequence "I am a teacher a student...", when decoding to "I am", it is identified that there are two candidate branches "Iam a teacher" and "I am a student". Then when decoding to I am a, if the large model predicts student, it can skip teacher and continue decoding directly to student, which can reduce the number of decoding times. Among them, "I", "am", "a" and "stuent" are all target inference texts. Multiple target inference texts are spliced ​​to generate a tree decoding output sequence, and the tree decoding output sequence is used as the output result of the large model.

[0112] In some embodiments, reference Figure 8 , Figure 8 This is an optional flowchart of step S105 in the embodiment of the present application. Step S105 may include but is not limited to steps S701 to S703:

[0113] Step S701: For each candidate text sequence, determine a consistent text sequence that matches the tree decoding output sequence from the candidate text sequence, and determine the text length corresponding to the consistent text sequence, wherein the consistent text sequence is the sequence corresponding to the text overlap portion between the candidate text sequence and the tree decoding output sequence;

[0114] Step S702: determining a target consistent text sequence from the consistent text sequences based on the text lengths corresponding to the consistent text sequences, wherein the target consistent text sequence has the longest text length.

[0115] Step S703: Determine the candidate text sequence to which the target consistent text sequence belongs as the optimal text sequence.

[0116] In some embodiments, each candidate text sequence is verified based on the tree decoding output sequence generated by the tree attention mechanism network, and the sequence with the longest consistent text length overlapping with the tree decoding output sequence is determined from each candidate text sequence, that is, the above-mentioned target consistent text sequence.

[0117] The target consistency text sequence is determined as the optimal text sequence, and then the relevant model inference parameters of the optimal text sequence are used to update the model. For example, the head_logits corresponding to the optimal text sequence is determined, and the corresponding key and value in the multi-head self-attention mechanism network of the large model are updated. Among them, the association between key, value and head_logits is a cache mechanism in the multi-head self-attention mechanism. In order to improve computational efficiency, the results of the previously calculated self-attention layer can be cached and reused when processing subsequent parts. The result cache of the self-attention layer usually contains the following parts: key: used to find the corresponding values, value: store the actual data, head_logits: weighted and scored in the self-attention calculation.

[0118] Reference Figure 9 , Figure 9 This is an optional structural diagram of a model reasoning verification device provided in an embodiment of the present application. The device can be used to implement the above-mentioned model reasoning verification method. The device may include:

[0119] The first module is used to obtain text prompt words;

[0120] The second module is used to use the large model to perform dynamic pruning reasoning on text prompt words to generate multiple candidate text sequences;

[0121] The third module is used to establish a corresponding candidate sequence combination tree based on each candidate text sequence;

[0122] The fourth module is used to perform tree decoding on the candidate sequence combination tree using the large model to determine the tree decoding output sequence;

[0123] The fifth module is used to verify the consistency of each candidate text sequence based on the tree decoding output sequence, and determine the optimal text sequence from multiple candidate text sequences.

[0124] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0125] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned model reasoning verification method when executing the computer program. The electronic device can be any smart terminal including a tablet computer.

[0126] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0127] See also Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0128] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0129] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the model reasoning verification method of the embodiments of this application.

[0130] Input / output interface 903, used to implement information input and output;

[0131] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0132] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0133] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0134] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, implements the above-mentioned model reasoning verification method.

[0135] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0137] The embodiments of the present application provide a model reasoning verification method, device, electronic device, and storage medium. The method obtains text prompt words, uses a large model to perform dynamic pruning reasoning on the text prompt words, generates multiple candidate text sequences, establishes a corresponding candidate sequence combination tree based on each candidate text sequence, uses the large model to perform tree decoding on the candidate sequence combination tree, determines the tree decoding output sequence, and performs consistency verification on each candidate text sequence based on the tree decoding output sequence, thereby determining the optimal text sequence from multiple candidate text sequences. The embodiments of the present application can realize model reasoning verification, introduce dynamic pruning reasoning, reduce the number of model verifications, improve the efficiency of model reasoning verification, enhance the model reasoning effect, and increase the speed of model reasoning.

[0138] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0139] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0141] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0142] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0143] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0147] It should be appreciated that embodiments of the present invention may be implemented or practiced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program may be implemented in assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed application-specific integrated circuit for this purpose.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A model reasoning verification method, characterized in that: The method comprises the following steps: Get text prompt words; Using a large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences; the large model includes a multi-head attention mechanism network and a tree-like attention mechanism network; According to each of the candidate text sequences, a corresponding candidate sequence combination tree is established; Performing tree-decoding on the candidate sequence combination tree using the large model to determine a tree-decoding output sequence. Specifically, performing tree-decoding on the candidate sequence combination tree using the tree-like attention mechanism network. Performing consistency verification on each of the candidate text sequences according to the tree decoding output sequence, and determining an optimal text sequence from the plurality of candidate text sequences; The method of using the large model to dynamically prune the text prompt words to generate multiple candidate text sequences specifically includes: Dividing the text prompt word into text characters, determining the position of each text character and the unit text corresponding to each text character position; Inputting the unit text corresponding to each of the text character positions into the multi-head attention mechanism network included in the large model, and outputting multiple inference texts corresponding to each of the unit texts and the attention accumulation probability corresponding to each of the inference texts; Dynamically determine a sampling threshold corresponding to each unit text according to the cumulative attention probability corresponding to each inference text corresponding to each unit text, and determine multiple candidate inference texts corresponding to the sampling threshold from the multiple inference texts to implement dynamic pruning processing; According to the multiple candidate inference texts and the text character positions corresponding to each of the unit texts, text arrangement and combination are performed to generate multiple candidate text sequences.

2. The model reasoning verification method according to claim 1, characterized in that: The step of establishing a corresponding candidate sequence combination tree according to each candidate text sequence specifically includes: Constructing the candidate sequence combination tree; Determining the node level of each unit text in the candidate sequence combination tree according to the text character position corresponding to each unit text; Determine, according to the node level corresponding to each unit text, the node position of each candidate inference text corresponding to each unit text in the candidate sequence combination tree, and fill each candidate inference text into the corresponding node position; According to each of the candidate text sequences, a text association relationship between each of the candidate inference texts is determined, and according to the text association relationship, an association mark is established in the candidate sequence combination tree.

3. The model reasoning verification method according to claim 2, characterized in that: The performing tree decoding on the candidate sequence combination tree using the large model to determine the tree decoding output sequence specifically includes: Input the candidate sequence combination tree into the tree-like attention mechanism network in the large model for decoding, and output multiple target inference texts and the sequence position corresponding to each target inference text; According to the sequence position of each target inference text, each target inference text is spliced ​​to generate the tree decoding output sequence.

4. The model reasoning verification method according to claim 1, characterized in that: The step of performing consistency verification on each of the candidate text sequences according to the tree decoding output sequence and determining the optimal text sequence from the plurality of candidate text sequences specifically includes: For each candidate text sequence, determining a consistent text sequence that matches the tree-decoding output sequence from the candidate text sequences, and determining a text length corresponding to the consistent text sequence, wherein the consistent text sequence is a sequence corresponding to a text overlap portion between the candidate text sequence and the tree-decoding output sequence; Determining a target consistent text sequence from the consistent text sequences according to the text lengths corresponding to the consistent text sequences, wherein the target consistent text sequence has the longest text length; The candidate text sequence to which the target consistent text sequence belongs is determined as the optimal text sequence.

5. The model reasoning verification method according to claim 1, characterized in that: The method of dynamically determining a sampling threshold corresponding to each unit text according to the attention accumulation probability corresponding to each inference text corresponding to each unit text, and determining a plurality of candidate inference texts corresponding to the sampling threshold from the plurality of inference texts specifically includes: Obtaining a preset attention accumulation probability threshold; Determining a plurality of target texts from the plurality of inference texts according to the cumulative attention probabilities corresponding to the inference texts corresponding to the unit texts, wherein the cumulative attention probabilities corresponding to the target texts are greater than the cumulative attention probability threshold; Counting the number of target texts, and dynamically setting the sampling threshold according to the number of target texts, wherein the sampling threshold is less than or equal to the number of target texts; According to the attention cumulative probability corresponding to each target text, the attention cumulative probability is sorted, and a number of the candidate inference texts are determined from the multiple target texts, wherein the number of the candidate inference texts corresponds to the sampling threshold.

6. The model reasoning verification method according to claim 1, characterized in that: The step of performing text permutation and combination based on the plurality of candidate inference texts and the text character positions corresponding to the respective unit texts to generate a plurality of candidate text sequences specifically includes: Determining the candidate text sequence position corresponding to each unit text according to the text character position corresponding to each unit text; The candidate inference texts corresponding to the unit texts are sequentially filled into the candidate text sequence positions corresponding to the unit texts in the form of permutations and combinations to generate the candidate text sequences.

7. A model reasoning verification device, characterized in that: The device comprises: The first module is used to obtain text prompt words; The second module is used to use a large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences; the large model includes a multi-head attention mechanism network and a tree-like attention mechanism network; the use of the large model to perform dynamic pruning reasoning on the text prompt words to generate multiple candidate text sequences specifically includes: Dividing the text prompt word into text characters, determining the position of each text character and the unit text corresponding to each text character position; Inputting the unit text corresponding to each of the text character positions into the multi-head attention mechanism network included in the large model, and outputting multiple inference texts corresponding to each of the unit texts and the attention accumulation probability corresponding to each of the inference texts; Dynamically determine a sampling threshold corresponding to each unit text according to the cumulative attention probability corresponding to each inference text corresponding to each unit text, and determine multiple candidate inference texts corresponding to the sampling threshold from the multiple inference texts to implement dynamic pruning processing; Performing text permutation and combination based on the plurality of candidate inference texts and the text character positions corresponding to the respective unit texts to generate a plurality of candidate text sequences; The third module is used to establish a corresponding candidate sequence combination tree according to each candidate text sequence; A fourth module is configured to perform tree-decoding on the candidate sequence combination tree using the large model to determine a tree-decoding output sequence. Specifically, the candidate sequence combination tree is tree-decoded using the tree-shaped attention mechanism network. The fifth module is used to perform consistency verification on each candidate text sequence according to the tree decoding output sequence, and determine the optimal text sequence from the multiple candidate text sequences.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the model reasoning verification method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the model reasoning verification method according to any one of claims 1 to 6 is implemented.

Images