Draft tree generation method, information processing method, device, equipment and storage medium
By introducing hierarchical confidence evaluation and lexical expansion in the draft tree generation, the problem of fixed depth and width limitations is solved, enabling dynamic adjustment and resource optimization of the draft tree, and improving the efficiency and accuracy of large language models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UC MOBILE CHINA CO LTD
- Filing Date
- 2026-03-06
- Publication Date
- 2026-07-21
AI Technical Summary
Existing large language models have fixed depth and width limitations in draft tree generation, which leads to increased computational overhead in generating low-quality lexical units and wastes validation resources for the target validation model.
By introducing the hierarchical confidence assessment of the current layer of the draft tree, the tree depth and width are dynamically adjusted, the generation of low-confidence layers is truncated, and the lexical expansion is performed to match the budget of the target validation model.
The distribution of lexical units was optimized, which avoided wasting computational resources, ensured that the generated lexical units met user needs, and improved the efficiency and accuracy of the validation model.
Smart Images
Figure CN122433682A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of language model processing technology, specifically to a draft tree generation method, information processing method, apparatus, device, and storage medium. Background Technology
[0002] In existing large language model inference acceleration frameworks, a number of candidate lexical sequences are first generated by the draft model, and then the target verification model performs parallel verification on these candidate lexical sequences, which can advance multiple lexical sequences at once. The existing draft model generates multiple candidate lexical sequences using a draft tree with a tree structure. This draft tree employs a static structure (fixed depth, fixed width), meaning that regardless of changes in the current input context, the draft model consistently generates the draft tree according to this fixed static structure and submits it to the target validation model for verification. This results in low-quality lexical units being generated when the draft model has high uncertainty at the current step, as further expansion leads to lower-quality lexical units. The generation of these low-quality lexical units increases the overhead of the draft model and makes them difficult for the target validation model to accept. When the draft model's depth is fixed, further exploration can still yield high-quality lexical units, but the fixed depth limits further exploration of the draft tree, thus missing the generation of high-quality lexical units. Furthermore, the validation budget of the target validation model is usually fixed, and the total number of lexical units in the existing draft tree cannot guarantee that it will exactly fill the validation budget of the target validation model, leading to a waste of computational resources for the target validation model.
[0003] Therefore, a new method for generating draft trees is needed to solve the above-mentioned technical problems. Summary of the Invention
[0004] This disclosure provides a new technical solution for a draft tree generation method.
[0005] According to a first aspect of this disclosure, a method for generating a draft tree is provided, the method comprising: During the construction of a draft tree for user prompt words using the draft model, an event is completed based on the draft model to generate the current layer lexicon for constructing the draft tree, and the level confidence of the current layer lexicon is determined, wherein the level confidence represents the degree of matching between the lexicon sequence of the growth path where the current layer lexicon is located and the user prompt word; If the level confidence of the current layer word is less than the set confidence threshold, the draft model is triggered to truncate the generation of the next layer word, thus obtaining the first draft tree for the user prompt word. If the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, the lexical units in the current layer are expanded to obtain a second draft tree for verification. The total number of lexical units in the second draft tree is consistent with the lexical verification budget. The target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
[0006] Optionally, determining the hierarchical confidence of the current layer terminology includes: Based on each word in the current layer, the cumulative confidence score of the node where the corresponding word is located is obtained. The cumulative confidence score of the node where the corresponding word is located is obtained by accumulating the confidence scores of all nodes in the growth path where the corresponding word is located. The confidence score of each node is determined based on the prediction probability of the word located in the corresponding node. The prediction probability of the word is determined when the draft model generates the corresponding word. Based on the cumulative confidence scores of the nodes where each word element in the current layer is located, select the maximum cumulative confidence score; The hierarchical confidence of the current layer terminology is determined based on the maximum cumulative confidence score.
[0007] Optionally, before performing lexical expansion on the current layer lexical units, the method further includes: Based on the total number of tokens in the first draft tree and the token validation budget, the current remaining token validation count for the target validation model is determined, where, Lexical expansion of the current layer lexical units includes: Based on the number of currently remaining word tokens verified, the word tokens of the current layer are expanded.
[0008] Optionally, the step of expanding the lexical units of the current layer includes: Get all candidate lexical units in the current layer; Determine the predicted probability of each candidate word, wherein the predicted probability is determined when the draft model generates the corresponding word; Based on the predicted probability of each candidate word, expanded words are selected, wherein the expanded words are high-probability candidate words with the same number of verifications as the current remaining words; The expanded lexical units are expanded to the corresponding nodes of the current layer.
[0009] Optionally, after expanding the lexical units of the current layer, the method further includes: Generate a parent node index list, wherein the parent node index list records the parent node index information of the node where each word element is located; The second draft tree contains multiple candidate sequences and the parent node index list, which are then sent to the target verification model.
[0010] Optionally, the method further includes: If the level confidence of the current layer lexical is greater than or equal to the set confidence threshold, the next layer lexical is generated based on the draft model to construct the draft tree.
[0011] According to a second aspect of this disclosure, an information processing method based on a target verification model is provided, comprising: Based on the draft model, a draft tree containing multiple candidate sequences is generated using any of the methods described in the first aspect; The draft tree is input into the target validation model to obtain the target sequence, wherein the target sequence is obtained by filtering from the plurality of candidate sequences.
[0012] According to a third aspect of this disclosure, an information processing method is provided, comprising: In response to user input of a prompt, the draft model is invoked to construct a draft tree based on the prompt; When the draft model completes the generation of the current layer lexical unit for constructing the draft tree, the level confidence of the current layer lexical unit is determined, wherein the level confidence represents the degree of matching between the lexical unit sequence of the growth path where the current layer lexical unit is located and the user prompt word; If the level confidence of the current layer word is less than the set confidence threshold, the draft model is triggered to truncate the generation of the next layer word, thus obtaining the first draft tree for the prompt word. If the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, the lexical units in the current layer are expanded to obtain a second draft tree for verification, wherein the total number of lexical units in the second draft tree is consistent with the lexical verification budget. The second draft tree is input into the target validation model to obtain the target sequence, wherein the target sequence is obtained by filtering multiple candidate sequences included in the second draft tree; Output the target sequence corresponding to the prompt word.
[0013] According to a fourth aspect of this disclosure, a draft tree generation apparatus is provided, comprising: The hierarchical confidence determination module is used to determine the hierarchical confidence of the current layer lexical unit based on the event of the current layer lexical unit being generated for constructing the draft tree by the draft model during the construction of the draft tree for the user prompt word by calling the draft model. The hierarchical confidence represents the degree of matching between the lexical unit sequence of the growth path where the current layer lexical unit is located and the user prompt word. The truncation module is used to trigger the draft model to truncate the generation of the next layer of words when the level confidence of the current layer word is less than a set confidence threshold, so as to obtain the first draft tree for the user prompt word. The lexical expansion module expands the lexical units of the current layer when the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, thereby obtaining a second draft tree for verification. The total number of lexical units in the second draft tree is consistent with the lexical verification budget. The target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
[0014] According to a fifth aspect of this disclosure, an electronic device is provided, including a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to perform the method according to either the first or second aspect.
[0015] According to a sixth aspect of this disclosure, a non-volatile computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described according to a first, second, or third aspect of this disclosure.
[0016] One beneficial effect of this disclosure is that, by introducing the hierarchical confidence of the current layer of the constructed draft tree, the embodiments of this disclosure evaluate whether the draft tree is worthwhile to continue exploring deeper layers. If it is determined that the hierarchical confidence of the current layer of the word is less than a set confidence threshold, the draft model is triggered to truncate the generation of the next layer of words, avoiding the investment of the target verification model's word verification budget in low-return depth directions. Furthermore, if the total number of words in the first draft tree is less than the word verification budget of the target verification model, word expansion is performed on the current layer of words, maximizing the remaining computing power of the target verification model for word expansion, ensuring that the total number of words in the draft tree fills the verification budget of the target verification model, and avoiding the waste of the target verification model's computing resources. In addition, the draft tree involved in this disclosure is no longer a fixed depth and fixed width, but a non-uniform, dynamically adjustable tree width and depth elastic tree, realizing the optimized allocation of word distribution, ensuring that every word submitted to the target verification model is a candidate word that meets the user's needs under the current probability distribution.
[0017] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the embodiments of the present disclosure.
[0019] Figure 1 A flowchart illustrating a draft tree generation method according to some embodiments is shown; Figure 2 A flowchart of an information processing method based on a target verification model according to some embodiments is shown; Figure 3 A flowchart of an information processing method based on a target verification model according to some embodiments is shown; Figure 4 A flowchart of an information processing method according to some embodiments is shown; Figure 5 A schematic diagram of the composition structure of a draft tree generation apparatus according to some embodiments is shown; Figure 6 A schematic diagram of the composition structure of an information processing apparatus based on a target verification model according to some embodiments is shown; Figure 7 A schematic diagram of the composition structure of an information processing apparatus according to some embodiments is shown; Figure 8 A hardware configuration diagram of an electronic device according to some embodiments is shown; Figure 9 A hardware configuration diagram of an electronic device according to some other embodiments is shown. Detailed Implementation
[0020] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the invention.
[0021] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.
[0022] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0023] In all the examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0024] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0025] This disclosure provides a draft tree generation method. By introducing the level confidence of the current layer words in the constructed draft tree, the method evaluates whether the draft tree is worthwhile to explore deeper layers. If the level confidence of the current layer words is less than a set confidence threshold, the draft model is triggered to truncate the generation of the next layer words, avoiding the allocation of the target validation model's word validation budget to low-return depth directions. Furthermore, if the total number of words in the first draft tree is less than the target validation model's word validation budget, the current layer words are expanded, maximizing the remaining computing power of the target validation model for word expansion. This ensures that the total number of words in the draft tree fills the target validation model's validation budget, avoiding waste of the target validation model's computing resources. In addition, the draft tree involved in this disclosure is no longer a fixed depth and fixed width, but a non-uniform, dynamically adjustable tree with adjustable width and depth, achieving optimized allocation of word distribution and ensuring that every word submitted to the target validation model is a candidate word that meets the user's needs under the current probability distribution.
[0026] <First Embodiment> Figure 1 A flowchart of a draft tree generation method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes steps S110 to S130.
[0027] Step S110: During the construction of the draft tree for the user prompt word by calling the draft model, the event of generating the current layer lexicon for constructing the draft tree is completed based on the draft model, and the level confidence of the current layer lexicon is determined. The level confidence represents the degree of matching between the lexicon sequence of the growth path where the current layer lexicon is located and the user prompt word.
[0028] The draft model generates multiple candidate word sequences based on user prompts. These candidate word sequences are distributed in a tree structure to construct a draft tree. Each node in the draft tree contains one word. The draft tree is generated layer by layer, starting from the root node and generating words for each layer.
[0029] After generating the current layer lexicons for constructing the draft tree based on the draft model, an evaluation is performed to determine whether the draft tree is worthwhile to explore deeper layers. This evaluation is conducted using the hierarchical confidence of the current layer lexicons.
[0030] In some embodiments, determining the hierarchical confidence of the current layer term includes steps S111 to S112.
[0031] Step S111: Based on each word in the current layer, obtain the cumulative confidence score of the node where the corresponding word is located. The cumulative confidence score of the node where the corresponding word is located is obtained by accumulating the confidence scores of all nodes in the growth path of the corresponding word. The confidence score of each node is determined based on the prediction probability of the word located in the corresponding node. The prediction probability of the word is determined when the draft model generates the corresponding word.
[0032] During the process of building the draft tree layer by layer, the draft model determines the predicted probability of each word located at each node in that layer based on a defined function when generating a new layer of words. The predicted probability of a word represents the degree of matching between the word and the user's suggestion.
[0033] The cumulative confidence score of the node containing the corresponding word element represents the confidence level of the growth path containing that word element. A high cumulative confidence score indicates that the growth path containing the corresponding word element is worth further exploration. A low cumulative confidence score indicates that the growth path containing the corresponding word element is not worth further exploration.
[0034] The confidence score for each node is the logarithm of the predicted probability of the word located at that node. The confidence score Q for each node is determined based on the following formula.
[0035] Where i represents the growth path of the corresponding node, and d represents the level of the corresponding node in the draft tree. For the word elements located in the corresponding node, This refers to the predicted probabilities determined when generating lexical units based on the draft model.
[0036] The cumulative confidence score of the node containing the corresponding word is determined based on the following formula. ,
[0037] in, This is the cumulative confidence score of the parent node of the node containing the corresponding word element.
[0038] Step S112: Select the maximum cumulative confidence score based on the cumulative confidence scores of the nodes where each word element in the current layer is located.
[0039] Step S113: Determine the hierarchical confidence of the current layer word based on the maximum cumulative confidence score.
[0040] In this embodiment, the cumulative confidence scores of the nodes containing each word in the current layer are input into a set function to obtain the hierarchical confidence score of the word in the current layer. The hierarchical confidence score of the word in the current layer is determined based on the following formula. ,
[0041] in, It is the maximum cumulative confidence score among the cumulative confidence scores of the nodes where each word in the current layer is located, and exp() is the setting function.
[0042] Step S120: If the level confidence of the current layer word is less than the set confidence threshold, trigger the draft model to truncate the generation of the next layer word, and obtain the first draft tree for the user prompt word.
[0043] If the confidence level of the current layer word is less than the set confidence threshold, it is determined that the uncertainty of the draft model's prediction of the current layer word in the draft tree is high. The quality of the next layer word generated based on the draft model is low, and the probability of it being verified by the target verification model is low. Therefore, there is no need to invest the word verification budget of the target verification model in the low-return depth direction. Thus, it is determined that the draft tree does not need to explore deeper and trigger the draft model to truncate the generation of the next layer word.
[0044] If the level confidence of the current layer lexical unit is greater than or equal to the set confidence threshold, it is determined that the draft model has high certainty in predicting the current layer lexical unit of the draft tree, and the quality of the next layer lexical unit generated based on the draft model is high. Therefore, it is determined that the draft tree is worth exploring further, and the next layer lexical unit used to construct the draft tree can be generated based on the draft model. The event of generating the next layer lexical unit for constructing the draft tree based on the draft model is executed, followed by the operation of determining the level confidence of the next layer lexical unit.
[0045] Step S130: If the total number of tokens in the first draft tree is less than the token verification budget of the target verification model, the tokens in the current layer are expanded to obtain a second draft tree for verification. The total number of tokens in the second draft tree is consistent with the token verification budget. The target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
[0046] The token validation budget for the target validation model is set to a fixed value to keep the validation computation of the target validation model running at high efficiency.
[0047] While truncating the generation of the next layer of tokens in the draft model can prevent the draft tree from growing ineffectively in the depth direction, it may result in the total number of tokens in the draft tree being far less than the token validation budget of the target validation model, leading to a waste of computational resources. Therefore, token expansion is performed on the tokens of the current layer. This ensures that the remaining computing power of the target validation model is maximized for token expansion, guaranteeing that the total number of tokens in the draft tree fills the validation budget of the target validation model, thus avoiding a waste of computational resources for the target validation model.
[0048] In some embodiments, before expanding the lexical units of the current layer, the method further includes: determining the current remaining lexical unit verification count of the target validation model based on the total number of lexical units in the first draft tree and the lexical unit validation budget. The current remaining lexical unit verification count of the target validation model is determined based on the following calculation formula. ,
[0049] in, Validate the budget quantity for lexical units. This represents the total number of lexical units in the first draft tree.
[0050] In this embodiment, based on the number of remaining word verifications, word expansion is performed on the current layer words, specifically including: obtaining all candidate words in the current layer; determining the prediction probability of each candidate word, wherein the prediction probability is determined when the draft model generates the corresponding word; selecting expanded words based on the prediction probability of each candidate word, wherein the expanded words are high-probability candidate words with the same number of remaining word verifications as the current words; and expanding the expanded words to the corresponding nodes in the current layer. The candidate lexical units in the current layer do not include lexical units located in the nodes of the current layer.
[0051] Based on the predicted probabilities of each candidate word, all candidate words are sorted in ascending or descending order to obtain a sorted candidate word sequence. From the ascending-sorted candidate word sequence, the candidate word with the highest ranking and the same number of verified words as the current remaining word is selected as the expanded word. Alternatively, from the descending-sorted candidate word sequence, the candidate word with the lowest ranking and the same number of verified words as the current remaining word is selected as the expanded word.
[0052] In some embodiments, after expanding the lexical units of the current layer, the method further includes: generating a parent node index list, wherein the parent node index list records the parent node index information of the node where each lexical unit is located; and sending the multiple candidate sequences contained in the second draft tree and the parent node index list to the target validation model.
[0053] Before validating multiple candidate sequences contained in the second draft tree, the target validation model constructs a tree attention mask using a list of parent node indices. The tree attention mask can maintain the causal dependencies within each candidate sequence and prevent information leakage between branches containing different candidate sequences, thus ensuring the accuracy of the target validation model's validation.
[0054] <Second Embodiment> Figure 2 A flowchart illustrating an information processing method based on a target verification model according to an embodiment of the present disclosure is shown. Figure 2 As shown, the method includes steps S210 to S220.
[0055] Step S210: Based on the draft model, generate a draft tree containing multiple candidate sequences using the method provided in any of the above embodiments.
[0056] The draft tree generated in this embodiment is no longer a fixed depth and fixed width, but a non-uniform, dynamically adjustable tree with adjustable width and depth, thereby optimizing the distribution of lexical units and improving the accuracy of the lexical sequences provided for the target validation model.
[0057] Step S220: Input the draft tree into the target validation model to obtain the target sequence, wherein the target sequence is obtained by screening multiple candidate sequences.
[0058] The target validation model determines the degree of matching between each candidate sequence and the user prompt word. Based on the matching degree of each candidate sequence, the candidate sequence with the highest matching degree is selected as the target sequence and displayed to the user. The sequences involved in this embodiment are the word sequences of each growth path.
[0059] In some embodiments, before validating multiple candidate sequences contained in the draft tree, the target validation model constructs a tree attention mask using a list of parent node indices. If node v is identified as the parent node of node u through the list of parent node indices, A[u, v] = 1 is set; otherwise, it is set to 0. This maintains the causal dependencies within each candidate sequence during candidate sequence validation, while preventing information leakage between branches of different candidate sequences, thus ensuring the accuracy of the target validation model's validation.
[0060] <Third Embodiment> Figure 3 A flowchart of a draft tree generation method according to an embodiment of the present disclosure is shown. Figure 3 As shown, the method includes steps S310 to S380.
[0061] Step S310: During the process of calling the draft model to construct a draft tree for the user prompt words, the current layer lexical units used to construct the draft tree are generated based on the draft model.
[0062] Step S320: Based on each word in the current layer, obtain the cumulative confidence score of the node where the corresponding word is located. Based on the cumulative confidence scores of the nodes where each word in the current layer is located, select the maximum cumulative confidence score and determine the level confidence of the word in the current layer based on the maximum cumulative confidence score.
[0063] Step S330: Determine whether the level confidence of the current layer word is less than the set confidence threshold.
[0064] If the result of step S330 is yes, step S340 is executed to trigger the generation of the next layer of lexical units in the draft model, thereby obtaining the first draft tree for the user prompt words.
[0065] Step S350: Determine the current remaining number of tokens to be verified in the target verification model based on the total number of tokens in the first draft tree and the token verification budget.
[0066] Step S360: If the total number of lexical units in the first draft tree is less than the number of lexical units to be verified in the target verification model, obtain all candidate lexical units in the current layer, determine the prediction probability of each candidate lexical unit, select the candidate lexical unit with a high probability that is equal to the number of remaining lexical units to be verified, and use it as an expanded lexical unit. Expand the expanded lexical unit to the corresponding node in the current layer to obtain the second draft tree.
[0067] Step S370: Send the list of multiple candidate sequences and parent node indices contained in the second draft tree to the target validation model.
[0068] Step S380: The target validation model validates multiple candidate sequences and outputs the target sequence.
[0069] If the result of step S330 is negative, return to step S310 to allow the draft model to generate the next level of lexical units for building the draft tree.
[0070] <Fourth Embodiment> Figure 4 A flowchart of an information processing method according to an embodiment of this disclosure is shown. This method can be applied to an application (APP). The APP can be a client-side APP or a web-based APP. Figure 4 As shown, the method includes steps S410 to S460.
[0071] Step S410: In response to the user inputting a prompt word, the draft model is invoked to construct a draft tree for the prompt word.
[0072] Prompts can be entered either through the text input box on the app interface or through the voice input control on the app interface. When inputting through the voice input control, the received voice signal is converted into text and used as the prompt.
[0073] Step S420: After the draft model has completed the generation of the current layer lexicon for constructing the draft tree, determine the level confidence of the current layer lexicon, where the level confidence represents the degree of matching between the lexicon sequence in the growth path of the current layer lexicon and the prompt word.
[0074] Step S430: If the level confidence of the current layer word is less than the set confidence threshold, trigger the draft model to truncate the generation of the next layer word, and obtain the first draft tree for the prompt word.
[0075] Step S440: If the total number of lexical units in the first draft tree is less than the number of lexical units in the target validation model's validation budget, the lexical units in the current layer are expanded to obtain a second draft tree for validation, wherein the total number of lexical units in the second draft tree is consistent with the number of lexical units in the validation budget.
[0076] Step S450: Input the second draft tree into the target validation model to obtain the target sequence, wherein the target sequence is obtained by filtering multiple candidate sequences included in the second draft tree.
[0077] In this embodiment, the specific operations of steps S420 to S450 can be referred to the above embodiment, and will not be elaborated further here.
[0078] Step S460: Output the target sequence corresponding to the prompt word.
[0079] The first response to the query is displayed in the settings area of the APP interface and shown to the user.
[0080] <Device Embodiment> This embodiment provides a draft tree generation device. Figure 5 A schematic diagram of the composition structure of a draft tree generation apparatus according to an embodiment of the present disclosure is shown. Figure 5 As shown, the draft tree generation device includes a hierarchy confidence determination module 510, a truncation module 520, and a lexical expansion module 530.
[0081] The hierarchical confidence determination module 510 is used to determine the hierarchical confidence of the current layer lexical unit based on the event of generating the current layer lexical unit for constructing the draft tree during the process of calling the draft model to construct the draft tree for the user prompt word. The hierarchical confidence represents the degree of matching between the lexical unit sequence of the growth path where the current layer lexical unit is located and the user prompt word.
[0082] The truncation module 520 is used to trigger the draft model to truncate the generation of the next layer of words when the level confidence of the current layer word is less than the set confidence threshold, so as to obtain the first draft tree for the user prompt word.
[0083] The lexical expansion module 530 is used to expand the lexical units of the current layer when the total number of lexical units in the first draft tree is less than the number of lexical unit verification budgets of the target verification model, so as to obtain a second draft tree for verification. The total number of lexical units in the second draft tree is consistent with the number of lexical unit verification budgets, and the target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
[0084] In some embodiments, the hierarchical confidence determination module 510 is used to obtain the cumulative confidence score of the node where the corresponding word is located based on each word in the current layer word, wherein the cumulative confidence score of the node where the corresponding word is located is obtained by accumulating the confidence scores of all nodes in the growth path of the corresponding word, and the confidence score of each node is determined based on the prediction probability of the word located in the corresponding node, the prediction probability of the word is determined when the draft model generates the corresponding word; the maximum cumulative confidence score is selected according to the cumulative confidence scores of the nodes where each word in the current layer is located; and the hierarchical confidence of the word in the current layer is determined according to the maximum cumulative confidence score.
[0085] In some embodiments, the apparatus further includes a module for determining the number of remaining token verifications.
[0086] The Remaining Token Validation Count Determination Module is used to determine the current remaining token validation count of the target validation model based on the total number of tokens in the first draft tree and the token validation budget.
[0087] In this embodiment, the lexical expansion module 530 is used to expand the lexical units of the current layer according to the number of remaining lexical unit verifications.
[0088] In some embodiments, the lexical expansion module 530 is used to obtain all candidate lexical units in the current layer; determine the prediction probability of each candidate lexical unit, wherein the prediction probability is determined when the draft model generates the corresponding lexical unit; select expanded lexical units according to the prediction probability of each candidate lexical unit, wherein the expanded lexical units are high-probability candidate lexical units with the same number of validations as the current remaining lexical units; and expand the expanded lexical units to the corresponding nodes in the current layer.
[0089] In some embodiments, the apparatus further includes a list generation module and a sending module.
[0090] The list generation module is used to generate a list of parent node indices, which records the parent node index information of the node where each word element is located.
[0091] The sending module is used to send the multiple candidate sequences and parent node index list contained in the second draft tree to the target validation model.
[0092] In some embodiments, if the level confidence of the current layer lexical is greater than or equal to a set confidence threshold, the next layer lexical for constructing the draft tree is generated based on the draft model. The level confidence determination module 510 is used to determine the level confidence of the next layer lexical after the event of generating the next layer lexical for constructing the draft tree based on the draft model is completed.
[0093] This embodiment provides an information processing device based on a target verification model. Figure 6A schematic diagram of the composition structure of an information processing apparatus based on a target verification model according to an embodiment of the present disclosure is shown. Figure 6 As shown, the information processing device based on the target verification model includes a draft tree generation module 610 and a target sequence determination module 620.
[0094] The draft tree generation module 610 is used to generate a draft tree containing multiple candidate sequences based on the draft model using the method provided in any of the above embodiments.
[0095] The target sequence determination module 620 is used to input the draft tree into the target verification model to obtain the target sequence, wherein the target sequence is obtained by screening multiple candidate sequences.
[0096] This embodiment provides an information processing device. Figure 7 A schematic diagram of the composition structure of an information processing apparatus according to an embodiment of the present disclosure is shown. Figure 7 As shown, the information processing device based on the target verification model includes a draft tree model calling module 710, a hierarchical confidence determination module 720, a truncation module 730, a lexical expansion module 740, a target sequence determination module 750, and an output module 760.
[0097] The draft tree model calling module 710 is used to respond to user input prompts and call the draft model to construct a draft tree based on the prompts.
[0098] The hierarchy confidence determination module 720 is used to determine the hierarchy confidence of the current layer word when the draft model has completed the generation of the current layer word for constructing the draft tree. The hierarchy confidence represents the degree of matching between the word sequence of the growth path where the current layer word is located and the user prompt word.
[0099] The truncation module 730 is used to trigger the draft model to truncate the generation of the next layer of words when the level confidence of the current layer word is less than the set confidence threshold, so as to obtain the first draft tree for the prompt word.
[0100] The lexical expansion module 740 is used to expand the lexical units of the current layer when the total number of lexical units in the first draft tree is less than the number of lexical units for the target validation model, so as to obtain a second draft tree for validation, wherein the total number of lexical units in the second draft tree is consistent with the number of lexical units for validation.
[0101] The target sequence determination module 750 is used to input the second draft tree into the target verification model to obtain the target sequence, wherein the target sequence is obtained by filtering multiple candidate sequences included in the second draft tree.
[0102] The output module 760 is used to output the target sequence corresponding to the prompt word.
[0103] <Electronic Device Examples> This disclosure also provides an electronic device for implementing any of the above method embodiments. Figure 8 A structural block diagram of an electronic device according to some embodiments is shown. This electronic device may be a server, PC, workstation, laptop, etc., and is not limited thereto.
[0104] like Figure 8 As shown, the electronic device includes a processor 810 and a memory 820 for storing executable instructions of the processor 810. The processor 810 is configured to implement the methods provided according to any embodiment of this disclosure when executing the instructions stored in the memory 820.
[0105] The processor 810 is used to execute computer instructions, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, and SSE. The memory 820 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk, etc., and is not limited thereto.
[0106] Figure 9 Structural block diagrams of an electronic device according to other embodiments are shown. For example... Figure 9 As shown, in addition to the processor 910 and memory 920, the electronic device may also include an interface device 930, a communication device 940, etc.
[0107] Interface device 930 includes, for example, a USB interface, a bus interface, a network interface, etc. Communication device 940 is capable of wired or wireless communication, and may include at least one short-range communication module, such as any module that performs short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. Communication device 940 may also include a long-range communication module, such as any module that performs WLAN, GPRS, 2G / 3G / 4G / 5G long-range communication.
[0108] This disclosure also provides a non-volatile computer-readable storage medium having computer program instructions stored thereon. When executed by a processor, the computer program instructions implement the methods provided in any of the above embodiments.
[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0110] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.
[0111] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.
[0113] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages (such as Smalltalk, C++, etc.) and conventional procedural programming languages (such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.
[0114] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0115] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.
[0118] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.
Claims
1. A method for generating a draft tree, characterized in that, include: During the construction of a draft tree for user prompt words using the draft model, an event is completed based on the draft model to generate the current layer lexicon for constructing the draft tree, and the level confidence of the current layer lexicon is determined, wherein the level confidence represents the degree of matching between the lexicon sequence of the growth path where the current layer lexicon is located and the user prompt word; If the level confidence of the current layer word is less than the set confidence threshold, the draft model is triggered to truncate the generation of the next layer word, thus obtaining the first draft tree for the user prompt word. If the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, the lexical units in the current layer are expanded to obtain a second draft tree for verification. The total number of lexical units in the second draft tree is consistent with the lexical verification budget. The target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
2. The method according to claim 1, characterized in that, Determining the hierarchical confidence of the current layer terminology includes: Based on each word in the current layer, the cumulative confidence score of the node where the corresponding word is located is obtained. The cumulative confidence score of the node where the corresponding word is located is obtained by accumulating the confidence scores of all nodes in the growth path where the corresponding word is located. The confidence score of each node is determined based on the prediction probability of the word located in the corresponding node. The prediction probability of the word is determined when the draft model generates the corresponding word. Based on the cumulative confidence scores of the nodes where each word element in the current layer is located, select the maximum cumulative confidence score; The hierarchical confidence of the current layer terminology is determined based on the maximum cumulative confidence score.
3. The method according to claim 1, characterized in that, Before performing lexical expansion on the current layer lexical units, the method further includes: Based on the total number of tokens in the first draft tree and the token validation budget, the current remaining token validation count for the target validation model is determined, where, Lexical expansion of the current layer lexical units includes: Based on the number of currently remaining word tokens verified, the word tokens of the current layer are expanded.
4. The method according to claim 3, characterized in that, The terminology expansion of the current layer includes: Get all candidate lexical units in the current layer; Determine the predicted probability of each candidate word, wherein the predicted probability is determined when the draft model generates the corresponding word; Based on the predicted probability of each candidate word, expanded words are selected, wherein the expanded words are high-probability candidate words with the same number of verifications as the current remaining words; The expanded lexical units are expanded to the corresponding nodes of the current layer.
5. The method according to claim 1, characterized in that, After expanding the lexical units of the current layer, the method further includes: Generate a parent node index list, wherein the parent node index list records the parent node index information of the node where each word element is located; The second draft tree contains multiple candidate sequences and the parent node index list, which are then sent to the target verification model.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: If the level confidence of the current layer lexical is greater than or equal to the set confidence threshold, the next layer lexical is generated based on the draft model to construct the draft tree.
7. An information processing method based on a target verification model, characterized in that, include: Based on the draft model, a draft tree containing multiple candidate sequences is generated using the method described in any one of claims 1-6; The draft tree is input into the target validation model to obtain the target sequence, wherein the target sequence is obtained by filtering from the plurality of candidate sequences.
8. An information processing method, characterized in that, include: In response to user input of a prompt, the draft model is invoked to construct a draft tree based on the prompt; When the draft model completes the generation of the current layer lexical unit for constructing the draft tree, the level confidence of the current layer lexical unit is determined, wherein the level confidence represents the degree of matching between the lexical unit sequence of the growth path where the current layer lexical unit is located and the user prompt word; If the level confidence of the current layer word is less than the set confidence threshold, the draft model is triggered to truncate the generation of the next layer word, thus obtaining the first draft tree for the prompt word. If the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, the lexical units in the current layer are expanded to obtain a second draft tree for verification, wherein the total number of lexical units in the second draft tree is consistent with the lexical verification budget. The second draft tree is input into the target validation model to obtain the target sequence, wherein the target sequence is obtained by filtering multiple candidate sequences included in the second draft tree; Output the target sequence corresponding to the prompt word.
9. A draft tree generation device, characterized in that, include: The hierarchical confidence determination module is used to determine the hierarchical confidence of the current layer lexical unit based on the event of the current layer lexical unit being generated for constructing the draft tree by the draft model during the construction of the draft tree for the user prompt word by calling the draft model. The hierarchical confidence represents the degree of matching between the lexical unit sequence of the growth path where the current layer lexical unit is located and the user prompt word. The truncation module is used to trigger the draft model to truncate the generation of the next layer of words when the level confidence of the current layer word is less than a set confidence threshold, so as to obtain the first draft tree for the user prompt word. The lexical expansion module expands the lexical units of the current layer when the total number of lexical units in the first draft tree is less than the lexical verification budget of the target verification model, thereby obtaining a second draft tree for verification. The total number of lexical units in the second draft tree is consistent with the lexical verification budget. The target verification model is used to verify the multiple candidate sequences contained in the second draft tree.
10. An electronic device, characterized in that, include: It includes a memory and a processor, the memory storing a computer program for controlling the processor to operate in order to perform the method according to any one of claims 1 to 8.
11. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.