A task processing method, device, electronic device and storage medium
Through the strategy of encoding, decoding and processing of the pending task data of the machine learning model and dynamically expanding the token collection, the problem of low decoding speed and accuracy in the prior art is solved, and the efficiency and quality of task processing are improved.
Patent Information
- Application Number
- CN202410408574.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-07
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-04-07
AI Technical Summary
When existing machine learning models deal with complex tasks or diversified output tasks, the decoding method lacks effective management of uncertainty and considers the difference in accuracy of token prediction, resulting in inefficiency and incorrect choices, which cannot meet the efficient and high-quality Application requirements.
By obtaining the pending task data for encoding and decoding, dynamically expanding the candidate token set, determining the target prediction token based on dynamic adjustment factors, generating task result data, and flexibly selecting and adjusting candidate tokens to improve decoding speed and accuracy.
Improve the decoding speed and decoding accuracy of task processing, enhance the quality and efficiency of task processing, reduce the dependence on single token prediction, and stronger adaptability, solving the lower speed and accuracy in traditional decoding methods. The problem.
Smart Images

Figure CN118133968B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the fields of data processing, natural language processing and artificial intelligence technology, and in particular to a task processing method, device, electronic device and storage medium. Background Art
[0002] In the field of Natural Language Processing (NLP), machine learning, as a core component of artificial intelligence, is the process of non-explicit computer programs learning data experience to optimize their own algorithms and learn to handle tasks. In simple terms, a machine learning model is a function that learns the relationship between data features and labels or learns the laws within data features. Machine learning models can automatically process large amounts of data and tasks, thereby reducing the workload of manual processing and improving efficiency.
[0003] Currently, many types of machine learning models need to predict the next or a series of the most appropriate tokens from a given related data type (such as text, speech, or image, etc.) when processing tasks. For example, text generation using a large language model (LLM) is a core task that requires predicting the next or a series of the most appropriate tokens (such as words or characters) from a given context.
[0004] Machine learning models often involve the decoding process of data when processing tasks. The traditional decoding methods currently used by machine learning models, such as greedy search and beam search, rely on gradually selecting the token with the highest probability, which performs well in simple application scenarios.
[0005] In the process of realizing the present invention, the inventors found that the prior art has the following defects: The decoding methods currently used by machine models face two major defects when processing complex tasks or diversified output tasks. First, these methods lack effective management of uncertainty, especially in scenarios where multiple possible outputs need to be balanced during the prediction process. Second, they usually do not consider the differences in the accuracy of the model's predictions for each token, which may lead to inefficiency or wrong choices at key decision points, thereby affecting the overall task processing quality and reasoning speed. In applications with extremely high requirements for decoding speed, quality, and efficiency, these traditional decoding methods often cannot meet performance requirements. Summary of the invention
[0006] The embodiments of the present invention provide a task processing method, device, electronic device and storage medium, which can improve the decoding speed and decoding accuracy during task processing, thereby improving the quality and efficiency of task processing.
[0007] According to one aspect of the present invention, there is provided a task processing method, comprising:
[0008] Obtaining pending task data of the pending task, and performing encoding and decoding processing on the pending task data to obtain a current prediction token;
[0009] The current prediction token is dynamically expanded according to the candidate token dynamic expansion strategy to obtain a dynamically expanded token set; wherein the number of candidate tokens included in the dynamically expanded token set is dynamically adjusted according to the dynamic adjustment factor of the current prediction token;
[0010] Determine a target prediction token that matches the current prediction token according to the dynamically extended token set;
[0011] Generate task result data matching the task data to be processed according to the target prediction token.
[0012] According to another aspect of the present invention, there is provided a task processing device, comprising:
[0013] A current prediction token acquisition module is used to acquire the pending task data of the pending task and perform encoding and decoding processing on the pending task data to obtain the current prediction token;
[0014] A dynamic expansion token set acquisition module, used to dynamically expand the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamic expansion token set; wherein the number of candidate tokens included in the dynamic expansion token set is dynamically adjusted according to the dynamic adjustment factor of the current prediction token;
[0015] A target prediction token determination module, configured to verify each candidate token included in the dynamically extended token set, and determine a target prediction token that matches the current prediction token according to the verification result;
[0016] A task result data generation module is used to generate task result data matching the task data to be processed according to the target prediction token.
[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0018] at least one processor; and
[0019] a memory communicatively connected to the at least one processor; wherein,
[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the task processing method described in any embodiment of the present invention.
[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the task processing method described in any embodiment of the present invention when executed.
[0022] The embodiment of the present invention obtains the current prediction token by encoding and decoding the acquired task data to be processed, and dynamically expands the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamic expansion token set, thereby determining the target prediction token that matches the current prediction token according to the dynamic expansion token set, and generating task result data that matches the task data to be processed according to the target prediction token. Since the number of candidate tokens included in the dynamic expansion token set can be dynamically adjusted according to the dynamic adjustment factor of the current prediction token, and the dynamic adjustment factor can be configured according to actual needs, it is possible to achieve flexible selection and adjustment of candidate tokens, improve the adaptability of the target prediction token to the task processing requirements, reduce the dependence on single token prediction, solve the problems of low decoding speed and decoding accuracy in the decoding method of the existing model, and can improve the decoding speed and decoding accuracy when performing task processing based on the model, thereby improving the quality and efficiency of task processing based on the model.
[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0025] Figure 1 is a flowchart of a task processing method provided by Embodiment 1 of the present invention;
[0026] Figure 2 is a flowchart of a task processing method provided by Embodiment 2 of the present invention;
[0027] Figure 3 It is a flowchart of an initial task processing model provided by the second embodiment of the present invention in combination with a large language model using a dynamic expansion strategy of candidate tokens to jointly process tasks;
[0028] Figure 4 is a schematic diagram of a task processing device provided by Embodiment 3 of the present invention;
[0029] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0032] Embodiment 1
[0033] Figure 1 This is a flowchart of a task processing method provided by the first embodiment of the present invention. This embodiment is applicable to the case where the token obtained in the model decoding process is dynamically expanded according to the candidate token dynamic expansion strategy to perform task processing. The method can be executed by a task processing device, which can be implemented by software and / or hardware, and can generally be integrated in an electronic device. The electronic device can be a terminal device or a server device. As long as it can run the model and perform related task processing, the embodiment of the present invention does not limit the specific device type of the electronic device. Accordingly, Figure 1 As shown, the method includes the following operations:
[0034] S110, obtaining the pending task data of the pending task, and performing encoding and decoding processing on the pending task data to obtain a current prediction token.
[0035] Among them, the task to be processed can be any task that needs to be automatically processed by a machine learning model (hereinafter referred to as the model). The task data to be processed can be the data that needs to be input into the model and processed by the model to automatically realize task processing. The current prediction token can be the token currently predicted by the model after encoding and decoding the task data to be processed. For the model, a token, or "token", usually refers to a discrete text unit, which can be a word, punctuation, number or other language element, which is used as the basic unit for training and generating text. In NLP, tokenization technology is usually used to segment text into token sequences. Specifically, tokenization is the process of segmenting a continuous text string into discrete words or symbols.
[0036] In an optional embodiment of the present invention, the tasks to be processed may include but are not limited to text generation tasks. Exemplarily, the tasks to be processed may be automated writing, machine translation, chatbots, and any automated processing tasks for application scenarios that require efficient and high-quality text output. The tasks to be processed may be automated processing tasks in simple application scenarios, or they may be complex tasks or diversified output tasks. Complex tasks may include but are not limited to tasks that have high requirements for output results, such as solving mathematical answers, and diversified output tasks may be automated processing tasks that require the output of mind map solutions or require the output of multiple optional solutions. The embodiments of the present invention do not limit the task types of the tasks to be processed.
[0037] Optionally, the tasks to be processed may also include multi-round dialogue tasks and image-text combined processing tasks. In these tasks, the model can perform relevant processing on the task data to be processed to obtain the corresponding prediction token. Exemplarily, when the task data to be processed is voice data, the model can first convert the voice data of the task data to be processed into text data, and perform encoding and decoding processing based on the text data to obtain the current prediction token.
[0038] Accordingly, the model can be applied to specific automatic task processing scenarios, and the task data to be processed can be obtained for the reception processing task. Exemplarily, the model can be applied to a chatbot to obtain the voice, text, image or audio and video data input by the user as the task data to be processed. After the model obtains the task data to be processed of the task to be processed, it can encode and decode the task data to be processed, and preliminarily generate preliminary result data corresponding to the task data to be processed. Among them, the preliminary result data can be composed of the current prediction token. Optionally, the number of current prediction tokens can be one or more. That is, the preliminary prediction results of the model for the task data to be processed can include multiple tokens.
[0039] S120. Dynamically expand the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamically expanded token set; wherein the number of candidate tokens included in the dynamically expanded token set is dynamically adjusted according to the dynamic adjustment factor of the current prediction token.
[0040] Among them, the candidate token dynamic expansion strategy can be a strategy for dynamically expanding the candidate tokens corresponding to the current prediction token. The dynamically expanded token set can be a set consisting of candidate tokens corresponding to the current prediction token. Dynamic adjustment factors can be configured as needed, for example, may include but are not limited to various indicator factors that can affect the token matching degree, such as confidence, context consistency or fluency. The embodiment of the present invention does not limit the specific indicator factor type of the dynamic adjustment factor. The candidate token is the token that the current prediction token can replace. For example, when the current prediction token is "the", its corresponding candidate token may be "The" or "that", etc.
[0041] In an embodiment of the present invention, each current prediction token can be configured with a dynamically extended token set, and the dynamically extended token set can be used to store candidate tokens corresponding to each current prediction token. The so-called candidate token is a token that can replace the current prediction token at the position of the current prediction token. The best token at the position of the current prediction token can be selected through the candidate token, thereby ensuring the accuracy of the final task result data.
[0042] In order to improve the accuracy and efficiency of token prediction, the embodiment of the present invention adopts a dynamic expansion method to configure the candidate tokens included in the dynamic expansion token set of each current prediction token. The so-called dynamic expansion is to dynamically adjust the number of candidate tokens included in the dynamic expansion token set corresponding to the current prediction token according to the dynamic adjustment factor of the current prediction token.
[0043] Exemplarily, the number of candidate tokens included in the dynamically extended token set can be determined based on the confidence of the current predicted token; the higher the confidence, the lower the number of candidate tokens included in the dynamically extended token set can be, so as to improve the decoding speed and efficiency; the lower the confidence, the higher the number of candidate tokens included in the dynamically extended token set can be, so as to improve the decoding accuracy.
[0044] Exemplarily, the number of candidate tokens included in the dynamically extended token set can also be determined based on the fluency of the current predicted token. The higher the fluency, the lower the number of candidate tokens included in the dynamically extended token set can be, so as to improve the decoding speed and efficiency; the lower the fluency, the higher the number of candidate tokens included in the dynamically extended token set can be, so as to improve the decoding accuracy.
[0045] It can be seen that by dynamically adjusting the number of candidate tokens by referring to the dynamic adjustment factors, the relationship between the decoding speed and the output quality of the task results is effectively balanced, the dependence on the prediction of a single token is reduced, and the robustness of the task result generation process is increased. It can effectively improve the speed and efficiency of the decoding process, thereby achieving optimized performance in a variety of automated generation tasks.
[0046] S130. Determine a target prediction token that matches the current prediction token according to the dynamically extended token set.
[0047] The target prediction token is a token that can replace and modify the current prediction token.
[0048] S140, generating task result data matching the task data to be processed according to the target prediction token.
[0049] The task result data may be result data generated after the model automatically processes the task data to be processed of the task to be processed.
[0050] Correspondingly, after determining the current prediction token, the best token at the current prediction token position can be selected from the dynamically expanded token set corresponding to each current prediction token in turn as the target prediction token, and the current prediction token can be replaced by the target prediction token. The target prediction token corresponding to each current prediction token after replacement can finally generate task result data that matches the task data to be processed.
[0051] It is understandable that the target prediction token can be any candidate token in the dynamically extended token set. When the dynamically extended token set includes the current prediction token, the target prediction token can be the current prediction token itself, or it can be other candidate tokens included in the dynamically extended token set, which is not limited in the embodiment of the present invention.
[0052] For example, when the task to be processed is an automatic writing task, and the writing title of the task data to be processed is "My Dad", the task result data can be an essay about "My Dad" automatically output by the model. When the task to be processed is a chatbot dialogue task, and the text or voice input by the user is the task data of "How is the weather today", the task result data can be the specific weather conditions and warm reminders of the date queried by the user, etc., automatically output by the model.
[0053] The embodiment of the present invention obtains the current prediction token by encoding and decoding the acquired task data to be processed, and dynamically expands the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamic expansion token set, thereby determining the target prediction token that matches the current prediction token according to the dynamic expansion token set, and generating task result data that matches the task data to be processed according to the target prediction token. Since the number of candidate tokens included in the dynamic expansion token set can be dynamically adjusted according to the dynamic adjustment factor of the current prediction token, and the dynamic adjustment factor can be configured according to actual needs, it is possible to achieve flexible selection and adjustment of candidate tokens, improve the adaptability of the target prediction token to the task processing requirements, reduce the dependence on single token prediction, solve the problems of low decoding speed and decoding accuracy in the decoding method of the existing model, and can improve the decoding speed and decoding accuracy when performing task processing based on the model, thereby improving the quality and efficiency of task processing based on the model.
[0054] Embodiment 2
[0055] Figure 2 is a flowchart of a task processing method provided by the second embodiment of the present invention. This embodiment is specific based on the above embodiment. In this embodiment, multiple specific optional implementation methods are provided for dynamically expanding the current prediction token according to the candidate token dynamic expansion strategy and determining the target prediction token that the current prediction token matches. Figure 2 As shown, the method of this embodiment may include:
[0056] S210, obtaining the pending task data of the pending task, and performing encoding and decoding processing on the pending task data to obtain a current prediction token.
[0057] In an optional embodiment of the present invention, the encoding and decoding of the task data to be processed may include: encoding and decoding the task data to be processed by using an initial task processing model.
[0058] The initial task processing model may be a model that preliminarily performs encoding and decoding processing on the task data to be processed to obtain preliminary prediction tokens.
[0059] Optionally, in order to improve the decoding speed, a model with optimized performance and low computational cost can be selected as the initial task processing model, and the task data to be processed of the task to be processed is received through the initial task processing model, and the data is encoded and decoded to obtain the current prediction token.
[0060] Optionally, if the initial task processing model is an encoder-decoder structure, after the initial task processing model receives the task data to be processed, it can use the encoder to encode it to obtain an encoding result, which can be a word vector representation of the task data to be processed. Further, the encoding result is input to the decoder to decode the encoding result through the decoder to obtain a preliminary task processing result. The preliminary task processing result can be composed of one or more current prediction tokens.
[0061] Optionally, if the initial task processing model does not include an encoder structure but only includes a decoder result, the decoder of the initial task processing model can complete the encoding and decoding processes at the same time. Accordingly, the initial task processing model can directly encode and decode the acquired task data to be processed through the decoder to obtain a preliminary task processing result. The preliminary task processing result can be composed of one or more current prediction tokens.
[0062] S220. Determine a current dynamic adjustment factor of the current predicted token according to the candidate token dynamic expansion strategy.
[0063] Among them, the current dynamic adjustment factor can be a factor that is referenced for determining the candidate token for the current prediction token. Optionally, the current dynamic adjustment factor can include but is not limited to various indicator factors that can affect the token matching degree, such as confidence, context consistency or fluency. The number of the current dynamic adjustment factors can be one or more, and the embodiment of the present invention does not limit this.
[0064] Optionally, when the type of the task to be processed is different, the type of the current dynamic adjustment factor selected accordingly may also be different. Exemplarily, when the type of the task to be processed is a question answering task, the type of the current dynamic adjustment factor selected accordingly may be confidence. When the type of the task to be processed is a human-computer dialogue task, the type of the current dynamic adjustment factor selected accordingly may be context consistency and confidence.
[0065] S230. Determine the current dynamic expansion token quantity of the current prediction token according to the current dynamic adjustment factor.
[0066] Accordingly, after determining the current dynamic adjustment factor adapted to the task to be processed, the current dynamic extension token quantity of the current prediction token can be dynamically configured according to the current dynamic adjustment factor. That is, the current dynamic extension token quantity of the current prediction token can be determined in real time according to the specific situation of the current dynamic adjustment factor in the current prediction token. The current dynamic extension token quantity of the current prediction token may also be different depending on the situation of the current dynamic adjustment factor in the current prediction token.
[0067] In an optional embodiment of the present invention, the current dynamic adjustment factor may include confidence; determining the current number of dynamic extension tokens of the current prediction token based on the current dynamic adjustment factor may include: determining the current confidence of the current prediction token; determining the current preset confidence range interval that matches the current confidence based on the candidate token dynamic extension strategy; determining the current number of dynamic extension tokens that matches the current preset confidence range interval based on the candidate token dynamic extension strategy.
[0068] The current confidence level is the confidence level of the current prediction token. The current preset confidence level range may be a preset range to which the current confidence level belongs. The current number of dynamically extended tokens is the number of candidate tokens dynamically configured for the current prediction token.
[0069] Specifically, if confidence is used as a dynamic adjustment factor, the number of candidate tokens corresponding to the current prediction token can be specifically determined based on the current confidence of the current prediction token. Before determining the current number of dynamically extended tokens for the current prediction token, the preset confidence range interval can be determined first, and each preset confidence range interval can correspond to a specific number of candidate tokens. Optionally, the smaller the confidence value corresponding to the preset confidence range interval, the more candidate tokens can be set corresponding to the area; conversely, the larger the confidence value corresponding to the preset confidence range interval, the fewer candidate tokens can be set corresponding to the area. Accordingly, after determining the current preset confidence range interval that matches the current confidence of the current prediction token according to the candidate token dynamic extension strategy, the number of candidate tokens configured corresponding to the current preset confidence range interval can be used as the current number of dynamically extended tokens for the current prediction token.
[0070] In a specific example, when the preset confidence range is [0,30%], that is, when the confidence value of the token is 0-30%, the number of candidate tokens set corresponding to the token can be 8. When the preset confidence range is [30%, 60%], that is, when the confidence value of the token is 30%-60%, the number of candidate tokens set corresponding to the token can be 6. When the preset confidence range is [60%, 80%], that is, when the confidence value of the token is 60%-80%, the number of candidate tokens set corresponding to the token can be 4. When the preset confidence range is [80%, 100%], that is, when the confidence value of the token is 80%-100%, the number of candidate tokens set corresponding to the token can be 2. Correspondingly, if the current confidence of the current prediction token is 75%, the current preset confidence range interval it matches is [60%, 80%], and it can be determined that the current number of dynamic expansion tokens of the current prediction token is 4.
[0071] S240, determining the preset candidate tokens of the current predicted token, and selecting a token with the current dynamically expanded token quantity from the preset candidate tokens as the current candidate token.
[0072] The preset candidate tokens may be multiple candidate tokens determined by the model for the current prediction token. The current candidate token is also the candidate token matched by the current prediction token.
[0073] Accordingly, after determining the number of current dynamic expansion tokens corresponding to each current prediction token, the corresponding preset candidate token can be calculated for each current prediction token. Alternatively, each current prediction token can also be calculated to correspond to the preset candidate token, and then the number of current dynamic expansion tokens corresponding to each current prediction token is determined. The embodiment of the present invention does not limit the execution order of the two. After determining the preset candidate tokens of the current prediction token and the number of current dynamic expansion tokens, the tokens of the current dynamic expansion number can be selected from the preset candidate tokens as the current candidate tokens of the current prediction token.
[0074] In an optional embodiment of the present invention, the determining of the preset candidate token of the current prediction token may include: obtaining the confidence of each associated prediction token of the current prediction token; and determining the preset candidate token from each associated prediction token according to the confidence of each associated prediction token.
[0075] The associated prediction token may be a token associated with the current prediction token.
[0076] In an embodiment of the present invention, the preset candidate tokens of the current prediction token can also be determined based on the confidence. Specifically, for each current prediction token, the tokens with a strong correlation therewith can be calculated as associated prediction tokens, and the confidence of each associated prediction token can be calculated at the same time. Optionally, the confidence of each associated prediction token can be calculated by the initial task processing model to reflect the confidence level of the initial task processing model in each associated prediction token through the confidence value. Furthermore, the confidence of each associated prediction token is sorted from high to low, and a set number of associated prediction tokens with the highest confidence are selected as preset candidate tokens.
[0077] Optionally, the number of preset candidate tokens can be based on the maximum number of the current dynamic expansion tokens, and the value can be relatively set to a larger point to meet the configuration requirements of the current candidate tokens. Exemplarily, when the maximum number of the current dynamic expansion tokens is 8, the number of preset candidate tokens can be 10.
[0078] S250: Construct a dynamically extended token set of the current prediction token according to the current prediction token and the current candidate token.
[0079] Optionally, after determining the current candidate token that matches each current prediction token, the current prediction token and the current candidate token that matches it may be simultaneously added to the dynamic expansion token set corresponding to the current prediction token.
[0080] S260, verifying each candidate token included in the dynamically extended token set in parallel through a large language model, and determining a target prediction token that matches the current prediction token according to the verification result.
[0081] Among them, the large language model is also called a large language model, which refers to a deep learning model that can process text sequences by training with a large amount of relevant data (such as text data, voice data, or text and picture data, etc.), which can generate natural language text or understand the meaning of language text. Such models usually have a parameter scale of more than billions. The large language model can handle a variety of natural language tasks, such as text classification, question and answer, dialogue, etc., and is widely used. The input of the large language model is data, such as text data, voice data, or text and picture data, etc. The large language model can obtain the corresponding word vector representation by encoding the input data, and further decodes the encoded word vector, thereby automatically completing the processing of the input data and obtaining the output data corresponding to the input data. Exemplarily, text can be input to the large language model, and the large language model processes and predicts the input text, and outputs the response text corresponding to the text.
[0082] In order to further improve the decoding speed of the model, after the embodiment of the present invention obtains the dynamic extension token set by encoding and decoding the task data to be processed through the initial task processing model, the candidate tokens included in the dynamic extension token set can be verified by the large language model. Optionally, a large language model that is more complex or computationally more expensive than the initial task processing model can be used to further verify the candidate tokens included in the dynamic extension token set, and each candidate token can be verified simultaneously using a parallel verification method to increase the speed of token verification, thereby increasing the decoding speed. Furthermore, the target prediction token that the current prediction token matches can be determined based on the verification results of each candidate token.
[0083] Figure 3 1 is a flowchart of an initial task processing model provided by the second embodiment of the present invention in combination with a large language model using a dynamic expansion strategy of candidate tokens to jointly process tasks. Figure 3 As shown, M D represents the initial task processing model, M T Represents a large language model. When the initial task processing model M DAfter the input task data "Attention" is processed initially to obtain the original text "is all the", the current prediction tokens "is", "all" and "the" of the original text can be dynamically expanded using the candidate token dynamic expansion strategy to obtain a dynamic expansion token set. Among them, the dynamic expansion token set of the current prediction token "is" is empty, the candidate token corresponding to the current prediction token "all" is "All", and the candidate token corresponding to the current prediction token "the" is "that, you". After obtaining the candidate tokens corresponding to each current prediction token, the large language model M T A dynamically expanding set of tokens is validated in parallel and synchronously to determine the final token selection.
[0084] In a specific example, the above task processing method can be deployed in an online chat application that requires quick response generation, such as a mobile chat assistant. In this application scenario, the initial task processing model first generates a quick response draft, then dynamically expands the candidate tokens corresponding to each predicted token based on the confidence level, and finally a high-performance large language model completes the precise parallel verification of each candidate token to ensure the accuracy and naturalness of the response.
[0085] In an optional embodiment of the present invention, determining the target prediction token that matches the current prediction token based on the verification result may include: determining the first target prediction token that matches the current prediction token based on the verification results of each of the candidate tokens; determining the second target prediction token that matches the current prediction token from each of the candidate tokens included in the dynamically expanded token set based on a token reference evaluation indicator; and screening the target prediction token from the first target prediction token and the second target prediction token.
[0086] Among them, the first target prediction token can be a target prediction token determined according to the dynamic expansion strategy of the candidate tokens. The token reference evaluation indicator can be an evaluation indicator different from the current dynamic adjustment factor. Exemplarily, when the current dynamic adjustment factor is confidence, the token reference evaluation indicator can be an additional evaluation indicator of context consistency or fluency. The second target prediction token can be a target prediction token determined according to the token reference evaluation indicator.
[0087] In order to further enrich the reliability and accuracy of task processing, after the target prediction token that the current prediction token matches is determined based on the verification results of each candidate token, it can be used as the first target prediction token. At the same time, additional evaluation logic can be used to evaluate and compare the suitability of different tokens based on token reference evaluation indicators, such as considering contextual consistency and language fluency, and determining the second target prediction token that the current prediction token matches from the candidate tokens included in the dynamically extended token set. Accordingly, after the first target prediction token and the second target prediction token are screened out from a multi-dimensional perspective, the best token can be further screened out from the first target prediction token and the second target prediction token as the target prediction token, solving the limitation of relying solely on a single dynamic adjustment factor to screen tokens.
[0088] S270, generating task result data matching the task data to be processed according to the target prediction token.
[0089] The above technical solution uses the initial task processing model to dynamically adjust the number of additional candidate tokens according to the confidence of each token, thereby providing a wider range of candidate tokens for each predicted token position. When the confidence of the current predicted token is low, more candidate tokens are provided, which reduces the dependence on a single prediction and increases the robustness of the generation process. When the confidence of the current predicted token is high, fewer candidate tokens are provided, which can improve the decoding speed. It can be seen that by introducing a flexible candidate token adjustment mechanism, customized decoding strategies can be provided for different confidence levels, thereby achieving optimized performance in a variety of generation tasks.
[0090] When the above task processing method is used in combination with speculative decoding, it can show a significant decoding acceleration effect for large language models of different sizes. By comparing the performance of using speculative decoding independently and in combination with the above task processing method, the introduction of the candidate token dynamic expansion strategy can significantly improve the decoding speed of the generated tasks. Compared with traditional speculative decoding, when the candidate token dynamic expansion strategy is added, the decoding speed of large language models can be increased by up to 2.5 times, which can significantly reduce the delay of large language models in the reasoning process, making the processing of time-sensitive tasks more efficient. At the same time, the method of task processing based on the candidate token dynamic expansion strategy can support larger batch sizes for reasoning by reducing the number of tokens to be verified in each batch. The maximum number of tokens for parallel verification in each step can be reduced by 1 times, which can process larger data batches while maintaining high efficiency. The enhanced batch processing capability further improves the throughput of the model. In addition, by comparing the application of fixed-size extended sets with the dynamic extended token sets in the candidate token dynamic extension strategy, it is found that the use of the candidate token dynamic extension strategy can dynamically adjust the size of the extended set according to the predicted confidence. This method can increase the decoding speed from 68.4 tokens / sec to 70.9 tokens / sec, demonstrating the importance of using confidence values to adjust the size of different extended sets, thereby improving the overall decoding efficiency.
[0091] In summary, the task processing method provided by the embodiments of the present invention has significant advantages in improving the decoding speed, enhancing the batch processing capability and effectively utilizing the confidence information for dynamic adjustment. By utilizing the confidence information to optimize the selection of the candidate token set, not only the processing efficiency and quality of the generation task are improved, while maintaining high generation quality, the computational cost and reasoning delay are significantly reduced, and a more flexible and efficient decoding strategy is provided for the application of large language models.
[0092] It should be noted that any arrangement and combination of the technical features in the above embodiments also falls within the protection scope of the present invention.
[0093] Embodiment 3
[0094] Figure 4 is a schematic diagram of a task processing device provided by Embodiment 3 of the present invention, such as Figure 4 As shown, the device includes: a current prediction token acquisition module 310, a dynamic expansion token set acquisition module 320, a target prediction token determination module 330 and a task result data generation module 340, wherein:
[0095] The current prediction token acquisition module 310 is used to acquire the pending task data of the pending task and perform encoding and decoding processing on the pending task data to obtain the current prediction token;
[0096] A dynamic expansion token set acquisition module 320, configured to dynamically expand the current prediction token according to a candidate token dynamic expansion strategy to obtain a dynamic expansion token set; wherein the number of candidate tokens included in the dynamic expansion token set is dynamically adjusted according to a dynamic adjustment factor of the current prediction token;
[0097] A target prediction token determination module 330 is used to verify each candidate token included in the dynamically extended token set, and determine a target prediction token that matches the current prediction token according to the verification result;
[0098] The task result data generating module 340 is used to generate task result data matching the task data to be processed according to the target prediction token.
[0099] The embodiment of the present invention obtains the current prediction token by encoding and decoding the acquired task data to be processed, and dynamically expands the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamic expansion token set, thereby determining the target prediction token that matches the current prediction token according to the dynamic expansion token set, and generating task result data that matches the task data to be processed according to the target prediction token. Since the number of candidate tokens included in the dynamic expansion token set can be dynamically adjusted according to the dynamic adjustment factor of the current prediction token, and the dynamic adjustment factor can be configured according to actual needs, it is possible to achieve flexible selection and adjustment of candidate tokens, improve the adaptability of the target prediction token to the task processing requirements, reduce the dependence on single token prediction, solve the problems of low decoding speed and decoding accuracy in the decoding method of the existing model, and can improve the decoding speed and decoding accuracy when performing task processing based on the model, thereby improving the quality and efficiency of task processing based on the model.
[0100] Optionally, the dynamic extension token set acquisition module 320 is also used to: determine the current dynamic adjustment factor of the current prediction token based on the candidate token dynamic extension strategy; determine the current dynamic extension token quantity of the current prediction token based on the current dynamic adjustment factor; determine the preset candidate tokens of the current prediction token, and select tokens with the current dynamic extension token quantity from the preset candidate tokens as the current candidate tokens; and construct the dynamic extension token set of the current prediction token based on the current prediction token and the current candidate tokens.
[0101] Optionally, the current dynamic adjustment factor includes confidence; the dynamic extension token set acquisition module 320 is also used to: determine the current confidence of the current prediction token; determine the current preset confidence range interval that matches the current confidence according to the candidate token dynamic extension strategy; determine the current number of dynamic extension tokens that match the current preset confidence range interval according to the candidate token dynamic extension strategy.
[0102] Optionally, the dynamically extended token set acquisition module 320 is further used to: acquire the confidence of each associated prediction token of the current prediction token; and determine the preset candidate token from each associated prediction token according to the confidence of each associated prediction token.
[0103] Optionally, the current prediction token acquisition module 310 is also used to: encode and decode the task data to be processed through the initial task processing model; the target prediction token determination module 330 is also used to: perform parallel verification on each candidate token included in the dynamically extended token set through a large language model; and determine the target prediction token that matches the current prediction token based on the verification result.
[0104] Optionally, the target prediction token determination module 330 is also used to: determine the first target prediction token that matches the current prediction token based on the verification results of each of the candidate tokens; determine the second target prediction token that matches the current prediction token from each of the candidate tokens included in the dynamically extended token set based on the token reference evaluation index; and filter the target prediction token from the first target prediction token and the second target prediction token.
[0105] Optionally, the tasks to be processed include text generation tasks.
[0106] The above task processing device can execute the task processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the task processing method provided by any embodiment of the present invention.
[0107] Since the task processing device introduced above is a device that can execute the task processing method in the embodiment of the present invention, based on the task processing method introduced in the embodiment of the present invention, the technicians in this field can understand the specific implementation of the task processing device of this embodiment and its various variations, so how the task processing device implements the task processing method in the embodiment of the present invention is not described in detail here. As long as the technicians in this field implement the device used by the task processing method in the embodiment of the present invention, it belongs to the scope of protection of this application.
[0108] Embodiment 4
[0109] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0110] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0111] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0112] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a task processing method.
[0113] Optionally, the task processing method may include: obtaining the pending task data of the pending task, and performing encoding and decoding processing on the pending task data to obtain a current prediction token; dynamically expanding the current prediction token according to a candidate token dynamic expansion strategy to obtain a dynamically expanded token set; wherein the number of candidate tokens included in the dynamically expanded token set is dynamically adjusted according to a dynamic adjustment factor of the current prediction token; determining a target prediction token that matches the current prediction token according to the dynamically expanded token set; and generating task result data that matches the pending task data according to the target prediction token.
[0114] In some embodiments, the task processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the task processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the task processing method in any other appropriate manner (e.g., by means of firmware).
[0115] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0116] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0117] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0118] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0119] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0120] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
Claims
1. A task processing method, characterized in that: include: Obtaining pending task data of the pending task, and performing encoding and decoding processing on the pending task data to obtain a current prediction token; The current prediction token is dynamically expanded according to the candidate token dynamic expansion strategy to obtain a dynamically expanded token set; wherein the number of candidate tokens included in the dynamically expanded token set is dynamically adjusted according to the dynamic adjustment factor of the current prediction token; Determine a target prediction token that matches the current prediction token according to the dynamically extended token set; Generate task result data matching the task data to be processed according to the target prediction token; The dynamically expanding the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamically expanded token set includes: Determine the current dynamic adjustment factor of the current prediction token according to the candidate token dynamic expansion strategy, wherein the current dynamic adjustment factor includes confidence, context consistency or fluency, and the current dynamic adjustment factor is a factor referenced by the current prediction token to determine the candidate token. When the type of the task to be processed is different, the type of the corresponding selected current dynamic adjustment factor is also different; Determining the current dynamic expansion token quantity of the current prediction token according to the current dynamic adjustment factor; Determine a preset candidate token of the current predicted token, and select a token with the current dynamically expanded token quantity from the preset candidate tokens as the current candidate token; Constructing a dynamically extended token set of the current prediction token according to the current prediction token and the current candidate token; The encoding and decoding of the task data to be processed includes: Performing encoding and decoding processing on the task data to be processed by using an initial task processing model; The step of determining a target prediction token that matches the current prediction token according to the dynamically extended token set includes: Performing parallel verification on each candidate token included in the dynamically extended token set by using a large language model; Determine a first target prediction token that matches the current prediction token according to the verification results of each of the candidate tokens; Determining a second target prediction token that matches the current prediction token from each candidate token included in the dynamically extended token set according to a token reference evaluation index, wherein the token reference evaluation index is an evaluation index different from the current dynamic adjustment factor; The target prediction token is filtered from the first target prediction token and the second target prediction token.
2. The method according to claim 1, characterized in that: The current dynamic adjustment factor includes a confidence level; and determining the current dynamic expansion token quantity of the current prediction token according to the current dynamic adjustment factor includes: determining a current confidence level of the current prediction token; Determining a current preset confidence range interval of the current confidence match according to the candidate token dynamic expansion strategy; The number of current dynamic expansion tokens that match the current preset confidence range interval is determined according to the candidate token dynamic expansion strategy.
3. The method according to claim 2, characterized in that The step of determining a preset candidate token for the current prediction token comprises: Obtaining the confidence of each associated prediction token of the current prediction token; The preset candidate token is determined from each of the associated prediction tokens according to the confidence of each of the associated prediction tokens.
4. The method according to any one of claims 1 to 3, characterized in that: The tasks to be processed include text generation tasks.
5. A task processing device, characterized in that: include: A current prediction token acquisition module is used to acquire the pending task data of the pending task and perform encoding and decoding processing on the pending task data to obtain the current prediction token; A dynamic expansion token set acquisition module, used to dynamically expand the current prediction token according to the candidate token dynamic expansion strategy to obtain a dynamic expansion token set; wherein the number of candidate tokens included in the dynamic expansion token set is dynamically adjusted according to the dynamic adjustment factor of the current prediction token; A target prediction token determination module, configured to verify each candidate token included in the dynamically extended token set, and determine a target prediction token that matches the current prediction token according to the verification result; A task result data generating module, used for generating task result data matching the task data to be processed according to the target prediction token; Wherein, the dynamic extension token set acquisition module is also used for: determining the current dynamic adjustment factor of the current prediction token according to the candidate token dynamic extension strategy, wherein the current dynamic adjustment factor includes confidence, context consistency or fluency, and the current dynamic adjustment factor is a factor for reference to determine the candidate token of the current prediction token. When the type of task to be processed is different, the type of current dynamic adjustment factor selected accordingly is also different; determining the current dynamic extension token quantity of the current prediction token according to the current dynamic adjustment factor; determining the preset candidate token of the current prediction token, and selecting the token with the current dynamic extension token quantity from the preset candidate token as the current candidate token; constructing the dynamic extension token set of the current prediction token according to the current prediction token and the current candidate token; The current prediction token acquisition module is also used to: perform encoding and decoding processing on the task data to be processed through the initial task processing model; The target prediction token determination module is also used to: perform parallel verification on each candidate token included in the dynamically extended token set through a large language model; determine a first target prediction token that matches the current prediction token based on the verification results of each candidate token; determine a second target prediction token that matches the current prediction token from each candidate token included in the dynamically extended token set based on a token reference evaluation index, wherein the token reference evaluation index is an evaluation index different from the current dynamic adjustment factor; and screen the target prediction token from the first target prediction token and the second target prediction token.
6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the task processing method described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the task processing method described in any one of claims 1 to 4 when executed.
