Task processing method and related apparatus

Abstract

A task processing method, which is applied to the field of artificial intelligence (AI). In the task processing method, a corresponding token is added before a prompt on the basis of a task type, and the added token can be converted into a corresponding vector, so that knowledge strongly related to a task can be added on the basis of a task input, thereby improving the precision of model output results. In addition, for different types of tasks, it is only necessary to add corresponding tokens for the different types of tasks and add into a target word vector table vectors that correspond to the tokens, so that the precision of models in processing various types of tasks can be improved, and it is not necessary to adjust weight parameters of the models themselves, thereby achieving isolation between different tasks, and thus ensuring that the models can process different types of tasks in batches.

Description

Task processing method and related device

[0001] The present application claims priority from the Chinese patent application No. 202410472838.4 filed on April 18, 2024, and entitled "A task processing method and related device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a task processing method and related device. BACKGROUND

[0003] A large language model (LLM) is a deep learning model trained based on massive text data. The large language model can not only generate natural language text, but also deeply understand the meaning of the text and process various natural language tasks, such as generating a text summary, intelligent question answering, text translation, etc.

[0004] Currently, to train a large language model for a specific field, a fine-tuning method can be used to adjust the parameters of the large language model based on a general large language model. Fine-tuning of the large language model usually involves inputting a portion of task data in a specific field into the large language model, and modifying part of the parameters of the large language model based on the output of the large language model, so that the large language model obtains a certain special ability to improve the ability of the large language model to process task data in a specific field. For example, taking the low-rank adapter (LoRA) as an example, LoRa adds a small number of trainable parameters (adapter) to each network layer of the large language model, and freezes all original parameters of the large language model, and only adjusts the weights of the adapter to improve the processing ability of the large language model for specific tasks.

[0005] However, the current fine-tuning method of the large language model is usually for a specific type of task, and the fine-tuned large language model can only improve the processing ability for a specific type of task, and cannot have good performance on other types of tasks, resulting in poor applicability of the fine-tuned large language model. SUMMARY

[0006] The present application provides a task processing method, which can improve the processing ability of the model for various types of tasks and ensure the applicability of the model.

[0007] In a first aspect, a task processing method is provided for improving the ability of a model to process a task. The task processing method comprises: first, obtaining a first prompt word, the first prompt word being used to describe a target task to be processed. The first prompt word can be in the form of text to describe the target task to be processed. For example, assuming that the target task to be processed is a question and answer task, the first prompt word can be "What is the ending of XX character in XX novel?".

[0008] Then, based on the type of the target task, at least one target word element is added to the first prompt word (for example, at least one target word element is added before the first prompt word) to obtain a second prompt word. The at least one target word element has a corresponding relationship with the type of the target task. That is, for any prompt word obtained, if the type of the task described by the prompt word is the same as the type of the target task, the at least one target word element described above can also be determined for the prompt word.

[0009] Secondly, based on a target word vector table, a vector corresponding to the second prompt word is determined to obtain a first vector sequence. The target word vector table is used to indicate the corresponding relationship between the word element and the vector, and the target word vector table includes at least one target vector corresponding to at least one target word element. The first vector sequence is actually composed of vectors corresponding to each word element in the second prompt word, and the arrangement order of the vectors in the first vector sequence corresponds to the arrangement order of the word elements in the second prompt word.

[0010] Finally, the first model processes the first vector sequence to obtain a task result output by the first model. The first model is, for example, a large language model.

[0011] In this scheme, when a prompt word describing a task is obtained, a word element corresponding to the type of the task described by the prompt word is added before the prompt word to obtain a new prompt word. When the new prompt word is processed, the newly added word element in the new prompt word can be determined to a corresponding vector in the target word vector table, so as to realize the conversion of the new prompt word into a vector sequence composed of the vector corresponding to the newly added word element and the vector corresponding to the original word element. Finally, the model processes the vector sequence to obtain a task result output by the model. By adding a corresponding word element before the prompt word based on the type of the task, and the added word element can be converted into a corresponding vector, the knowledge strongly related to the task can be added based on the task input, so as to improve the accuracy of the model output result. In addition, for different types of tasks, only the corresponding word elements for different types of tasks and the vectors corresponding to these word elements in the target word vector table need to be added, so as to improve the accuracy of the model in processing various types of tasks, and the weight parameters of the model itself do not need to be adjusted, realizing the isolation between different tasks, and ensuring that the model can process different types of tasks in batches.

[0012] In a possible implementation, the target word vector table is obtained by adding at least one target vector to an original word vector table, which is a word vector table originally used by the first model. That is, for the original first model, the first model originally uses the original word vector table to convert the input prompt word into a vector sequence. However, in the present scheme, some word units are added to the prompt word, so the original word vector table used by the first model can be expanded, so that the expanded word vector table (i.e., the target word vector table) can include vectors corresponding to the added word units in the prompt word.

[0013] In the present scheme, by expanding the word vector table originally used by the model to establish the correspondence between the newly added word units in the prompt word and the vectors, the model can be fine-tuned for the task by only expanding the original word vector table, without adjusting the weight parameters of the model itself, ensuring that various types of tasks can be adapted to the same model, and improving the applicability of the scheme.

[0014] In a possible implementation, any one of the at least one target word unit does not have a corresponding vector in the original word vector table. That is, in the original word vector table, no corresponding vector can be found for the at least one target word unit, that is, no correspondence between the at least one target word unit and the vector is established. In this way, during the model fine-tuning stage, when updating the corresponding word vector table of the first model (i.e., updating the vectors corresponding to the newly added word units in the word vector table), it is ensured that the existing vectors in the original word vector table will not be adjusted to avoid affecting the model to perform other types of tasks.

[0015] In a possible implementation, the task processing method further includes: adding at least one initial vector corresponding to the at least one target word unit in the original word vector table to obtain a target word vector table before updating. Then, based on the target word vector table before updating, a vector sequence corresponding to the training data is generated, wherein the training data is obtained by adding at least one target word unit before the preset prompt word, and the task described by the preset prompt word is of the same type as the target task. Secondly, the vector sequence corresponding to the training data is processed by the first model to obtain the output result of the first model, and based on the output result, the at least one initial vector is updated to obtain at least one target vector.

[0016] In the present scheme, by introducing the model fine-tuning process, and without adjusting the weight parameters of the model itself in the model fine-tuning process, but adjusting the vectors corresponding to the specific task type in the word vector table, the vectors that can improve the processing capability of the model for the specific task can be learned based on the task type.

[0017] In a possible implementation, the vector sequence corresponding to the training data is generated based on the target word vector table before the update, and specifically can include: determining the vector corresponding to each word piece in the training data based on the target word vector table before the update to obtain a second vector sequence; and adding random noise to the vector corresponding to at least one target word piece in the second vector sequence to obtain the vector sequence corresponding to the training data.

[0018] In this solution, by adding random noise to the vector in the vector updating process first, and then inputting the obtained vector sequence into the model for processing, the stability of the vector corresponding to the learning target word piece can be effectively improved, and an effective and accurate target vector can be finally obtained.

[0019] In a possible implementation, before the vector corresponding to the word piece in the second prompt word is determined, the word piece can be obtained by performing word segmentation processing on the second prompt word by using a word segmenter; and the vector corresponding to each word piece in the plurality of word pieces is determined based on the target word vector table to obtain a first vector sequence. The word segmenter is used to decompose a text into a plurality of predefined word pieces, so as to convert the text into a corresponding vector in the granularity of a word piece.

[0020] In a possible implementation, based on the type of the target task, at least one target word piece is added before the first prompt word, and specifically can include: determining a target word piece set having a corresponding relationship with the type of the target task from a plurality of word piece sets based on the type of the target task, the target word piece set including at least one target word piece, and in the plurality of word piece sets, different word piece sets have corresponding relationships with different task types; and based on the target word piece set, at least one target word piece is added before the first prompt word.

[0021] In this solution, by establishing the corresponding relationship between the plurality of word piece sets and the different task types, the solution can be adapted to various different types of tasks, so as to determine the word piece to be added for the prompt word describing the different types of tasks, and ensure the applicability of the solution.

[0022] In a possible implementation, the target word vector table includes the vector corresponding to the word piece in each word piece set in the plurality of word piece sets.

[0023] In a second aspect, a task processing apparatus is provided, which includes various modules for executing the task processing method in the first aspect or any possible implementation of the first aspect.

[0024] In a third aspect, a task processing apparatus is provided, which includes a processor and a memory, the memory is used to store computer instructions, when the processor executes the instructions, the task processing apparatus executes the method of any of the above aspects.

[0025] In a fourth aspect, a computer readable storage medium is provided, which stores instructions, when the instructions are run on a computer, the computer can execute the method of any of the above aspects.

[0026] In a fifth aspect, a computer program product is provided, which contains instructions, when the instructions are run on a computer, the computer can execute the method of any of the above aspects.

[0027] In a sixth aspect, a chip system is provided, which includes a processor and a communication interface, the communication interface is used to communicate with a module outside the chip, the processor is used to run a computer program or instructions, so that the device installed with the chip system can execute the method of any of the above aspects.

[0028] In a seventh aspect, a computing device is provided, which includes the task processing device of the third aspect or the chip of the sixth aspect, the task processing device or the chip system in the computing device is used to implement the operation steps of the method of any of the above aspects.

[0029] In an eighth aspect, a computing device cluster is provided, which includes at least one computing device, any of the computing devices is used to run a computer program or instructions, so that the computing device cluster can execute the method of any of the above aspects. Alternatively, part or all of the computing devices are collectively used to run a computer program or instructions, so that the computing device cluster can execute the method of any of the above aspects.

[0030] On the basis of the implementation manners of the above aspects provided by the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0031] FIG. 1 is a schematic diagram of a system architecture 100 provided by the present application;

[0032] FIG. 2 is a flowchart of a task processing method provided by the present application;

[0033] FIG. 3 is a flowchart of a task processing provided by the present application;

[0034] FIG. 4 is a schematic diagram of a process of adjusting vectors corresponding to a specific task type in a word vector table provided by the present application;

[0035] FIG. 5 is a schematic diagram of a system architecture provided by the present application;

[0036] FIG. 6 is a schematic diagram of adding a new word unit in a word segmenter provided by the present application;

[0037] FIG. 7 is a schematic diagram of newly adding vectors in an original word vector table provided by the present application;

[0038] FIG. 8 is a flowchart of a model fine-tuning process according to the present application;

[0039] FIG. 9 is a flowchart of a large language model performing batch task inference according to the present application;

[0040] FIG. 10 is a structural diagram of a task processing device according to the present application;

[0041] FIG. 11 is a structural diagram of a computing device according to the present application;

[0042] FIG. 12 is a structural diagram of a computing device cluster according to the present application;

[0043] FIG. 13 is a structural diagram of another computing device cluster according to the present application;

[0044] FIG. 14 is a structural diagram of a chip according to the present application;

[0045] FIG. 15 is a structural diagram of a computer-readable storage medium according to the present application. DETAILED DESCRIPTION

[0046] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Those skilled in the art can know that as new application scenarios appear, the technical solutions provided by the present application are also applicable to similar technical problems.

[0047] The terms "first", "second", and the like in the description and in the claims of the present application and above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the descriptions thus used can be interchanged, where appropriate, to enable the embodiments to be implemented in an order other than that illustrated or described in the present application. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a list of steps or modules as processes, methods, systems, products, or apparatuses are not necessarily limited to those steps or modules clearly listed, but can include other steps or modules not clearly listed or inherent to these processes, methods, products, or apparatuses. The naming or numbering of steps in the present application does not mean that the steps in the method flow must be performed in the order / time / logical order indicated by the naming or numbering. The flow steps that have been named or numbered can change the order of execution according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of units in the present application is a logical division, and in actual application, it can have another division method, for example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed, in addition, the coupling or direct coupling or communication connection between the units shown or discussed can be through some interface, the indirect coupling or communication connection between the units can be electrical or other similar forms, which are not limited in the present application. And the units or sub-units described as separate components can or can not be physically separated, can or can not be physical units, or can be distributed to multiple circuit units, and some or all of the units can be selected according to actual needs to achieve the purpose of the present application.

[0048] For ease of understanding, some technical terms related to the present application are introduced first.

[0049] (1) Large language model (LLM)

[0050] A large language model refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc., and are an important way to artificial intelligence.

[0051] Specifically, large language models are a technology that has emerged in recent years. Because large language models have undergone a fine data engineering and training process, a large amount of existing natural language processing knowledge has been learned inside their parameters. This knowledge can already replace humans for many language-related tasks, such as having large language models write code or having large language models summarize text.

[0052] (2) Prompt

[0053] Prompt originates from the input form designed by researchers for downstream tasks. Its role is to help the pre-trained model "recall" what it has "learned" during pre-training, so it can also be called prompt. For large language models, prompt is the user's input, indicating the task the large language model needs to complete. Prompt can be a simple question, a longer text, or a set of instructions, depending on the user's specific needs. Generally, prompt is usually a short text string that provides context and task-related information to help the model better understand the requirements and generate correct output. For example, in a question-answering task, the prompt may contain the question or topic description to help the large language model generate the correct answer. Moreover, prompt is usually designed by humans to help the large language model better understand specific tasks or domains.

[0054] In this way, when the large language model generates content, it first processes the prompt and then outputs based on its understanding of the prompt. The working principle of the large language model is to predict the probability of the next word appearing based on the user's input context, and then generate the next context word by word. Therefore, the difference in the user's input prompt will directly affect the quality of the large language model output. In some cases, the difference in the user's input prompt is just a few words, and the large language model may also generate significantly different content.

[0055] (3) Neural network

[0056] The neural network can be composed of neural units, and the neural unit can refer to an operation unit with xs (input data) and intercept 1 as input. The output of the operation unit can be:

[0057] where s = 1, 2, … n, n is a natural number greater than 1, Ws is the weight of xs, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. The neural network is a network formed by connecting multiple single neural units, i.e. the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of several neural units.

[0058] (4) Transformer network

[0059] Transformer network is a neural network model based on self-attention mechanism, which is used to process sequence data. Compared with traditional recurrent neural network model, Transformer network has better parallel performance and shorter training time, so it has been widely used in natural language processing field.

[0060] Transformer network contains multiple layers of encoder and decoder, each layer is composed of multiple attention mechanism modules and feedforward neural network modules. Among them, encoder is used to encode input sequence into a high-dimensional feature vector representation, and decoder is used to decode the vector representation into target sequence. In Transformer network, techniques such as residual connection and layer normalization are also used to accelerate model convergence and improve model performance.

[0061] Specifically, the core of Transformer network is self-attention mechanism, which assigns a weight to each position in the input sequence, and then uses these weighted position vectors as output. Generally, the calculation process of self-attention mechanism includes three steps: 1, calculate attention weight, calculate the attention weight between each position and other positions, that is, the importance of each position to other positions; 2, calculate the weighted sum, multiply each position vector with the attention weight, and then add them to get the weighted sum vector; 3, linear transformation, linear transformation is performed on the weighted sum vector to get the final output vector.

[0062] (5) Low-Rank Adaptation (LoRA)

[0063] LoRA is a low-rank approximation technique for large language models, mainly used to deal with the problem of fine-tuning large language models. In LoRA technology, the weights of the pre-trained large language model are fixed, and trainable layers (low-rank decomposition matrices) are injected in each Transformer block, and only the parameters of the trainable layers are fine-tuned. Because it does not need to adjust the original weight parameters in the large language model, but only adjusts the parameters of the newly injected trainable layers, it greatly reduces the amount of parameters that need to be adjusted.

[0064] (6) Token

[0065] A token is the smallest semantic unit in natural language processing, which can be obtained by decomposing long texts such as combined words, sentences, paragraphs, articles, etc. Generally, a token can be composed of letters, words or symbols, for example, composed of one or more letters, or composed of one or more words. Simply put, a token can be understood as a "fragment" of a word or a phrase, that is, the smallest semantic unit that cannot be further divided. For example, "waterfall" can be divided into two tokens: water and fall. In addition, punctuation marks can also be decomposed into tokens, because punctuation marks also affect the semantic understanding of the whole text. For example, "I don't know." can be divided into 5 tokens: "I", "don", "'t", "know", and "·".

[0066] The current large language model fine-tuning method is usually for a specific type of task, and the fine-tuned large language model can only improve the processing ability for a specific type of task, and cannot have better performance on other types of tasks, resulting in poor applicability of the fine-tuned large language model. For example, when you want to improve the processing ability of the large language model for sentiment analysis tasks, you can fine-tune the weight parameters of the large language model based on the training data related to sentiment analysis tasks collected in advance, so that the fine-tuned large language model can effectively process sentiment analysis tasks. However, since the weight parameters in the fine-tuned large language model have been adjusted, and only for sentiment analysis tasks, the fine-tuned large language model will not have performance improvement when processing other types of tasks, and may even have performance decline, that is, the fine-tuning of the large language model for different tasks will interfere with each other. For example, the fine-tuned large language model may have a precision decline when processing intelligent question and answer tasks.

[0067] Therefore, in order to improve the precision of the large language model for various types of tasks, it is currently often necessary to fine-tune the large language model based on different types of task data, thereby obtaining multiple fine-tuned large language models for different types of tasks. Obviously, the fine-tuned large language models obtained based on the current fine-tuning method have poor applicability and are difficult to apply to processing different types of tasks.

[0068] Therefore, the application provides a task processing method. When a prompt word describing a task is obtained, a word element corresponding to a task type described by the prompt word is added before the prompt word to obtain a new prompt word. When the new prompt word is processed, the newly added word element in the new prompt word can be determined to a corresponding vector in a target word vector table, so as to realize conversion of the new prompt word into a vector sequence composed of the vector corresponding to the newly added word element and the vector corresponding to the original word element. Finally, a model processes the vector sequence to obtain a task result output by the model. By adding a corresponding word element before the prompt word based on the task type, and converting the added word element into a corresponding vector, knowledge strongly related to the task can be added based on the task input, so as to improve the accuracy of the model output result. In addition, for different types of tasks, only corresponding word elements need to be added for different types of tasks, and vectors corresponding to the word elements need to be added in the target word vector table, so as to realize improvement of the accuracy of the model in processing various types of tasks, and the weight parameters of the model itself do not need to be adjusted, isolation between different tasks is realized, and it is ensured that the model can process different types of tasks in batches.

[0069] Referring to FIG. 1, FIG. 1 is a schematic diagram of a system architecture 100 provided by the application. As shown in FIG. 1, in the system architecture 100, the execution device 110 can be implemented by at least one computing instance of a physical host (computing device), a virtual machine, or a container. When the execution device 110 is implemented by a virtual machine or a container, the execution device 110 actually exists in the form of a cloud computing product, and can provide cloud services.

[0070] Optionally, the execution device 110 cooperates with other computing devices, such as a data storage device, a load balancer, and the like. The execution device 110 can be arranged on one physical site, or distributed on multiple physical sites.

[0071] In addition, the system architecture 100 is also provided with a database system 120. The database system 120 is used to store word elements corresponding to various types of tasks and word vector tables and the like.

[0072] Optionally, in order to perform persistent storage of data, the database system 120 can be located outside the execution device 110, and exchange data with the execution device 110 through a network. Optionally, in the case where the execution device 110 is a physical host, the database system 120 can also be located inside the execution device 110, such as the database system 120 exchanging data with the processor through a bus. At this time, the database system 120 behaves as a hard disk. In the case where the database system 120 is provided, the execution device 110 can use data (such as a word vector table) in the database system 120, or call program code in the data storage system to implement the task processing method provided by the application.

[0073] Optionally, a user can operate respective user devices (e.g., local device 101 and local device 102) to interact with the execution device 110. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smartphone, a tablet computer, a notebook computer, a smart car, and the like.

[0074] Each local device of a user can interact with the execution device 110 through a communication network of any communication mechanism / standard, which can be a wide area network, a local area network, a point-to-point connection, or the like, or any combination thereof.

[0075] In one implementation, the execution device 110 is configured to implement the task processing method provided by the present application, thereby obtaining a corresponding text clustering result.

[0076] Optionally, during the implementation of the task processing method by the execution device 110, the local device 101 and the local device 102 can provide the execution device 110 with a task request or a prompt word, so that the execution device 110 can process the prompt word through a model to implement the processing of the task. Moreover, after the execution device 110 implements the task processing method to obtain a task result, the task result can be fed back to the local device 101 and the local device 102.

[0077] Optionally, the prompt word required by the execution device 110 when implementing the task processing method can also be pre-stored in the database system 120, and the execution device 110 can obtain the prompt word to be processed by reading the prompt word data from the database system 120.

[0078] In another implementation, one aspect or multiple aspects of the execution device 110 can be implemented by each local device, for example, the local device 101 can provide the execution device 110 with local data or feedback calculation results, or implement the task processing method provided by the present application.

[0079] In general, the task processing method provided by the present application can be applied to an electronic device, such as the execution device 110, the local device 101, or the local device 102 described above.

[0080] Referring to FIG. 2, FIG. 2 is a flow diagram of a task processing method provided by the present application. As shown in FIG. 2, the task processing method provided by the present application includes the following steps 201-204.

[0081] Step 201, obtaining a first prompt word, the first prompt word being used to describe a target task to be processed.

[0082] Generally, when a user employs a model to perform a task, a corresponding prompt word is often needed to be provided to enable the model to output a corresponding task result based on the input prompt word. In this application, the first prompt word can be a prompt word provided by the user. The first prompt word can also be a prompt word generated based on the task expected to be completed by the model. For example, when the user sends a task request to the execution device, the execution device performs prompt word engineering according to the task request, thereby generating a corresponding first prompt word. The source of the first prompt word is not specifically limited in this application.

[0083] In addition, the first prompt word can describe the target task to be processed in the form of text. For example, referring to FIG. 3, FIG. 3 is a flowchart of task processing provided by the present application. As shown in FIG. 3, assuming that the target task to be processed is a question and answer task, the first prompt word can be specifically "Please ask what is the ending of XX character in XX novel".

[0084] For another example, assuming that the target task to be processed is a translation task, the first prompt word can be specifically "Please help to translate the following sentence into English, the sentence is: XXX". For another example, assuming that the target task to be processed is a text generation task, the first prompt word can be specifically "Please refer to the following example XXX, write a story about XXX". For another example, assuming that the target task to be processed is a sentiment analysis task, the first prompt word can be specifically "Please analyze the following review content XXX, and output whether the customer's evaluation is recognized or not recognized".

[0085] Step 202, based on the type of the target task, at least one target token is added to the first prompt word to obtain a second prompt word, wherein the at least one target token has a corresponding relationship with the type of the target task.

[0086] After obtaining the first prompt word, at least one target token can be added before the first prompt word based on the type of the target task described in the first prompt word, thereby obtaining a second prompt word composed of at least one target token and the first prompt word. Wherein, the at least one target token has a corresponding relationship with the type of the target task in advance. That is, for any prompt word obtained, if the type of the task described by the prompt word is the same as the type of the target task, the at least one target token can also be determined for the prompt word.

[0087] The at least one target word piece can include one or more word pieces, and the number of word pieces included in the at least one target word piece can be specified according to the type of the task in the actual scenario, so as to improve the processing accuracy of the corresponding task as much as possible. In addition, the at least one target word piece added before the first prompt word can be in the form of one or more separated target word pieces, or in the form of a word or a sentence composed of one or more target word pieces. The form of the target word piece is not limited in the present application, as long as the vector corresponding to each target word piece can be determined in the target word vector table.

[0088] Optionally, for different types of tasks, the present application can set corresponding word piece sets for different types of tasks in advance, so that the corresponding word piece set can be quickly determined for the prompt word describing various types of tasks when processing various types of tasks.

[0089] For example, after obtaining the first prompt word, the target word piece set corresponding to the type of the target task described by the first prompt word can be determined from the plurality of word piece sets. The target word piece set includes the at least one target word piece described above. Each word piece set in the plurality of word piece sets includes one or more word pieces. Each word piece set has a corresponding task type, and different word piece sets have a corresponding relationship with different task types, that is, the plurality of word piece sets and the plurality of task types have a one-to-one correspondence. Therefore, based on the type of the task, the corresponding word piece set can be found for each prompt word describing different types of tasks.

[0090] After determining the target word piece set corresponding to the first prompt word, at least one target word piece included in the target word piece set can be added before the first prompt word based on the target word piece set.

[0091] In the present scheme, by establishing the corresponding relationship between the plurality of word piece sets and the different task types, the scheme can be adapted to various different types of tasks, so as to determine the word piece to be added for various prompt words describing different types of tasks, and ensure the applicability of the scheme.

[0092] Exemplarily, it is assumed that the question and answer task corresponds to a word set 1, the word set 1 includes the words [abc, def, ghi, jkl]; the sentiment analysis task corresponds to a word set 2, the word set 1 includes the words [hjk, wih, lkp]. Then, as shown in FIG. 3, in the case that the first prompt word is used to describe the question and answer task, and specifically is “What is the ending of XX character in XX novel?”, the second prompt word “abc def ghi jkl What is the ending of XX character in XX novel?” can be obtained by adding the words in the word set 1 in front of the first prompt word. Similarly, in the case that the second prompt word is used to describe the sentiment analysis task, and specifically is “Please analyze the following review content XXX, and output whether the customer's evaluation is recognized or not recognized”, the second prompt word “hjk wih lkp Please analyze the following review content XXX, and output whether the customer's evaluation is recognized or not recognized” can be obtained by adding the words in the word set 2 in front of the first prompt word.

[0093] Optionally, in the case that there are multiple word sets, the target word vector table includes vectors corresponding to the words in each of the multiple word sets. Since each of the multiple word sets corresponds to a task type, any one of the word sets can be added to the prompt word, and therefore the target word vector table needs to include vectors corresponding to the words in each of the word sets, so as to be able to find the corresponding vector for the added words in any prompt word.

[0094] Step 203, determining the vector corresponding to the second prompt word based on the target word vector table, obtaining a first vector sequence, the target word vector table includes at least one target vector corresponding to at least one target word.

[0095] In this application, the target word vector table is used to indicate the correspondence between the words and the vectors, and the target word vector table includes vectors corresponding to various words. Therefore, based on the target word vector table, the vector corresponding to each word in the second prompt word can be determined, thereby obtaining the first vector sequence. The first vector sequence is actually composed of the vectors corresponding to each word in the second prompt word, and the arrangement order of the vectors in the first vector sequence corresponds to the arrangement order of the words in the second prompt word.

[0096] In the target word vector table, at least one target vector corresponding to the at least one target word unit is included, that is, each word unit in the at least one target word unit has a corresponding target vector. Therefore, based on the target word vector table, the at least one target word unit in the second prompt word can be converted into at least one target vector, and the at least one target vector is located in front of the first vector sequence (that is, before the vector corresponding to the word unit in the first prompt word).

[0097] Optionally, in order to facilitate the determination of the vector corresponding to each word unit in the second prompt word, the tokenizer can be used to perform tokenization processing on the second prompt word to obtain a plurality of word units constituting the second prompt word; then, based on the target word vector table, the vector corresponding to each word unit in the plurality of word units constituting the second prompt word is determined, thereby obtaining the first vector sequence.

[0098] The tokenizer is used to decompose text into a plurality of predefined word units, so as to convert the text into a corresponding vector at the granularity of the word unit. It should be noted that, since the present application introduces a corresponding word unit set according to the task type for different types of tasks, and the word unit set includes some self-defined word units (that is, word units without specific meaning in reality), the tokenizer also needs to adapt to these newly introduced word unit sets to ensure that the tokenizer can effectively identify and segment the word units added in the prompt word.

[0099] For example, when the second prompt word is “abc def ghi jk Please ask in XX novel, what is the ending of XX character”, the tokenization result obtained after the tokenizer performs tokenization processing is “abc / def / ghi / jkl / please ask / in / XX / novel / , / XX / character’s / ending / is / what / ”. When the second prompt word is “hjk wih lkp Please analyze the following comment content XXX, and output the customer’s evaluation to determine whether it is recognized or not”, the tokenization result obtained after the tokenizer performs tokenization processing is “hjk / wih / lk / please / analyze / following / comment / content / XXX / , / and / output / customer’s / evaluation / to / determine / whether / it / is / recognized / or / not / ”.

[0100] It should be noted that different word sets are established for different types of tasks in order to add task corresponding word units before the prompt words of various task types, so as to facilitate subsequent conversion of task corresponding word units into task corresponding vectors. Since there is a corresponding relationship between word units and vectors, the task corresponding word units are actually determined by the task corresponding vectors. In the model fine-tuning stage, it is actually to determine what kind of vector needs to be added for various types of tasks in order to improve the accuracy of the model in processing these types of tasks. Moreover, the vector to be added for various types of tasks is determined by processing the training data by the model, so the vector to be added for various types of tasks can be completely different or partially the same, and the present embodiment does not limit this. Based on this, in the plurality of word sets, the word units included in different word sets can also be completely different or partially the same. Alternatively, even if the vectors to be added for various types of tasks are partially the same, the word units included in the word sets corresponding to various types of tasks can also be completely different, that is, different word units can correspond to the same vector.

[0101] In step 204, the first model is used to process the first vector sequence to obtain a task result.

[0102] After obtaining the first vector sequence, the first vector sequence can be input into the first model, and the first vector sequence is processed by the neural network layer in the first model, and finally the corresponding task result is output by the first model. The first model is, for example, a natural language processing model such as a large language model, a long short-term memory (LSTM) network, a gated recurrent unit (GRU) network, an attention network, a bidirectional encoder representations from transformers (BERT), a generative pre-trained transformer (GPT), etc. The specific structure of the first model is not limited in the present embodiment.

[0103] The task result output by the first model is related to the target task described by the first prompt. For example, assuming that the target task described by the first prompt is a question and answer task, the task result output by the first model is an answer to the question asked by the first prompt. As shown in FIG. 3, in the case where the first prompt is “Please ask in XX novel, what is the ending of XX character”, the task result output by the first model is “In XX novel, the ending of XX character is XXX”. For another example, assuming that the target task described by the first prompt is a translation task, the task result output by the first model is a translation result of the content to be translated indicated by the first prompt.

[0104] It can be understood that in the present solution, the task-related word units are actually added to the prompt, and the vectors corresponding to the added word units are expanded in the target word vector table, so that the task strongly related knowledge is introduced in the processing of the prompt. Since only word units are added before the prompt in the present solution, and these word units can also be determined to the corresponding vector based on the target word vector table in the process of converting word units to vectors, the overall process of the model processing task is actually not changed, and only the task corresponding word units need to be added before the prompt, so as to ensure that the model can process various types of tasks in batches. On the contrary, if the vectors corresponding to the tasks are dynamically loaded according to the type of the task in the model inference stage, the model cannot process different types of tasks in batches, thereby affecting the efficiency of the model processing task.

[0105] Optionally, in the present application, the target word vector table described above can be obtained by adding at least one target vector in the original word vector table, and the original word vector table is the word vector table originally used by the first model. That is, for the original first model, the first model originally uses the original word vector table to convert the input prompt into a vector sequence. However, in the present solution, some self-defined word units are added to the prompt, so the original word vector table used by the first model can be expanded, so that the expanded word vector table (i.e. the target word vector table) can include the vectors corresponding to the added word units in the prompt.

[0106] It should be noted that when the first model of the present solution is used to process various types of tasks, the vectors corresponding to various newly added word units can also be added in the original word vector table for the word units added to the prompts for various types of tasks, so that the prompts with various word units can determine the corresponding vectors in the target word vector table.

[0107] In the scheme, the correspondence between the newly added word units in the prompt words and the vectors is established by expanding the original word vector table used by the model, which can realize the expansion of the original word vector table to complete the model fine-tuning for the task, without adjusting the weight parameters of the model itself, ensuring that various types of tasks can adapt to the same model, and improving the applicability of the scheme.

[0108] For the original word vector table, any one of the at least one target word unit newly added in the first prompt word does not have a corresponding vector in the original word vector table. That is, in the original word vector table, no corresponding vector can be found for the at least one target word unit, that is, no correspondence between the at least one target word unit and the vector is established. In this way, in the model fine-tuning stage, when updating the corresponding word vector table of the first model (i.e., updating the vector corresponding to the newly added word unit in the word vector table), it is ensured that the existing vector in the original word vector table will not be adjusted to avoid affecting the model to perform other types of tasks.

[0109] Since the at least one target vector added in the target word vector table actually corresponds to a specific type of task, and the scheme actually adds the at least one target vector before the vector corresponding to the original prompt word, the representation of the target vector becomes the ability to improve the model to process a specific type of task. In order to ensure that the at least one target vector added before the vector corresponding to the original prompt word can effectively improve the ability of the model to process a specific type of task, the model fine-tuning process can be introduced in the scheme, and the weight parameters of the model itself are not adjusted in the model fine-tuning process, but the vectors corresponding to the specific type of task in the word vector table are adjusted.

[0110] Exemplarily, refer to FIG. 4, which is a schematic diagram of a process of adjusting the vectors corresponding to the specific type of task in the word vector table provided by the present application. As shown in FIG. 4, the process of performing the model fine-tuning process to adjust the vectors corresponding to the specific type of task in the word vector table can include steps 401-403.

[0111] Step 401, adding at least one initial vector corresponding to at least one target word unit in the original word vector table to obtain the target word vector table before updating.

[0112] In the present application, for the same type of task as the target task introduced above, it is set to add the at least one target word unit before the prompt word corresponding to these tasks. And in order to realize the conversion of the word unit to the vector, the at least one target word unit needs to have a corresponding vector in the word vector table. Therefore, in the present application, at least one initial vector corresponding to the at least one target word unit is added in the original word vector table first, so as to obtain the target word vector table before updating. Among them, the at least one initial vector may be at least one vector initialized with a value, and the dimension of the at least one initial vector may be consistent with the dimension of the original vector in the original word vector table, so as to facilitate subsequent conversion of the target word unit to a vector that can be effectively recognized by the model.

[0113] It should be noted that in order to facilitate drawing, there is only one vector added in the original word vector table in FIG. 4 (i.e., the at least one target vector described above is an example of one vector), and in actual application, the vector added for each task type can be one or more.

[0114] Step 402, based on the target word vector table before updating, generate a vector sequence corresponding to the training data, the training data is obtained by adding at least one target word unit before the preset prompt word, and the task described by the preset prompt word is of the same type as the target task.

[0115] Among them, the training data can be constructed based on the preset prompt word and the at least one target word unit. The task described by the preset prompt word is of the same type as the target task, for example, both are sentiment analysis tasks or intelligent question and answer tasks, so the at least one target word unit needs to be added in front of the preset prompt word when processing. Since the word units in the preset prompt word can find corresponding vectors in the original word vector table, and the at least one target word unit added before the preset prompt word can also find at least one target vector in the target word vector table before updating, based on the target word vector table before updating, the vector sequence corresponding to the training data can be generated.

[0116] Optionally, when generating the vector sequence corresponding to the training data, the vector corresponding to each word unit in the training data can be determined based on the target word vector table before updating to obtain a second vector sequence. Then, random noise is added to the vectors corresponding to the at least one target word unit in the second vector sequence to obtain the vector sequence corresponding to the training data. In the present application, by first adding random noise to the vector in the vector updating process, and then inputting the obtained vector sequence into the model for processing, the stability of learning the vector corresponding to the target word unit can be effectively improved, and the effective and accurate target vector can be finally obtained.

[0117] In step 403, the first model processes the vector sequence corresponding to the training data to obtain an output result of the first model, and updates at least one initial vector based on the output result to obtain at least one target vector.

[0118] In this way, by inputting the vector sequence corresponding to the training data into the first model, the output result obtained after the first model processes the vector sequence can be obtained. Based on the output result of the first model and the correct task result corresponding to the preset prompt word, a loss function can be constructed; and by finding the value of the minimum loss function, at least one target vector in the word vector table is updated to improve the processing capability of the first model for the current type of task as much as possible.

[0119] It should be noted that the above steps 401-403 introduce a round of updating the added vectors in the original word vector table based on the training data. In actual application, the added vectors in the original word vector table can be updated for multiple rounds based on multiple training data, until the added vectors can make the accuracy of the first model processing the training data reach the requirement or the number of iteration reaches a certain number, and finally realize the conversion of the at least one initial vector added in the original word vector table to the at least one target vector.

[0120] The above introduces the execution flow of the task processing method provided by the present application. For the sake of understanding, the model fine-tuning method provided by the present application and the task processing process after the model fine-tuning are introduced in detail below with specific examples. It should be noted that the model fine-tuning method introduced in the present application does not adjust the weight parameters of the model itself, but only adjusts the original segmenter and word vector table, so that the word units added before the prompt word can be recognized by the segmenter and the corresponding vector can be found in the word vector table.

[0121] Please refer to FIG. 5, which is a schematic diagram of a system architecture provided by the present application. As shown in FIG. 5, the segmenter and the word vector table are separated and deployed in the present application, so that the weight parameters and structure of the large language model itself do not need to be changed, but only the original segmenter and word vector table are adjusted.

[0122] The word segmenter and the word vector table can be deployed on a central processing unit (CPU) or AI hardware as a preprocessing service, and the large language model can be deployed on AI hardware. The AI hardware can be a graphics processing unit (GPU), a tensor processing unit (TPU), or a neural network processing unit (NPU), for example.

[0123] During the operation of the system architecture, the input data is first subjected to word segmentation processing by the word segmenter to obtain a token sequence. Then, by looking up the word vector table, the vector corresponding to each token in the token sequence is determined to obtain a vector sequence. Finally, the vector sequence is input into the large language model to obtain the task result output by the large language model.

[0124] In this application, for different types of tasks, corresponding token sets can be set for these different types of tasks, and each token set can include one or more tokens. In this way, when obtaining a prompt word corresponding to any type of task, the token set corresponding to the current type of task needs to be added before the prompt word. Moreover, in order to ensure that the subsequent table lookup based on these token sets will not find the original vector in the word vector table, the tokens included in these token sets are all new tokens defined by the application, i.e., tokens without semantics in reality, but only for adapting various types of tasks.

[0125] After setting the corresponding token sets for various types of tasks, since the original word segmenter recognizes all tokens with semantics in reality, and the token sets set in this application include new tokens without specific meanings, it is necessary to add the token sets set in this application to the word segmenter, so that the word segmenter can recognize the tokens added before the prompt word and perform token division.

[0126] By way of example, refer to FIG. 6, which is a schematic diagram of adding tokens to a word segmenter according to the application. As shown in FIG. 6, the word segmenter includes an original token set, which includes some tokens with semantics defined in real life. Then, the application sets corresponding token sets for intelligent question answering tasks, sentiment analysis tasks, translation tasks, and text generation tasks, which are token set 1-token set 4. In this way, the token sets set for specific types of tasks, i.e., token set 1-token set 4, can be added to the word segmenter, so that the word segmenter can recognize each token in token set 1-token set 4, thereby successfully performing token division operations.

[0127] Furthermore, for word-gram sets set for different types of tasks, corresponding vectors can also be set for the word-grams included in these word-gram sets, so that when the prompt word is converted into a vector sequence, the vectors corresponding to the word-grams in these word-gram sets can be found. Specifically, by adding the vector corresponding to each word-gram in each word-gram set to the original word vector table, a corresponding relationship between the newly defined word-grams and the vectors can be established.

[0128] For example, please refer to Figure 7, which is a schematic diagram of a new vector added to the original word vector table provided by the present application. As shown in Figure 7, it is assumed that the original word vector table includes N vectors, namely vector 1-vector N; and, each of the word element set 1-word element set 4 includes only 1 vector. For the word element set 1 corresponding to the intelligent question and answer task, the vector N+1 corresponding to the word element set 1 can be added on the basis of the original word vector table; for the word element set 2 corresponding to the sentiment analysis task, the vector N+2 corresponding to the word element set 2 can be added; for the word element set 3 corresponding to the translation task, the vector N+3 corresponding to the word element set 3 can be added; for the word element set 4 corresponding to the text generation task, the vector N+4 corresponding to the word element set 4 can be added. In this way, after adding the above-mentioned vector N+1-vector N+4 to the original word vector table, the target word vector table can be obtained.

[0129] It should be noted that the above example is based on an example in which there is only one word in the word set. In actual applications, the word set may include one or more words, and the vector corresponding to the word set may also be one or more, that is, the word and the vector are one-to-one corresponding.

[0130] Please refer to Figure 8, which is a schematic diagram of a process of model fine-tuning provided by this application. As shown in Figure 8, the process of implementing model fine-tuning includes the following steps 801-807.

[0131] Step 801: Construct prompt words.

[0132] First, after obtaining the original prompt word corresponding to the task (i.e., the first prompt word in the above embodiment), based on the type of task described by the original prompt word, one or more lemmas included in the lemma set corresponding to the task are added before the original prompt word. For example, in Figure 8, the task corresponding to the original prompt word is a text generation task, and the original prompt word is specifically "Have you not seen the water of the Yellow River come from the sky?" Therefore, based on the lemma set 4 corresponding to the text generation task, one or more lemmas from lemma set 4 can be added before the original prompt word to construct the prompt word, thereby obtaining the input prompt word (i.e., the second prompt word in the above embodiment).

[0133] At step 802, word segmentation processing is performed on the prompt word.

[0134] After obtaining the input prompt word, word segmentation processing can be performed on the input prompt word based on the word segmenter, so as to divide the input prompt word into multiple word pieces and obtain a word piece sequence. As shown in FIG. 8, for the input prompt word “[Word Piece Set 4] Jun Buzheng Huanghe Zhishui Tian Shang lai”, the following word piece sequence “[Word Piece Set 4] Jun / Buzheng / Huanghe / Zhishui / Tian / Shang / lai” can be obtained. In the case where the word piece set 4 includes multiple word pieces, the word segmenter will also perform word piece division on the multiple word pieces in the word piece set 4.

[0135] At step 803, the word piece sequence obtained by the word segmentation processing is converted into an index sequence.

[0136] Generally, in order to facilitate the implementation of finding the vector corresponding to the word piece, the word piece is usually converted into a corresponding digital index, and then the corresponding vector is found based on the digital index. That is, there is a layer of digital index between the word piece and the vector. Based on this, in the present scheme, the digital index corresponding to each word piece in the word piece sequence obtained by the word segmentation processing is found, so as to convert the word piece sequence obtained by the word segmentation processing into an index sequence.

[0137] For example, in FIG. 8, each word piece in the word piece set 4 corresponds to a newly added index. The word piece “Jun” corresponds to the index “10”, the word piece “Buzheng” corresponds to the index “3”… and the word piece “lai” corresponds to the index “543”. In general, each word piece in the word piece sequence can find a corresponding index, and the index is used to find the corresponding vector in the word vector table.

[0138] At step 804, the word vector table is found based on the index sequence.

[0139] In the word vector table, each vector has a unique corresponding index, so the vector corresponding to each index in the index sequence can be determined based on the index sequence.

[0140] At step 805, sequence vectorization is performed based on the found vectors.

[0141] After finding the vector corresponding to each index based on the index sequence, the found vectors can be sequentially spliced according to the arrangement order of the multiple indexes in the index sequence, so as to obtain a vector sequence.

[0142] At step 806, the vector sequence is processed by a large language model.

[0143] After obtaining the vector sequence, it is input into the large language model, which processes the resulting vector sequence to produce the large language model's output. For example, in Figure 8, the large language model's output is specifically "Have you not seen the Yellow River's waters come from heaven, rushing to the sea and never returning..." This means that the large language model generated the second half of the poem based on the first half of the prompt word.

[0144] Step 807: Update the newly added vector in the word vector table based on the output result of the large language model.

[0145] Finally, based on the output results of the large language model and the content that the user expects the large language model to output, a loss function can be constructed, and then the newly added vectors in the word vector table (that is, the vectors corresponding to the text generation task) can be updated based on the loss function, so that the updated vectors can improve the ability of the large language model to handle text generation tasks.

[0146] Furthermore, the process of updating the newly added vectors in the word embedding table can include multiple rounds of iterative updates, i.e., repeating steps 801-807 multiple times, with each round of iterative updates fine-tuning the vectors corresponding to the text generation task. Furthermore, when fine-tuning the vectors corresponding to the text generation task, random noise can be added to improve stability during the vector update process, ensuring that vectors that effectively improve the processing capabilities of the large language model are ultimately learned.

[0147] It should be noted that the above steps 801-807 use the text generation task as an example to describe how to update the vectors corresponding to the text generation task in the word embedding table. In actual applications, for other tasks, the above steps 801-807 can also be used to update the vectors corresponding to these tasks in the word embedding table. This allows the large language model to effectively improve its processing capabilities for a variety of different types of tasks without adjusting the weight parameters.

[0148] Please refer to Figure 9, which is a flow chart of a large language model provided by this application for performing batch task reasoning. As shown in Figure 9, since this solution performs word-unit expansion on the original word segmenter and vector expansion on the original word vector table, the entire task reasoning process is actually unchanged. After the word segmenter and word vector table are updated, it is only necessary to add the word-unit set corresponding to the task to the original prompt word, so that the subsequent processing flow of the prompt word is the same as the original processing flow, and no other new processes need to be introduced.

[0149] In FIG. 9, after obtaining the original prompt word of task 1 and the original prompt word of task 2, a corresponding set of word pieces can be added before the original prompt word according to the types of task 1 and task 2, respectively. For example, the set of word pieces 1 is added before the original prompt word of task 1, and the set of word pieces 2 is added before the original prompt word of task 2. Then, the prompt word after adding the set of word pieces is subjected to word segmentation processing, and the vector corresponding to the word piece is found to obtain a vector sequence. And a large language model is used to batch process the obtained multiple vector sequences, and finally multiple task results are obtained, that is, the task 1 result and the task 2 result shown in FIG. 9.

[0150] The above describes the method provided by the present application in detail. Next, the device provided by the present application for executing the above method will be introduced.

[0151] Please refer to FIG. 10, which is a structural schematic diagram of a task processing device provided by the present application. As shown in FIG. 10, the task processing device comprises: an acquisition module 1001 configured to acquire a first prompt word, the first prompt word being used to describe a target task to be processed; a processing module 1002 configured to add at least one target word piece on the basis of the first prompt word based on the type of the target task, to obtain a second prompt word, wherein the at least one target word piece has a corresponding relationship with the type of the target task; the processing module 1002 is further configured to determine a vector corresponding to the second prompt word based on a target word vector table, to obtain a first vector sequence, the target word vector table comprising at least one target vector corresponding to the at least one target word piece; and the processing module 1002 is further configured to process the first vector sequence by a first model to obtain a task result.

[0152] In a possible implementation, the target word vector table is obtained by adding at least one target vector in an original word vector table, and the original word vector table is a word vector table originally used by the first model.

[0153] In a possible implementation, any one of the at least one target word piece does not have a corresponding vector in the original word vector table.

[0154] In a possible implementation, the processing module 1002 is further configured to: add at least one initial vector corresponding to the at least one target word piece in the original word vector table to obtain a target word vector table before updating; generate a vector sequence corresponding to training data based on the target word vector table before updating, the training data being obtained by adding the at least one target word piece before a preset prompt word, the task described by the preset prompt word being of the same type as the target task; process the vector sequence corresponding to the training data by the first model to obtain an output result of the first model, and update the at least one initial vector based on the output result to obtain the at least one target vector.

[0155] In a possible implementation, the processing module 1002 is further configured to: determine, based on the target word vector table before the update, a vector corresponding to each word piece in the training data, to obtain a second vector sequence; and add random noise to the vector corresponding to the at least one target word piece in the second vector sequence, to obtain the vector sequence corresponding to the training data.

[0156] In a possible implementation, the processing module 1002 is further configured to: perform word segmentation processing on the second prompt word by using a word segmenter, to obtain a plurality of word pieces; and determine, based on the target word vector table, a vector corresponding to each word piece in the plurality of word pieces, to obtain a first vector sequence.

[0157] In a possible implementation, the processing module 1002 is further configured to: determine, based on the type of the target task, a target word piece set corresponding to the type of the target task from the plurality of word piece sets, the target word piece set including at least one target word piece, and in the plurality of word piece sets, different word piece sets correspond to different task types; and add the at least one target word piece before the first prompt word based on the target word piece set.

[0158] In a possible implementation, the target word vector table includes a vector corresponding to a word piece in each word piece set in the plurality of word piece sets.

[0159] The obtaining module 1001 or the processing module 1002 can be implemented by software or by hardware. For example, the implementation of the processing module 1002 is described below.

[0160] As an example of a software functional unit, the processing module 1002 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the A module can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region, or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ), or in different AZs, and each AZ includes one data center or multiple data centers in a similar geographical location. Generally, one region can include multiple AZs.

[0161] Likewise, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, usually one VPC is set in one region, and communication between two VPCs in the same region and between VPCs in different regions needs to set a communication gateway in each VPC to realize the interconnection between VPCs through the communication gateway.

[0162] As an example of a hardware functional unit, the processing module 1002 can include at least one computing device, such as a server, etc. Alternatively, the processing module 1002 can also be a device implemented by a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or a programmable logic device (PLD), etc. Among them, the above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system on chip (SoC), an offload card, an acceleration card, or any combination thereof.

[0163] The multiple computing devices included in the processing module 1002 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 1002 can be distributed in the same AZ or in different AZs. Likewise, the multiple computing devices included in the processing module 1002 can be distributed in the same VPC or in multiple VPCs. Among them, the multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offload cards, acceleration cards, etc.

[0164] Referring to FIG. 11, FIG. 11 is a structural diagram of a computing device provided by the present application. The computing device 1100 shown in FIG. 11 can be used to perform the task processing method provided by the present embodiment. As shown in FIG. 11, the computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate through the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that the number of processors and memories in the computing device 1100 is not limited by the present application.

[0165] The bus 1102 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 11, but it does not mean that there is only one bus or only one type of bus. The bus 1102 can include a path for transmitting information between various components (e.g., the memory 1106, the processor 1104, the communication interface 1108) of the computing device 1100.

[0166] The processor 1104 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0167] The memory 1106 can include a volatile memory (e.g., a random access memory (RAM)). The processor 1104 can also include a non-volatile memory (e.g., a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD)).

[0168] The executable program code stored in the memory 1106 is executed by the processor 1104 to realize the functions of the aforementioned obtaining module and processing module respectively, so as to realize the model processing method described above. That is, the memory 1106 stores instructions for executing the model processing method.

[0169] The communication interface 1108 uses a transceiving module such as but not limited to a network interface card and a transceiver to realize the communication between the computing device 1100 and other devices or communication networks.

[0170] The present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

[0171] Please refer to FIG. 12, which is a structural schematic diagram of a computing device cluster provided by the present application. As shown in FIG. 12, the computing device cluster includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster can store the same instructions for executing the task processing method.

[0172] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster can also respectively store partial instructions for executing the task processing method. In other words, the combination of one or more computing devices 1100 can collectively execute the instructions for executing the task processing method.

[0173] It should be noted that the memories 1106 in different computing devices 1100 in the computing device cluster can store different instructions for respectively executing partial functions of the data processing apparatus. That is, the instructions stored in the memories 1106 in different computing devices 1100 can realize the functions of one or more of the aforementioned obtaining module and processing module.

[0174] In some possible implementation manners, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. FIG. 13 shows a possible implementation manner. FIG. 13 is a structural schematic diagram of another computing device cluster provided in this application. As shown in FIG. 13, in the computing device cluster 1300, two computing devices 1100A and 1100B are connected through a network. Specifically, the computing devices are connected to the network through communication interfaces in the computing devices. In this kind of possible implementation manner, the memory 1106 in the computing device 1100A stores instructions for performing the function of the obtaining module. Meanwhile, the memory 1106 in the computing device 1100B stores instructions for performing the function of the processing module.

[0175] It should be understood that the functions of the computing device 1100A shown in FIG. 13 can also be completed by a plurality of computing devices 1100. Similarly, the functions of the computing device 1100B can also be completed by a plurality of computing devices 1100.

[0176] The application also provides a chip, which includes a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin, a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the electronic device performs the method described in the above embodiments. Alternatively, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0177] Specifically, refer to FIG. 14, which is a structural schematic diagram of a chip provided in this application. The chip can be a neural processing unit (NPU), a graphics processing unit (GPU), or a tensor processing unit (TPU). Hereinafter, the chip will be taken as an example of the NPU 1400. The NPU 1400 is mounted as a coprocessor on a host CPU, and the host CPU allocates tasks. The core part of the NPU is an operation circuit 1403, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1404.

[0178] In some implementations, the arithmetic circuit 1403 includes a plurality of processing units (PEs) inside. In some implementations, the arithmetic circuit 1403 is a two-dimensional systolic array. The arithmetic circuit 1403 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.

[0179] For example, assume that there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit takes the data of the matrix B corresponding from the weight memory 1402 and caches it on each PE of the arithmetic circuit. The arithmetic circuit takes the data of the matrix A from the input memory 1401 and performs matrix operations with the matrix B to obtain partial results or final results of the matrix, which are saved in the accumulators 1408.

[0180] The unified memory 1406 is used to store input data and output data. The weight data is transferred to the weight memory 1402 through the direct memory access controller (DMAC) 1405. The input data is also transferred to the unified memory 1406 through the DMAC.

[0181] The bus interface unit (BIU) 1410 is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 1409.

[0182] The bus interface unit (BIU) 1410 is used for the instruction fetch buffer 1409 to obtain instructions from the external memory and is also used for the direct memory access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0183] The DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 1406 or to transfer the weight data to the weight memory 1402 or to transfer the input data to the input memory 1401.

[0184] The vector calculation unit 1407 includes a plurality of arithmetic processing units and further processes the output of the arithmetic circuit 1403 as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / full connection layer network calculation in neural networks, such as batch normalization, pixel-level summation, upsampling of feature planes, etc.

[0185] In some implementations, the vector computation unit 1407 can store the processed output vector to the unified memory 1406. For example, the vector computation unit 1407 can apply a linear function; or, a non-linear function to the output of the arithmetic circuit 1403, such as linear interpolation to the feature planes extracted by a convolution layer, and further such as accumulating the values of the vector to generate the activation values. In some implementations, the vector computation unit 1407 generates normalized values, pixel-wise summed values, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 1403, such as for use in a subsequent layer in a neural network.

[0186] The controller 1404 is connected to an instruction fetch buffer 1409 for storing instructions used by the controller 1404;

[0187] The unified memory 1406, the input memory 1401, the weight memory 1402, and the instruction fetch buffer 1409 are all on-chip memories. Off-chip memories are private to the NPU hardware architecture.

[0188] Any of the processors mentioned above can be a general purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the programs mentioned above.

[0189] Referring to FIG. 15, FIG. 15 is a structural diagram of a computer readable storage medium provided by the present disclosure. The present disclosure also provides a computer readable storage medium, in some embodiments, the method disclosed in FIG. 2 above can be implemented as computer program instructions encoded in a computer readable storage medium in a machine readable format or encoded in other non-transitory media or articles of manufacture.

[0190] FIG. 15 schematically illustrates a conceptual partial view of an example computer readable storage medium arranged in accordance with at least some embodiments presented herein, the example computer readable storage medium including computer program for executing a computer process on a computing device.

[0191] In one embodiment, the computer readable storage medium 1500 is provided using a signal bearing medium 1501. The signal bearing medium 1501 can include one or more program instructions 1502 which, when executed by one or more processors, can provide at least some of the functionality described above with respect to FIG. 2.

[0192] In some examples, the signal bearing medium 1501 can embody a computer readable medium 1503, such as, but not limited to, a hard disk drive, a compact disk (CD), a digital video disk (DVD), a digital tape, memory, ROM, RAM, etc.

[0193] In some embodiments, the signal bearing medium 1501 can comprise one or more computer- readable media 1503, one or more processor-readable media 1504, and / or one or more computer- readable and / or processor-readable media 1505. For example, examples of computer-readable media 1503, processor-readable media 1504, and computer-readable and / or processor-readable media 1505 can include, but are not limited to, primary storage, secondary storage, and / or other storage. Such computer-readable media 1503, processor-readable media 1504, and computer-readable and / or processor-readable media 1505 can reside with or within one or more components of the computing device 1500, such as, but not limited to, the storage component 1506, the processor component 1507, and / or the like. In some embodiments, the signal bearing medium 1501 can comprise a computer- recordable medium 1504, such as, but not limited to, memory, a read / write (R / W) CD, a R / W DVD, and / or the like. In some embodiments, the signal bearing medium 1501 can comprise a communication medium 1505, such as, but not limited to, a digital and / or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, and / or the like). Thus, for example, the signal bearing medium 1501 can embody the communication medium 1505 in a wireless form (e.g., a wireless communication medium complying with the IEEE 1202.X standard or other transmission protocol).

[0194] The one or more program instructions 1502 can be, for example, computer- executable instructions or logic-implementing instructions. In some examples, the computing device of the computing device can be configured to provide various operations, functions, or actions in response to the program instructions 1502 being conveyed to the computing device by one or more of the computer-readable medium 1503, the computer-recordable medium 1504, and / or the communication medium 1505.

[0195] It should be noted that the above-described apparatus embodiments are merely illustrative, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. In addition, the connection relationship between the modules in the apparatus embodiments provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0196] Those skilled in the art can clearly understand the application by the description of the above embodiments, and the application can be realized by software and necessary general hardware, or by special hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, functions completed by computer program can be easily realized by corresponding hardware, and specific hardware structure for realizing the same function can be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on the understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method of each embodiment of the application.

[0197] In the above embodiments, the application can be realized by software, hardware, firmware or any combination thereof, wholly or partially. When realized by software, the application can be realized in the form of computer program product, wholly or partially.

[0198] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the flow or function according to the application is generated, wholly or partially. The computer can be general-purpose computer, special-purpose computer, computer network or other programmable device. The computer instructions can be stored in computer readable storage medium, or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website site, computer, training device or data center to another website site, computer, training device or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be stored by computer, or data storage device such as training device, data center, etc. integrated with one or more available media. The available medium can be magnetic medium (such as floppy disk, hard disk, magnetic tape), optical medium (such as DVD) or semiconductor medium (such as solid state disk (SSD)), etc.

Claims

1. A task processing method characterized by, The method comprises: obtaining a first prompt word, the first prompt word being used to describe a target task to be processed; adding at least one target word element to the first prompt word based on the type of the target task to obtain a second prompt word, wherein the at least one target word element has a corresponding relationship with the type of the target task; determining a vector corresponding to the second prompt word based on a target word vector table to obtain a first vector sequence, the target word vector table comprising at least one target vector corresponding to the at least one target word element; processing the first vector sequence through a first model to obtain a task result.

2. The method of claim 1, wherein, The target word vector table is obtained by adding the at least one target vector to an original word vector table, which is a word vector table originally used by the first model.

3. The method of claim 2, wherein, Any one of the at least one target word element does not have a corresponding vector in the original word vector table.

4. The method according to claim 2 or 3, characterized in that, The method further comprises: adding at least one initial vector corresponding to the at least one target word element to the original word vector table to obtain a target word vector table before updating; generating a vector sequence corresponding to training data based on the target word vector table before updating, the training data being obtained by adding the at least one target word element before a preset prompt word, the task described by the preset prompt word being the same as the type of the target task; processing the vector sequence corresponding to the training data through the first model to obtain an output result of the first model, and updating the at least one initial vector based on the output result to obtain the at least one target vector.

5. The method of claim 4, wherein, The generating of the vector sequence corresponding to the training data based on the target word vector table before updating comprises: determining a vector corresponding to each word element in the training data based on the target word vector table before updating to obtain a second vector sequence; adding random noise to the vectors corresponding to the at least one target word element in the second vector sequence to obtain the vector sequence corresponding to the training data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: performing word segmentation on the second prompt word through a word segmenter to obtain a plurality of word elements; The determining of the vector corresponding to the word element in the second prompt word based on the target word vector table to obtain the first vector sequence comprises: determining a vector corresponding to each word element in the plurality of word elements based on the target word vector table to obtain the first vector sequence.

7. The method according to any one of claims 1 to 6, characterized in that, The adding of the at least one target word element before the first prompt word based on the type of the target task comprises: determining a target word element set having a corresponding relationship with the type of the target task from a plurality of word element sets based on the type of the target task, the target word element set comprising the at least one target word element, and different word element sets in the plurality of word element sets having corresponding relationships with different task types; adding the at least one target word element before the first prompt word based on the target word element set.

8. The method of claim 7, wherein, The target word vector table comprises vectors corresponding to word elements in each word element set in the plurality of word element sets.

9. A task processing apparatus characterized by comprising: The method comprises: The acquisition module is configured to acquire a first prompt word, the first prompt word being used to describe a target task to be processed. The processing module is configured to add at least one target word element to the first prompt word based on a type of the target task to obtain a second prompt word, the at least one target word element having a corresponding relationship with the type of the target task. The processing module is further configured to determine a vector corresponding to the second prompt word based on a target word vector table to obtain a first vector sequence, the target word vector table including at least one target vector corresponding to the at least one target word element. The processing module is further configured to process the first vector sequence by using a first model to obtain a task result.

10. The apparatus of claim 9, wherein, The target word vector table is obtained by adding the at least one target vector to an original word vector table, the original word vector table being a word vector table originally used by the first model.

11. The apparatus of claim 10, wherein, Any one of the at least one target word element does not have a corresponding vector in the original word vector table.

12. The apparatus of claim 10 or 11, wherein, The processing module is further configured to: add at least one initial vector corresponding to the at least one target word element to the original word vector table to obtain a target word vector table before updating; generate a vector sequence corresponding to training data based on the target word vector table before updating, the training data being obtained by adding the at least one target word element before a preset prompt word, the task described by the preset prompt word being of the same type as the target task; process the vector sequence corresponding to the training data by using the first model to obtain an output result of the first model, and update the at least one initial vector based on the output result to obtain the at least one target vector.

13. The apparatus of claim 12, wherein, The processing module is further configured to: determine a vector corresponding to each word element in the training data based on the target word vector table before updating to obtain a second vector sequence; add random noise to the vectors corresponding to the at least one target word element in the second vector sequence to obtain the vector sequence corresponding to the training data.

14. The apparatus of any of claims 9-13, wherein, The processing module is further configured to: perform word segmentation on the second prompt word by using a word segmenter to obtain a plurality of word elements; determine a vector corresponding to each word element in the plurality of word elements based on the target word vector table to obtain the first vector sequence.

15. The apparatus of any of claims 9-14, wherein, The processing module is further configured to: determine a target word element set having a corresponding relationship with the type of the target task from a plurality of word element sets based on the type of the target task, the target word element set including the at least one target word element, and different word element sets in the plurality of word element sets having corresponding relationships with different task types; add the at least one target word element before the first prompt word based on the target word element set.

16. A computing device, comprising: A computing device includes a memory and a processor, the memory storing code, and the processor being configured to execute the code, when the code is executed, the computing device performing the method of any one of claims 1 to 8.

17. A cluster of computing devices, characterized in that, A computing device includes a memory and a processor, the memory storing code, and the processor being configured to execute the code, when the code is executed, the computing device performing the method of any one of claims 1 to 8. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the operational steps of the method of any of claims 1 to 8.

18. A chip system, characterized by The chip system comprises a processor and a communication interface configured to communicate with a module outside the chip system, and the processor is configured to perform the method of any of claims 1 to 8.

19. A computer storage medium, comprising, The computer storage medium stores instructions which, when executed by a computer, cause the computer to implement the method of any of claims 1 to 8.

20. A computer program product, characterised in that, The computer program product stores instructions which, when executed by a computer, cause the computer to implement the method of any of claims 1 to 8.

Images