Task execution method and device based on large model, intelligent agent and electronic equipment

Abstract

The invention provides a task execution method and device based on a large model, an intelligent agent and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical fields of voice interaction, deep learning, large models and the like. The method comprises the following steps: acquiring a demand feature representing a demand intention; a target processing task is executed according to the demand features by utilizing a large model, a reply text matched with the demand intention is obtained, and target reply words in the reply text are determined on the basis of the following modes: respective query features of a plurality of attention sub-tasks in the target processing task are determined on the basis of associated reply word features, the associated reply character features are determined based on associated reply characters in the reply text; based on the demand feature read from the storage unit as a value feature and a key feature shared by the plurality of attention sub-tasks, the plurality of attention sub-tasks are executed by the computing unit according to the plurality of query features, the value feature and the key feature.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, particularly to technologies such as voice interaction, deep learning, large models, etc., and can be applied to application scenarios such as knowledge search, autonomous driving, intelligent customer service, intelligent voice control, intelligent e-commerce, AI medical care, etc. Background Art

[0002] With the rapid development of artificial intelligence technologies, in the process of human-computer interaction, multi-modal input information such as text, voice, video, etc. input by users can be processed according to the Artificial Intelligence Generated Content (AIGC) technology. By processing the input information based on the relatively large-scale model parameters of the large model, information such as retrieval content and question answers required by users can be generated. Summary of the Invention

[0003] The present disclosure provides a task execution method, apparatus, intelligent agent, electronic device, and storage medium based on a large model.

[0004] According to one aspect of the present disclosure, there is provided a task execution method based on a large model, including: obtaining a demand feature representing a demand intention; using the large model to perform a target processing task according to the demand feature to obtain a reply text matching the demand intention, wherein the target reply word in the reply text is determined based on the following method: determining query features for respective multiple attention subtasks in the target processing task based on associated reply word features, the associated reply word features being determined based on associated reply words in the reply text; using the demand feature read from the storage unit as a value feature and a key feature shared by the multiple attention subtasks, and using a calculation unit to perform the multiple attention subtasks according to the multiple query features, value feature, and key feature to obtain the target reply word.

[0005] According to another aspect of the present disclosure, there is provided a task execution apparatus based on a large model, including: an obtaining module, configured to obtain a demand feature representing a demand intention; a reply text obtaining module, configured to use the large model to perform a target processing task according to the demand feature to obtain a reply text matching the demand intention, wherein the target reply word in the reply text is determined based on the following method: determining query features for respective multiple attention subtasks in the target processing task based on associated reply word features, the associated reply word features being determined based on associated reply words in the reply text; using the demand feature read from the storage unit as a value feature and a key feature shared by the multiple attention subtasks, and using a calculation unit to perform the multiple attention subtasks according to the multiple query features, value feature, and key feature to obtain the target reply word.

[0006] According to another aspect of the present disclosure, there is provided an intelligent agent of artificial intelligence, including:

[0007] An input module for receiving input information;

[0008] A processing module for determining a target task based on the input information received by the input module, determining a large model based on the target task, and obtaining output information by calling the large model to execute the task execution method based on the large model provided in the embodiments of the present disclosure;

[0009] An output module for outputting the output information obtained by the processing module.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the task execution method based on the large model provided in the embodiments of the present disclosure.

[0011] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the task execution method based on the large model provided in the embodiments of the present disclosure.

[0012] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, where the computer program, when executed by a processor, implements the task execution method based on the large model provided in the embodiments of the present disclosure.

[0013] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0014] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0015] Figure 1 Schematically shows an exemplary system architecture to which the task execution method and device based on the large model according to the embodiments of the present disclosure can be applied;

[0016] Figure 2A Schematically shows a flowchart of the task execution method based on the large model according to the embodiments of the present disclosure;

[0017] Figure 2B Schematically shows a flowchart of determining the target reply word in the reply text according to the embodiments of the present disclosure;

[0018] Figure 3Schematically shows the schematic diagram of the task execution method based on the large model according to an embodiment of the present disclosure;

[0019] Figure 4 Schematically shows the schematic diagram of the principle of executing multiple attention subtasks;

[0020] Figure 5 Schematically shows the application scenario diagram of the task execution method based on the large model according to an embodiment of the present disclosure;

[0021] Figure 6 Schematically shows the block diagram of the task execution device based on the large model according to an embodiment of the present disclosure;

[0022] Figure 7 Schematically shows the block diagram of the structure of the intelligent agent of artificial intelligence according to an embodiment of the present disclosure; and

[0023] Figure 8 Shows the schematic block diagram of the electronic device 800 that can be used to implement the task execution method based on the large model of the embodiments of the present disclosure. Detailed implementation manners

[0024] The following makes an explanation of the exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0025] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and public order and good customs are not violated.

[0026] The inventors found that when using a large model to process the user's multimodal input information to generate a reply to the user, it is difficult for the large model used to generate the reply information to efficiently and stably generate content that matches the user's demand intention, which easily leads to problems such as large computational overhead and excessive storage resource occupation of the computing device that executes the reply information generation task based on the model parameters of the large model.

[0027] Embodiments of the present disclosure provide a method, apparatus, intelligent agent, electronic device, and storage medium for task execution based on a large model. The method for task execution for a large model includes: obtaining a requirement feature representing a requirement intention; using the large model to execute a target processing task according to the requirement feature to obtain a response text matching the requirement intention, where the target response word in the response text is determined based on the following method: determining query features for respective multiple attention subtasks in the target processing task based on associated response word features, and the associated response word features are determined based on associated response words in the response text; using the requirement feature read from the storage unit as a value feature and a key feature shared by the multiple attention subtasks, and using a computing unit to execute the multiple attention subtasks according to the multiple query features, value features, and key features to obtain the target response word.

[0028] According to embodiments of the present disclosure, by obtaining a requirement feature representing a requirement intention and using the requirement feature read from the storage unit as a value feature and a key feature shared by multiple attention subtasks in the large model, the computing unit is used to execute the multiple attention subtasks, which can avoid excessive storage space occupation caused by the computing unit calculating and storing the value feature and the key feature for each attention subtask. By using the computing unit to execute the attention subtasks according to the query features of the multiple attention subtasks determined based on the associated response word features, as well as the shared value features and key features, the computing overhead and storage space occupation generated by the computing device based on the large model in executing the target processing task can be saved, and the generation efficiency of the generated response text can be improved.

[0029] To facilitate understanding of the embodiments of the present disclosure, the meanings of the English abbreviations or technical terms involved in the embodiments of the present disclosure can be explained based on the following content.

[0030] Artificial Intelligence Generated Content (AIGC) is a technology that uses artificial intelligence technology, especially methods such as large pre-trained models, to generate relevant content through learning and pattern recognition of existing data and with appropriate generalization ability. The core idea of AIGC technology is to use artificial intelligence algorithms to generate content with certain creativity and quality, and it can generate related articles, images, audio, etc. according to the input conditions or instructions.

[0031] Large models can include deep learning models with a large number of parameters and complex structures. Large models are capable of processing massive amounts of data and performing various types of complex tasks based on large-scale model parameters and complex model structures. For example, they can perform complex tasks such as natural language processing, computer vision, speech recognition, etc. Large models can be constructed based on deep neural networks and have billions or even hundreds of billions of parameters. They learn complex patterns and features by training on massive amounts of data, have stronger generalization capabilities, and can make accurate predictions on unseen data. Large models can include large language models (LLMs). A large language model can be a model based on machine learning and natural language processing technologies, which learns the ability to serve human language understanding and generation by training on a large amount of text data. Large models can have billions or even hundreds of billions of parameters, which enables them to capture more language knowledge and complex grammatical structures. In terms of the network structure of large language models, for example, network structures such as Transformer can be adopted. Large language models have powerful context awareness capabilities when processing text, can understand and generate text content that depends on the previous text, and can more accurately understand text content in conversations, article generation, and situation understanding. The large models involved in the embodiments of the present disclosure can include large language models, or can also include other types of generative large models in addition to large language models.

[0032] Figure 1 Schematically shows an exemplary system architecture to which the task execution method and apparatus based on a large model according to an embodiment of the present disclosure can be applied.

[0033] It should be noted that Figure 1 The figure shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, the exemplary system architecture to which the task execution method and apparatus based on a large model can be applied can include a terminal device, but the terminal device can implement the task execution method and apparatus based on a large model provided by the embodiments of the present disclosure without interacting with the server.

[0034] As Figure 1 shown, the system architecture 100 according to this embodiment can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired and / or wireless communication links, etc.

[0035] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0036] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.

[0037] Server 105 can be a server that provides various services, such as a background management server that supports the content browsed by users using terminal devices 101, 102, and 103 (for example only). The background management server can analyze and process data such as received user requests, etc., and feedback the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0038] Server 105 can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS" for short). Server 105 can also be a server of a distributed system, or a server combined with a blockchain.

[0039] It should be noted that the task execution method based on a large model provided by the embodiments of the present disclosure can generally be executed by server 105. Correspondingly, the task execution device based on a large model provided by the embodiments of the present disclosure can generally be set in server 105. The task execution method based on a large model provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103, and / or server 105. Correspondingly, the task execution device based on a large model provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103, and / or server 105.

[0040] It should be understood, Figure 1 The numbers of terminal devices, networks, and servers in

[0041] Figure 2AA flowchart of a task execution method based on a large model according to an embodiment of the present disclosure is schematically shown.

[0042] As Figure 2A shown, the task execution method based on the large model includes operations S210 to S220.

[0043] In operation S210, demand features characterizing the demand intention are obtained.

[0044] In operation S220, the large model is used to perform a target processing task according to the demand features, and a response text matching the demand intention is obtained.

[0045] According to an embodiment of the present disclosure, the demand intention may be, for example, the user's intention to inquire about the quality of goods, the intention to operate a device, etc. The embodiment of the present disclosure does not limit the specific type of the demand intention. The demand features may be obtained by extracting features from the demand information of the target object. The demand information may include information in any modality, such as demand information in modalities of text, voice, image, etc. The demand features characterize the demand intention expressed by information in any modality.

[0046] According to an embodiment of the present disclosure, the demand features may imply the demand intention of the target object. By using the large model to perform the target processing task according to the demand features, the large model can more clearly understand the demand intention, and based on the powerful text generation ability of the large model, a response text matching the demand intention is generated, so that the response text can meet the demand intention of the target object.

[0047] Figure 2B A flowchart of determining target response words in the response text according to an embodiment of the present disclosure is schematically shown.

[0048] As Figure 2B shown, the target response words in the response text are determined based on operations S221 to S222.

[0049] In operation S221, query features for each of multiple attention subtasks in the target processing task are determined based on associated response word features.

[0050] In operation S222, using the demand features read from the storage unit as value features and key features shared by the multiple attention subtasks, the computing unit performs the multiple attention subtasks according to the multiple query features, value features, and key features to obtain the target response words.

[0051] According to an embodiment of the present disclosure, the response text may be structured text information such as a table, or may also include unstructured text information such as a novel, an abstract, etc. The embodiments of the present disclosure do not limit the specific type of the response text. The target response word and the associated response words may be response words in the response text. The associated response word feature is determined based on the associated response words in the response text. For example, the associated response word feature may be obtained by performing feature extraction on one or more associated response words.

[0052] In one example, the associated response words may be one or more response words arranged before the target response word in the response text. The associated response word feature may be obtained by performing feature extraction on the associated response words based on the attention mechanism.

[0053] According to an embodiment of the present disclosure, the multiple attention subtasks in the target processing task may include data processing processes executed based on multiple attention heads in the multi-head attention network, and the multiple query features are respectively applied to the multiple attention subtasks. Determining the query features for each of the multiple attention subtasks in the target processing task based on the associated response word feature may include calculating the associated response word feature based on the query weights of each of the multiple attention subtasks to obtain the query features corresponding to each of the multiple attention subtasks.

[0054] According to an embodiment of the present disclosure, the computing unit may include a Graphics Processing Unit (GPU), a Tensor Processing Unit (TPU), a Central Processing Unit (CPU), a Neural network Processing Unit (NPU), an artificial intelligence chip, etc. The embodiments of the present disclosure do not limit the specific type of the computing unit.

[0055] According to an embodiment of the present disclosure, demand features can be stored in storage units such as a cache unit and a video memory unit. Some or all of the demand features can be determined from the storage unit as the value features and key features of any attention subtask, so that the computing unit can execute the attention subtask according to the query feature, value feature, and key feature to implement the data processing process of multiple attention subtasks. This can avoid separately generating corresponding value features and key features for each attention subtask and storing multiple different value features and key features for multiple attention subtasks in the storage unit, thereby explaining the storage space occupied by the computing unit when executing the target processing task based on the large model. At the same time, using the demand features as the value features and key features shared by multiple attention subtasks can also save the computing overhead generated by the computing unit calculating the demand features based on the value weight or key weight, thereby reducing the overall computing overhead of the computing unit, improving the execution efficiency of the computing device when executing the target processing task based on the large model, and improving the generation efficiency of the reply information.

[0056] The following will further illustrate the Figure 2A and Figure 2B method shown with reference to the accompanying drawings in conjunction with specific embodiments.

[0057] In one example, the reply text includes multiple reply characters arranged in sequence. Multiple associated reply characters are arranged before the target reply character, and the associated reply characters can be used as the context of the target reply character. The associated reply character feature is obtained by fusing the features of multiple associated reply characters based on the attention mechanism. For example, the features of multiple associated reply characters can be fused based on the self-attention mechanism, so that the associated reply character feature represents the generated reply semantics. By executing multiple attention subtasks based on the large model to fuse the demand features and the context semantics represented by the associated reply character features, the target reply character semantically coherent with the current multiple associated reply characters can be generated in turn, so that the reply text can match the demand intention.

[0058] According to an embodiment of the present disclosure, the demand features are determined by extracting the features of the demand information of the target object. The arrangement order of multiple sub-demand features in the demand features is determined according to the arrangement order of multiple sub-demand information in the demand information. The multiple sub-demand information can be sorted based on a preset sorting rule, or the multiple sub-demand information can also be sorted based on the semantic rule of the demand information and the generation time sequence of the sub-demand information. The specific setting method for the arrangement order of the multiple sub-demand information in the embodiments of the present disclosure is not limited.

[0059] According to embodiments of the present disclosure, the arrangement order of multiple sub - requirement features can represent the semantic relationship between multiple sub - requirement information in the requirement information. Thus, the semantics of the requirement intention representation can be more accurately represented based on the arrangement order of multiple sub - requirement features. Furthermore, by using multiple sub - requirement features in the requirement feature as value features and key features, it is possible to enable the query feature to perform attention fusion based on the arrangement order of multiple sub - requirement features without performing position encoding on the requirement feature, and to implicitly represent the positions of multiple sub - features in the value feature or key feature required for performing the attention sub - task based on the arrangement order of multiple sub - requirement features, avoiding the redundant occupation of storage space caused by performing position encoding on the value feature or key feature. It is also possible to avoid problems such as low accuracy and poor quality of the response text caused by position mismatch due to performing position encoding on the value feature or key feature based on the arrangement order of multiple sub - requirement features.

[0060] According to embodiments of the present disclosure, the sub - requirement information includes at least one of the following: requirement words in the requirement text, requirement speech frames in the requirement speech, and image blocks in the requirement image.

[0061] According to embodiments of the present disclosure, the requirement text includes multiple requirement words arranged in an order corresponding to semantic rules. Multiple sub - requirement features can be arranged in the order corresponding to semantic rules. The sub - requirement feature can include the semantic attribute represented by the corresponding requirement word, or the sub - requirement feature can also include the semantic attribute represented by the corresponding requirement word and the context in the requirement text.

[0062] According to embodiments of the present disclosure, the requirement speech can be represented based on a sequence of requirement speech frames. The order of multiple requirement speech frames can represent the order of multiple words in the text expressed by the requirement speech. The arrangement order of multiple sub - requirement features can be the same as the order of multiple requirement speech frames. The sub - requirement feature can represent the semantic recognition attribute of the requirement speech frame, such as representing the recognition semantics of the requirement word in the requirement text. Or, the sub - requirement feature can also include the corresponding requirement speech frame and the context semantic attribute in the requirement speech.

[0063] According to embodiments of the present disclosure, the image blocks in the requirement image can be arranged based on a preset order. Multiple sub - requirement features in the requirement feature corresponding to the requirement image blocks can be the same as the arrangement order of multiple image blocks. The sub - requirement feature can represent the image semantics of the corresponding image block. Or, the sub - requirement feature can also represent the image semantics of the corresponding image block and the image semantics of other image blocks in the requirement image.

[0064] In one example, feature extraction can be performed on requirement information based on a deep learning model. For example, a deep learning model constructed based on an attention network algorithm can be used to perform feature extraction and attention fusion on each sub-requirement information in the requirement information to obtain multiple sub-requirement features. The sub-requirement features can correspond to the sub-requirement information, and the sub-requirement features can fuse the context semantics in the requirement information based on the attention mechanism so that the sub-requirement features can more fully represent the requirement intention. In this way, multiple sub-requirement features can be used as value features and key features to more accurately represent the requirement intention, so that the large language model can, on the condition of fully understanding the requirement intention of the requirement information, use the computing unit to perform the target processing task based on the large language model and generate a reply text with a relatively high matching degree to the requirement intention, improving the quality of the reply text.

[0065] According to an embodiment of the present disclosure, multi-level feature extraction can be performed on requirement information based on a cascade of multiple feature extraction layers to achieve deep feature extraction and fusion of the requirement information. By using the hidden features output by the last feature extraction layer or the hidden features output by the specified feature extraction layer among the multiple feature extraction layers as the requirement features, it is possible to avoid all the hidden features output by the multiple feature extraction layers being input into the large model, thereby reducing the computational overhead of the computing unit performing the target processing task based on the large model. Using the hidden features output by the network layer with the specified depth among the multiple feature extraction layers as the requirement features can make the representation of the semantic attributes of the requirement features for the requirement intention adapt to the attention mechanisms of multiple attention subtasks in the large model, reduce the data calculation amount of the target processing task, and at the same time reduce the data transmission bandwidth and storage redundancy amount required for the computing unit to perform the target processing task based on the large model, thereby achieving efficient execution of the target processing task for the large model and improving the reply efficiency for the target object.

[0066] In one example, the following example can be used to perform multi-level feature extraction on requirement information based on a cascade of multiple feature extraction layers: The requirement features include requirement text features, and the requirement text features are determined based on the following operations: performing first-level semantic feature extraction on the requirement text of the target object to obtain first-level intermediate requirement text features; performing n-level semantic feature extraction on the (n - 1)-level intermediate requirement text features to obtain n-level intermediate requirement text features including multiple n-level intermediate sub-requirement features; in the case of N = n, determining the N-level intermediate requirement text features as the requirement text features.

[0067] According to an embodiment of the present disclosure, N ≥ n > 1, and N and n are integers.

[0068] According to an embodiment of the present disclosure, the first-level intermediate requirement text features include a plurality of first-level intermediate sub-requirement features corresponding to a plurality of requirement words in the requirement text. The plurality of sub-requirement features in the requirement text features correspond to the plurality of requirement words, and the arrangement order of the plurality of sub-requirement features is the same as the arrangement order of the plurality of requirement words. The requirement text features can implicitly represent the positions of the respective sub-features in the value feature and the key feature based on the arrangement order of the plurality of sub-requirement features.

[0069] Exemplarily, for semantic feature extraction of the requirement text, it may include feature extraction of the requirement text based on any type of algorithm such as an attention network algorithm, a convolutional neural network algorithm, etc.

[0070] In one example, the multi-level feature extraction of the requirement information can be performed based on the following example on a cascade of multiple feature extraction layers: The requirement features include requirement image features, and the requirement image features are determined based on the following operations: performing first-level requirement image feature extraction on the requirement image of the target object to obtain first-level intermediate requirement image features; performing nth-level requirement image feature extraction on the (n - 1)th-level intermediate requirement image features to obtain nth-level intermediate requirement image features including a plurality of nth-level intermediate sub-requirement features; in the case of N = n, determining the Nth-level intermediate requirement image features as the requirement image features.

[0071] Among them, the first-level intermediate requirement image features include a plurality of first-level intermediate sub-requirement features corresponding to a plurality of image blocks in the requirement image; the plurality of sub-requirement features in the requirement image features correspond to the plurality of image blocks, and the arrangement order of the plurality of sub-requirement features in the requirement image features is the same as the preset order among the plurality of image blocks.

[0072] The sub-requirement features in the requirement image features can represent the image semantics of the image blocks with a mapping relationship in the arrangement position, or the sub-requirement features in the requirement image features can represent the image semantics of the image blocks with a mapping relationship in the arrangement position, as well as the context image semantics represented by the plurality of image blocks.

[0073] In one example, the requirement information is requirement speech, and the sub-requirement information of the requirement speech can be requirement speech frames. The multi-level feature extraction of the requirement information is performed based on a cascade of multiple feature extraction layers to obtain a plurality of sub-requirement speech recognition features corresponding to the arrangement order of the plurality of requirement speech frames.

[0074] According to an embodiment of the present disclosure, the requirement features include a plurality of sub-requirement features arranged in sequence; the sub-requirement features arranged in sequence can represent the requirement intention of the requirement information in any modality based on the arrangement position of the plurality of sub-requirement features and the semantic attributes implied by the sub-requirement features.

[0075] According to an embodiment of the present disclosure, using a computing unit to perform multiple attention subtasks according to multiple query features, value features, and key features includes: based on the permutation order among multiple sub-requirement features in the key feature, using the computing unit to perform a first calculation operation based on the multiple sub-requirement features and multiple sub-query features in the query feature to obtain attention weights for the attention subtask; based on the permutation order among multiple sub-requirement features in the value feature, using the computing unit to perform a second calculation operation based on the attention weights and multiple sub-requirement features to obtain a sub-execution result of the attention subtask.

[0076] According to an embodiment of the present disclosure, the first calculation operation may include an operation process of fusing a query feature (Query) and a key feature (Key) based on an attention algorithm to obtain attention weights (Attention_Weights). For example, the first calculation operation may include a dot product operation and a normalization operation, and obtain attention scores (scores) output by the attention subtask by calculating the dot product between the query feature (Query) and the key feature (Key). By using the computing unit to perform a normalization operation on the attention scores of the attention subtask, the attention weights of the attention subtask are obtained.

[0077] For example, the first calculation operation of the attention subtask may be represented based on the following formulas (1) and (2).

[0078] (1);

[0079] attention_weights = softmax(scores) (2);

[0080] Where scores are attention scores, attention_weights are attention weights, softmax() is a normalization function, is a constant used to scale the dot product result, Q head is the query feature for the attention subtask, K represents the key feature of the attention subtask, and the requirement feature is used as the key feature. Therefore, the key feature may include multiple sub-requirement features ci arranged in order. The key feature K is represented as K = {c1, c2... ci}. Multiple sub-query features in the query feature may perform the first calculation operation based on the respective positions of the multiple sub-requirement features arranged in order in the key feature, so as to perform the first calculation operation using the computing unit without adding position encoding to the key feature and the query feature, further saving the computing overhead and storage space occupied by the computing unit for executing the attention subtask.

[0081] According to an embodiment of the present disclosure, the value feature may include a plurality of sub-requirement features ci arranged in sequence. The value feature can be represented as V = {c1, c2... ci}. The second calculation operation may include performing a weighted operation on the value feature based on the attention weights.

[0082] For example, the second calculation operation of the attention sub-task can be performed based on formula (3).

[0083] O head = attention_weights × V (3);

[0084] where O head is the sub-execution result of the attention sub-task, and V is the value feature read from the storage unit. It should be understood that the sub-execution result can be a feature vector with the same dimension as the value feature or the key feature.

[0085] According to an embodiment of the present disclosure, the target reply word is determined based on the sub-execution results of multiple attention sub-tasks. For example, the calculation results output by each of the multiple attention heads can be used as the sub-execution results of the multiple attention sub-tasks. The target reply word is determined by fusing the sub-execution results of the multiple attention sub-tasks based on the attention mechanism, and the reply text is obtained by iteratively generating multiple target reply words.

[0086] According to an embodiment of the present disclosure, for multiple attention sub-tasks, the requirement features stored in the storage unit are used as their respective value features and key features to perform attention calculation. This can save the storage space of the storage unit and utilize the positional information implicitly represented by the arrangement order of the multiple sub-requirement features in the requirement feature to perform attention fusion with the query feature, enabling the attention sub-task to learn the mapping relationship between the query position and the sub-value feature position and the sub-key feature position, realizing cross-modal positional relationship adaptive calibration between the requirement feature and the associated reply word feature, further reducing the storage resource occupancy of the calculation unit, and enhancing the storage capacity and computing power adaptability of the computing device, achieving efficient utilization of storage resources and computing resources during the process of performing the target processing task for the large model.

[0087] Figure 3 Schematically shows the principle diagram of the task execution method based on the large model according to an embodiment of the present disclosure.

[0088] Such as Figure 3As shown in the figure, the task execution method based on the large model can execute the target processing task by using the computing unit based on the feature extraction network 310 and the large model 320. The feature extraction network 310 may include N cascaded feature extraction units 311, and the feature extraction unit 311 may include a basic self-attention layer 3111 and a basic feed-forward layer 3112. The large model 320 includes M attention feature fusion units 321. The attention feature fusion unit 321 includes a self-attention layer 3211 constructed based on the self-attention network algorithm, a cross-attention layer 3212 constructed based on the attention network algorithm, and a feed-forward layer 3213. The cross-attention layer 3212 may be constructed based on the multi-head attention network algorithm, and the cross-attention layer 3212 may include multiple attention heads. It should be noted that both N and M are arbitrary integers greater than 1.

[0089] The demand information 301 may be information in any modality such as audio, text, image, etc. The demand information 301 is input into the feature extraction network 310. Using the attention mechanism, the computing unit performs multi-level feature extraction on the demand information 301 based on the model parameters of the N cascaded feature extraction units 311, and deeply fuses the context semantics of the demand information 301 to obtain the demand feature F301 representing the demand intention. The demand feature F301 is written into the storage unit for the computing unit, so that when the computing unit executes the target processing task based on the M attention feature fusion units 321 of the large model 320, it can execute multiple attention subtasks by reusing the demand feature F301 in the storage unit as the value feature and the key feature, and obtain the response text 302.

[0090] For example, when there are p target response words in the current response text, for the (p + 1)-th target response word to be generated, the p target response words are used as associated response words, and the first initial associated response word feature is determined based on the response word features used to generate each of the p target response words. For example, the response word features of each of the p target response words can be fused to obtain the first initial associated response word feature. The first initial associated response word feature is input into the self-attention layer 3211 of the first attention feature fusion unit 321, and the computing unit performs attention fusion on the initial associated response word feature based on the model parameters of the self-attention layer 3211, so that the obtained first associated response word feature can fully fuse the semantic information of the context before the (p + 1)-th target response word in the response text. The first associated response word feature is input into the cross-attention layer 3212 of the first attention feature fusion unit 321. The computing unit processes the first associated response word feature based on the model parameters of each of the multiple attention heads in the cross-attention layer 3212 to obtain multiple query features for the multiple attention heads. The requirement feature is used as the value feature and the key feature, and the computing unit performs an attention subtask on the query feature, the value feature, and the key feature based on the model parameters of the attention head to obtain the sub-execution result output by the attention head. Multiple attention heads in the cross-attention layer 3212 of the first attention feature fusion unit 321 can reuse the same requirement feature as the value feature and the key feature to perform the attention subtasks corresponding to each of the multiple attention heads, and obtain multiple sub-execution results. By fusing the multiple sub-execution results, the first intermediate fusion feature output by the cross-attention layer 3212 of the first attention feature fusion unit 321 can be obtained. The first intermediate fusion feature is input into the feed-forward layer 3213 of the first attention feature fusion unit 321, and the computing unit performs an arithmetic operation on the first intermediate fusion feature based on the feed-forward layer 3213 of the first attention feature fusion unit 321 to obtain the second initial associated response word feature. The second initial associated response word feature is input into the second attention feature fusion unit, and the computing unit can reuse the requirement feature F301 stored in the storage unit to perform multiple attention subtasks to obtain the third initial associated response word feature.

[0091] It should be understood that when the (m - 1)-th initial associated response word feature is input into the (m - 1)-th attention feature fusion unit, the computing unit can reuse the requirement feature F301 stored in the storage unit to perform multiple attention subtasks to obtain the m-th initial associated response word feature. Where M ≥ m > 1, when M = m, the M-th initial associated response word feature can be used as the target response word feature for generating the (p + 1)-th target response word. The (p + 1)-th target response word is generated based on the target response word feature of the (p + 1)-th target response word. Thus, the response text 302 output by the large model 320 can be obtained by iteratively generating multiple target response words.

[0092] According to an embodiment of the present disclosure, determining query features for respective multiple attention subtasks in a target processing task based on associated response word features includes: processing the associated response word features by a computing unit based on query weights and key weights for the attention subtasks to obtain query features of the attention subtasks.

[0093] According to an embodiment of the present disclosure, a computing unit may be utilized to perform a weighted calculation operation on the associated response word features based on a combined result of query weights and key weights to obtain query features of the attention subtasks.

[0094] For example, the query features of the attention subtasks may be obtained based on the following formula (4)

[0095] Q head = Wq * W k * X (4);

[0096] where Wq is the query weight, W k is the key weight, and X is the associated response word feature.

[0097] By performing a weighted calculation operation on the associated response word features by combining the query weights and key weights to obtain query features of the attention subtasks, the query features of respective multiple attention subtasks can be stored in a storage unit, and the computing unit can reuse the same required features as value features and key features from the storage unit to perform the attention subtasks, enabling multiple query features to correspond to the same value features and key features, so as to reduce the computational overhead and storage space occupied by calculating value features and key features for respective attention subtasks by performing weighted calculations on the required features, improve the execution efficiency of the computing unit in performing multiple attention subtasks, and further improve the target processing efficiency.

[0098] It should be noted that the query weights, value weights, and key weights for the attention subtasks involved in the embodiments of the present disclosure may be model parameters of a large model obtained by training the large model. The method provided in the embodiments of the present disclosure may utilize the computing unit to call the value weights and key weights to process the associated response word features, so as to avoid storing value features and key features of respective multiple attention subtasks by combining the weight calculation processes.

[0099] According to an embodiment of the present disclosure, the multiple query features include a first query feature and a second query feature, and the multiple attention subtasks include a first subtask and a second subtask. The first query feature and the second query feature may be respectively used for the first subtask and the second subtask.

[0100] According to an embodiment of the present disclosure, using a computing unit to perform multiple attention subtasks according to multiple query features, value features, and key features includes: based on the value features and key features, using the computing unit to perform a first subtask according to a first query feature to obtain a first sub-execution result; in the case of reusing the value features and key features, using the computing unit to perform a second subtask according to a second query feature to obtain a second sub-execution result; and using the computing unit to fuse the first sub-execution result and the second sub-execution result to obtain a target response word.

[0101] According to an embodiment of the present disclosure, based on the value features and key features, using the computing unit to perform a first subtask according to a first query feature may include using the computing unit to process the first query feature Q based on an attention algorithm head1 , and using the demand features read from the storage unit as the value features and key features to obtain a first execution result O of the first subtask head1 . When using the computing unit to perform the second subtask, the second subtask can be executed by reusing the same value features and key features by reading the same demand features as the value features and key features used for the first subtask from the storage unit. Using the computing unit to process the second query feature Q head2 , value features, and key features to obtain a first execution result O of the second subtask head2 .

[0102] Figure 4 Schematically shows a schematic diagram of the principle of performing multiple attention subtasks.

[0103] As Figure 4 shown, the associated response word feature F401 may be obtained by the self-attention layer of the large model performing attention fusion on the initial associated response word features for multiple associated response words. The demand feature F410 may be obtained by performing feature extraction on the demand information. The demand feature F410 may include a plurality of sub-demand features arranged in sequence. The demand feature F410 may be stored in the storage unit.

[0104] Input the associated response word feature F401 into the first attention head 411, the second attention head 412, ……, the i-th attention head 41i, so as to implement using the computing unit to execute i attention subtasks according to the first attention head 411, the second attention head 412, ……, the i-th attention head 41i. Where i is any integer greater than 1. The associated response word feature F401 is input into the first weight fusion sublayer of the first attention head 411, and the computing unit can perform a weighted operation on the associated response word feature F401 based on the first query weight and the first key weight of the first weight fusion sublayer to obtain the first query feature. The demand feature F410 read from the storage unit is used as the value feature and the key feature and input into the first attention sublayer, and the first query feature is input into the first attention sublayer. The computing unit can perform an attention calculation operation on the demand feature F410 and the first query feature based on the model parameters of the first attention sublayer to obtain the first sub-execution result.

[0105] As Figure 4 shown, input the associated response word feature F401 into the second weight fusion sublayer of the second attention head 411, and the computing unit can perform a weighted operation on the associated response word feature F401 based on the second query weight and the second key weight of the second weight fusion sublayer to obtain the second query feature. The demand feature F410 read from the storage unit is used as the value feature and the key feature and input into the second attention sublayer, and the second query feature is input into the second attention sublayer. The computing unit can perform an attention calculation operation on the demand feature F410 and the second query feature based on the model parameters of the second attention sublayer to obtain the second sub-execution result.

[0106] Input the associated response word feature F401 into the i-th weight fusion sublayer of the i-th attention head 411, and the computing unit can perform a weighted operation on the associated response word feature F401 based on the i-th query weight and the i-th key weight of the i-th weight fusion sublayer to obtain the i-th query feature. The demand feature F410 read from the storage unit is used as the value feature and the key feature and input into the i-th attention sublayer, and the i-th query feature is input into the i-th attention sublayer. The computing unit can perform an attention calculation operation on the demand feature F410 and the i-th query feature based on the model parameters of the i-th attention sublayer to obtain the i-th sub-execution result. Thus, multiple attention subtasks can reuse the demand feature in the storage unit to execute multiple attention subtasks, so as to improve the processing efficiency of the target processing task of the large model.

[0107] According to an embodiment of the present disclosure, the sub-execution results of multiple attention subtasks each include a first sub-execution result and a second sub-execution result; using a computing unit to fuse the first sub-execution result and the second sub-execution result to obtain a target reply word includes: based on the conversion weight and value weight read from a storage unit, using the computing unit to perform weighted calculation on the multiple sub-execution results to obtain an intermediate fusion feature; and using the computing unit to determine the target reply word based on the intermediate fusion feature.

[0108] According to an embodiment of the present disclosure, using a computing unit based on the conversion weight W O and the value weight W v , perform a weighted calculation operation on the concatenation result of the sub-execution results of multiple attention subtasks to perform a linear transformation on the multiple sub-execution results, obtain the final output result U of performing multiple attention subtasks, and use the final output result U of performing multiple attention subtasks as the intermediate fusion feature.

[0109] For example, the intermediate fusion feature can be determined based on the following formula (5).

[0110] U = W O * W v * [O1; O2; O3; ……Oi;] (5);

[0111] where U is the final output result of multiple attention subtasks as the intermediate fusion feature, W v is the value weight, and Oi is the i-th sub-execution result of the i-th attention subtask.

[0112] According to the task execution method based on a large model provided by an embodiment of the present disclosure, it is possible to reuse the demand features in the storage unit as value features and key features to perform multiple attention subtasks, realize that multiple different query features form a mapping relationship with the same value features and key features, so as to realize that when the number of attention subtasks increases, the storage space occupied by value features and key features is not increased, thereby reducing the storage resource occupancy of the computing unit. Thus, under the condition of retaining the multi-head attention mechanism to perform multi-head attention calculation on demand features and associated reply word features, it is possible to avoid redundant storage resource occupancy caused by separately storing the value features and key features of multiple attention subtasks respectively, and realize a reduction in storage space occupancy by dozens of times. This can enable the large model to more effectively reduce the resource occupancy of performing target processing tasks when deployed to electronic devices in scenarios such as intelligent e-commerce, so that the large model can be deployed on computing devices with lower computing power performance, and reduce the deployment cost of the large model for generating reply texts.

[0113] Meanwhile, by avoiding performing the corresponding value feature calculation and key feature calculation operations for each attention subtask during the execution of multiple attention subtasks, the computational complexity generated by the computing unit in performing the target processing task based on the large model can be reduced, thereby reducing the load pressure on the computing unit caused by the large model in performing the attention calculation task, optimizing the computing power consumption level of the computing unit during the process of performing the target processing task based on the large model, and further improving the generation efficiency of the response text.

[0114] In one example, the demand information may be the demand voice input by the user based on the terminal device, and the response text may be generated based on the task execution method based on the large model provided by the embodiments of the present disclosure.

[0115] According to an embodiment of the present disclosure, the demand feature includes a demand speech recognition feature, and the demand speech recognition feature is determined based on the following operations: extracting features from the demand speech of the target object to obtain an initial speech feature; decoding the initial speech feature to obtain a plurality of initial decoding features; and fusing the plurality of initial decoding features and the initial speech feature based on the attention mechanism to obtain the demand speech recognition feature.

[0116] According to an embodiment of the present disclosure, the initial decoding feature represents a demand speech frame in the demand speech, the demand speech frame represents a demand word, and the demand speech frame may be speech frame data obtained by pronouncing the demand word. The initial decoding feature represents the recognition result of the demand word in the demand speech. For example, the initial decoding feature may be a first speech recognition result corresponding to the demand word in the text expressed by the demand speech, and the plurality of initial decoding features correspond one by one to the plurality of demand words represented by the demand speech.

[0117] According to an embodiment of the present disclosure, any type of attention network algorithm such as Transformer may be used to process the initial decoding feature and the initial speech feature to obtain the speech recognition feature.

[0118] According to an embodiment of the present disclosure, the attention mechanism fuses the plurality of initial decoding features and the initial speech feature, so that the speech recognition feature can include the text semantics of the demand word corresponding to the speech segment of the demand speech, as well as various types of speech attributes such as the speech rate, intonation, and emotional attributes represented by the demand speech. When the speech recognition feature accurately represents the text semantics of the demand text, it can also include rich speech attributes, improving the expression accuracy of the speech recognition feature for the demand intention.

[0119] According to an embodiment of the present disclosure, fusing multiple initial decoding features and initial speech features based on an attention mechanism to obtain demand features includes: fusing the initial decoding features and the initial speech features based on the attention mechanism to obtain demand word audio features corresponding to demand speech frames; globally fusing the multiple demand word audio features to obtain intermediate speech features; and fusing the intermediate speech features and the multiple initial decoding features based on the attention mechanism to obtain demand speech recognition features.

[0120] According to an embodiment of the present disclosure, fusing the initial decoding features and the initial speech features based on the attention mechanism can use the initial decoding features corresponding to each demand word as prior information, and fully fuse the initial decoding features corresponding to each demand word with the audio segments corresponding to the initial speech features, so that the demand word audio features can more accurately represent the text semantics and speech attributes of the audio segments corresponding to the demand words.

[0121] According to an embodiment of the present disclosure, globally fusing the multiple demand word audio features may include fusing the multiple demand word audio features based on an attention network algorithm, and the obtained intermediate speech features may include intermediate speech word features corresponding to multiple demand words. From the audio information of the demand speech at the frame level, unequal-length speech feature information is extracted to obtain word-level equal-length unified audio features, which are represented as the initial speech features of the demand speech. Using the multiple initial decoding features as priors to fuse the initial speech features and the multiple initial decoding features can overcome the defect that the feature representation lengths of speech frames are inconsistent, and improve the accuracy of speech recognition.

[0122] According to an embodiment of the present disclosure, fusing the intermediate speech features and the multiple initial decoding features based on the attention mechanism to obtain speech recognition features may include using the initial decoding features as speech query features, and using the intermediate speech features as speech key features and speech value features of the attention algorithm. By fusing the speech query features, speech key features, and speech value features based on the attention algorithm, speech recognition features are obtained. The speech recognition features can represent the text semantics and speech attributes represented by the speech audio segments corresponding to the demand words. By inputting the speech recognition features into multiple attention fusion layers of a large model to achieve multi-modal feature fusion, the demand speech in the audio modality can directly perform calculations with the associated reply word features after modality conversion, avoiding performing modality conversion calculations such as feature embedding on the demand speech in each network structure of the large model, and enabling in-depth calculations between different modality information. In this way, the speech recognition large model and multiple levels of attention fusion layers can be used to more efficiently align the information represented by the speech modality and the text modality, and improve the response speed of the large model to reply to the demand speech.

[0123] Figure 5 Schematically shows an application scenario diagram of a task execution method based on a large model according to an embodiment of the present disclosure. InFigure 5 In the illustrated embodiment, the demand information may be demand voice 501,

[0124] such as Figure 5 shown, this application scenario may include a speech recognition large model 510 and a large model 520. The large model 520 may include a cascaded first feature fusion network 521 and a second feature fusion network 522. The first feature fusion network 521 includes a cascaded first text feature fusion layer, a first cross-attention fusion layer, and a first feed-forward layer. The second feature fusion network 522 includes a cascaded second text feature fusion layer, a second cross-attention fusion layer, and a second feed-forward layer. The first cross-attention fusion layer and the second cross-attention fusion layer may be constructed based on the multi-head attention network algorithm. In the first cross-attention fusion layer, multiple attention subtasks for the first cross-attention fusion layer may be executed through the multi-head attention sublayer, and in the second cross-attention fusion layer, multiple attention subtasks for the second cross-attention fusion layer may be executed through the multi-head attention sublayer.

[0125] The demand voice 501 may be voice audio information representing the demand text "How to make pickled fish with pickled cabbage". The demand voice 501 is input into the speech recognition large model 510 to perform speech recognition on the demand voice 501, and a demand speech recognition feature is output. This demand speech recognition feature may be stored in the storage unit as the value feature and the key feature for the first cross-attention fusion layer and the second cross-attention fusion layer.

[0126] The initial associated reply word feature F501 may be related to the four reply words "of pickled fish with pickled cabbage" in the already generated reply text.

[0127] The initial associated reply word feature F501 is input into the first text feature fusion layer, and the computing unit may perform self-attention fusion on the initial associated reply word feature F501 based on the model parameters of the first text feature fusion layer to output the first associated reply word feature. The first associated reply word feature and the speech recognition feature are input into the first cross-attention fusion layer, and the computing unit may use the speech recognition feature as the value feature and the key feature, and determine the query features of each of the multiple attention heads in the first cross-attention fusion layer according to the first associated reply word feature to execute multiple attention subtasks, and output the first intermediate fusion feature. The first intermediate fusion feature is processed by the cascaded first feed-forward layer and the second text feature fusion layer to obtain the second associated reply word feature.

[0128] Input the second associated reply word feature and the speech recognition feature into the second cross-attention fusion layer. The computing unit can use the speech recognition feature read from the storage unit as the value feature and the key feature, and determine the query feature of each of the multiple attention heads in the second cross-attention fusion layer according to the second associated reply word feature to perform multiple attention subtasks, and output the second intermediate fusion feature. Thus, it is realized to perform the attention subtasks of multiple multi-head attention layers in the large model by reusing the required speech recognition feature in the storage unit. By inputting the second intermediate fusion feature into the second feed-forward layer, the target reply word feature F5011 is output. The target reply word feature F5011 can represent the target reply word "do".

[0129] It should be understood that for the target reply word "do", the five reply words "how to cook pickled fish" in the reply text 502 can be used as the associated reply words, and the associated reply word features related to "how to cook pickled fish" are processed based on the large model-based task execution method provided by the embodiments of the present disclosure to obtain a new target reply word. Until all the reply words arranged in sequence are generated to obtain the reply text 502.

[0130] According to the large model-based task execution method of the embodiments of the present disclosure, by using the required features extracted from the required information as the value feature and the key feature for the computing unit to perform multiple attention subtasks, it is possible to realize performing multiple attention subtasks representing the multi-head attention mechanism based on reusing the same value feature and key feature in the storage unit, and it is possible to make full use of the multiple query features in the multi-head attention mechanism to capture diverse information from the required features while reducing the problem of large storage space occupation existing in the multi-head attention mechanism. At the same time, it can enable multiple network layers constructed based on the attention mechanism in the relatively complex model structure of the large model to perform attention tasks based on the same value feature and key feature, thereby avoiding the computing unit from performing value feature and key feature calculations for each attention task, and realizing meeting the semantic understanding and text generation requirements of attention tasks based on the shared value feature and key feature stored singly in the storage space; at the same time, by using the position information implicitly contained in the arrangement order of multiple sub-required features in the required features to perform attention tasks, it solves the defect of unstable training or inference performance caused by the mismatch of feature position encoding in cross-model attention tasks, and realizes performing a generation task by executing an Efficient All Query Attention mechanism for cross-modal required features based on the large model, improving the execution efficiency of performing target processing tasks based on large language models and other large models, and reducing the computing overhead of the computing unit and the storage space occupation of the storage unit for performing target processing tasks based on the large model.

[0131] According to an embodiment of the present disclosure, the method for task execution based on a large model further includes: obtaining requirement information of a target object; extracting features from the requirement information to obtain requirement features; and pushing a response text to the target object.

[0132] According to an embodiment of the present disclosure, the requirement information may include information in any modality such as requirement voice, requirement text, requirement image, etc. input by the target object through an intelligent terminal such as a mobile phone. By extracting features from the requirement information and processing the requirement features based on the method provided in the embodiment of the present disclosure, a response text matching the requirement intention of the target object can be obtained. The response is made by pushing the response text to the intelligent terminal device of the target object.

[0133] According to an embodiment of the present disclosure, the response text pushed to the target object may be pushed to the intelligent terminal of the target object based on character formats such as text and code, or may also be pushed to the intelligent terminal of the target object based on other data formats such as voice. The embodiment of the present disclosure does not limit the specific data format of the pushed response text.

[0134] Figure 6 A block diagram of a task execution device based on a large model according to an embodiment of the present disclosure is schematically shown.

[0135] As Figure 6 shown, the task execution device 600 based on a large model includes: an acquisition module 610 and a response text acquisition module 620.

[0136] The acquisition module 610 is configured to acquire requirement features representing a requirement intention.

[0137] The response text acquisition module 620 is configured to use a large model to perform a target processing task according to the requirement features to obtain a response text matching the requirement intention. Among them, the target response words in the response text are determined based on the following method: determining query features for each of multiple attention subtasks in the target processing task based on associated response word features, where the associated response word features are determined based on associated response words in the response text; using the requirement features read from the storage unit as value features and key features shared by the multiple attention subtasks, and using a calculation unit to perform the multiple attention subtasks according to the multiple query features, value features and key features to obtain the target response words.

[0138] According to an embodiment of the present disclosure, a plurality of query features include a first query feature and a second query feature, and a plurality of attention subtasks include a first subtask and a second subtask; a computing unit is configured to perform the following operations to execute the plurality of attention subtasks according to the plurality of query features, the value feature, and the key feature: based on the value feature and the key feature, execute the first subtask according to the first query feature to obtain a first subtask execution result; in the case of reusing the value feature and the key feature, execute the second subtask according to the second query feature to obtain a second subtask execution result; and fuse the first subtask execution result and the second subtask execution result to obtain a target response word.

[0139] According to an embodiment of the present disclosure, the subtask execution results of the plurality of attention subtasks respectively include a first subtask execution result and a second subtask execution result; the computing unit is configured to perform the following operations to fuse the first subtask execution result and the second subtask execution result to obtain a target response word: based on the transformation weight and the value weight read from the storage unit, perform a weighted calculation on the plurality of subtask execution results by using the computing unit to obtain an intermediate fusion feature; and use the computing unit to determine the target response word based on the intermediate fusion feature.

[0140] According to an embodiment of the present disclosure, the computing unit is configured to perform the following operations to implement determining the query features for the respective attention subtasks in the target processing task based on the associated response word feature: based on the query weight and the key weight for the attention subtask, process the associated response word feature to obtain the query feature of the attention subtask.

[0141] According to an embodiment of the present disclosure, the requirement feature includes a plurality of sub-requirement features arranged in sequence; the computing unit is configured to perform the following operations to implement executing a plurality of attention subtasks according to the plurality of query features, the value feature, and the key feature: based on the arrangement order between the plurality of sub-requirement features in the key feature, perform a first calculation operation based on the plurality of sub-requirement features and the plurality of sub-query features in the query feature to obtain an attention weight for the attention subtask; based on the arrangement order between the plurality of sub-requirement features in the value feature, perform a second calculation operation based on the attention weight and the plurality of sub-requirement features to obtain a subtask execution result of the attention subtask, wherein the target response word is determined based on the subtask execution results of the plurality of attention subtasks.

[0142] According to an embodiment of the present disclosure, the requirement feature is determined by performing feature extraction on the requirement information of the target object, and the arrangement order of the plurality of sub-requirement features in the requirement feature is determined according to the arrangement order of the plurality of sub-requirement information in the requirement information; wherein, the sub-requirement information includes at least one of the following:

[0143] Requirement words in the requirement text, requirement speech frames in the requirement speech, and image blocks in the requirement image.

[0144] According to an embodiment of the present disclosure, the response text includes a plurality of response words arranged in sequence, and a plurality of associated response words are arranged before the target response word; the associated response word feature is obtained by performing feature fusion on the plurality of associated response words based on an attention mechanism.

[0145] According to an embodiment of the present disclosure, the demand feature includes a demand speech recognition feature, and the demand speech recognition feature is determined based on the following operations: performing feature extraction on the demand speech of the target object to obtain an initial speech feature; decoding the initial speech feature to obtain a plurality of initial decoding features, where the initial decoding features represent demand speech frames in the demand speech, and the demand speech frames represent demand words; and fusing the plurality of initial decoding features and the initial speech feature based on an attention mechanism to obtain the demand speech recognition feature.

[0146] According to an embodiment of the present disclosure, fusing the plurality of initial decoding features and the initial speech feature based on an attention mechanism to obtain the demand feature includes: fusing the initial decoding feature and the initial speech feature based on an attention mechanism to obtain a demand word audio feature corresponding to the demand speech frame; performing global feature fusion on the plurality of demand word audio features to obtain an intermediate speech feature; and fusing the intermediate speech feature and the plurality of initial decoding features based on an attention mechanism to obtain the demand speech recognition feature.

[0147] According to an embodiment of the present disclosure, the demand feature includes a demand text feature, and the demand text feature is determined based on the following operations: performing first-level semantic feature extraction on the demand text of the target object to obtain a first-level intermediate demand text feature, where the first-level intermediate demand text feature includes a plurality of first-level intermediate sub-demand features corresponding to a plurality of demand words in the demand text; performing n-level semantic feature extraction on the (n - 1)-level intermediate demand text feature to obtain an n-level intermediate demand text feature including a plurality of n-level intermediate sub-demand features; and when N = n, determining the N-level intermediate demand text feature as the demand text feature, where a plurality of sub-demand features in the demand text feature correspond to a plurality of demand words, N ≥ n > 1, and N and n are integers.

[0148] According to an embodiment of the present disclosure, the demand feature includes a demand image feature, and the demand image feature is determined based on the following operations: performing first-level demand image feature extraction on the demand image of the target object to obtain a first-level intermediate demand image feature, where the first-level intermediate demand image feature includes a plurality of first-level intermediate sub-demand features corresponding to a plurality of image blocks in the demand image; performing n-level demand image feature extraction on the (n - 1)-level intermediate demand image feature to obtain an n-level intermediate demand image feature including a plurality of n-level intermediate sub-demand features; and when N = n, determining the N-level intermediate demand image feature as the demand image feature, where a plurality of sub-demand features in the demand image feature correspond to a plurality of image blocks, N ≥ n > 1, and N and n are integers.

[0149] According to an embodiment of the present disclosure, the large model-based task execution device 600 further includes: a requirement information acquisition module, a feature extraction module, and a push module.

[0150] The requirement information acquisition module is configured to acquire the requirement information of the target object.

[0151] The feature extraction module is configured to extract features from the requirement information to obtain requirement features.

[0152] The push module is configured to push a reply text to the target object.

[0153] Figure 7 Schematically shows a structural block diagram of an intelligent agent of artificial intelligence according to an embodiment of the present disclosure.

[0154] In an embodiment of the present disclosure, as Figure 7 shown, the AI intelligent agent 700 may include an input module 710, a processing module 720, and an output module 730.

[0155] The input module 710 is configured to receive input information;

[0156] The processing module 720 is configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling a large language model to execute the large model-based task execution method provided according to an embodiment of the present disclosure;

[0157] The output module 730 is configured to output the output information obtained by the processing module.

[0158] According to an embodiment of the present disclosure, the input module 710 is responsible for receiving or perceiving information such as queries, requests, instructions, signals, or data from the outside world (such as users or the external environment), and converting it into a format that the AI intelligent agent 700 can understand and process. The input module 710 is the primary link for the AI intelligent agent 700 to interact with the outside world. It enables the AI intelligent agent 700 to efficiently and accurately obtain necessary "sensory" information from the outside world and respond to this information.

[0159] In an example, the input module 710 may input the requirement features or requirement voices, requirement texts, requirement images, requirement information, etc. described above.

[0160] In an example, the processing module 720 is the core support for the AI intelligent agent 700 to handle complex tasks. The processing module 720 may execute the large model-based task execution method described above.

[0161] In the example, the performance of the processing module 720 can be closely related to the large model on which the AI agent 700 is based. To fully utilize the capabilities of the large model, the internal structure of the processing module 720 can be designed to be highly configurable and extensible to handle various different types of tasks and requirements in real-world scenarios.

[0162] In the example, after the AI agent 700 obtains the requirement features, the processing module 720 can use the large model to process the requirement features, obtain the response text, and pass the response text to the output module 730.

[0163] It can be understood that although large language models have excellent language understanding and generation capabilities, like humans, without the aid of any tools, the tasks they can solve are very limited. When the AI agent 700 is given the ability to call tools, it can perform tasks such as completing mathematical operations with the help of a calculator, performing data analysis with the help of Python, and obtaining weather forecasts with the help of a search engine.

[0164] In the example, the output module 730 can output the response text described above.

[0165] According to the embodiments of the present disclosure, the AI agent 700 can simply and effectively improve the degree of intelligence, and enhance flexibility and versatility.

[0166] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0167] According to the embodiments of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0168] According to the embodiments of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0169] According to the embodiments of the present disclosure, a computer program product includes a computer program, and the computer program, when executed by a processor, implements the method as described above.

[0170] Figure 8FIG. 0 shows a schematic block diagram of an electronic device 800 for a large model-based task execution method that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0171] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0172] Multiple components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disc, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0173] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the method for task execution based on a large model. For example, in some embodiments, the method for task execution based on a large model can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method for task execution based on a large model described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the method for task execution based on a large model in any other suitable manner (e.g., by means of firmware).

[0174] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), system-on-chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. This program code can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program code is executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0176] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0177] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0178] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0179] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0180] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0181] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A task execution method based on a large model, comprising: Obtain demand features that characterize demand intent; The target processing task is performed according to the demand characteristics by using the large model to obtain a reply text that matches the demand intention, wherein the target reply word in the reply text is determined based on the following method: Determining query features for each of the plurality of attention subtasks in the target processing task based on associated reply word features, wherein the associated reply word features are determined based on associated reply words in the reply text; Based on the requirement features read from the storage unit as value features and key features shared by multiple attention subtasks, a computing unit is used to execute multiple attention subtasks according to multiple query features, value features and key features to obtain the target reply word.

2. The method according to claim 1, wherein: The plurality of query features include a first query feature and a second query feature, and the plurality of attention subtasks include the first subtask and the second subtask; The using a computing unit to perform the plurality of attention subtasks according to the plurality of query features, the value features and the key features comprises: Based on the value feature and the key feature, using the computing unit to execute the first subtask according to the first query feature to obtain a first sub-execution result; In the case of reusing the value feature and the key feature, using the computing unit to execute the second subtask according to the second query feature to obtain a second sub-execution result; and The first sub-execution result and the second sub-execution result are merged by using the calculation unit to obtain the target reply word.

3. The method according to claim 2, wherein: The sub-execution results of each of the plurality of attention sub-tasks include the first sub-execution result and the second sub-execution result; and the using the computing unit to fuse the first sub-execution result and the second sub-execution result to obtain the target reply word includes: Based on the conversion weight and the value weight read from the storage unit, using the calculation unit to perform weighted calculation on the plurality of sub-execution results to obtain an intermediate fusion feature; and The target reply word is determined by the computing unit based on the intermediate fusion feature.

4. The method according to any one of claims 1 to 3, wherein: The determining of query features for each of the plurality of attention subtasks in the target processing task based on the associated reply word features comprises: Based on the query weight and key weight for the attention subtask, the associated reply word features are processed by the computing unit to obtain the query features of the attention subtask.

5. The method according to claim 1, wherein: The demand feature includes a plurality of sub-demand features arranged in sequence; The using a computing unit to perform the plurality of attention subtasks according to the plurality of query features, the value features and the key features comprises: Based on the arrangement order between the plurality of sub-requirement features in the key feature, using the calculation unit to perform a first calculation operation based on the plurality of sub-requirement features and the plurality of sub-query features in the query feature, to obtain an attention weight for the attention subtask; Based on the arrangement order between the multiple sub-requirement features in the value feature, the calculation unit is used to perform a second calculation operation based on the attention weight and the multiple sub-requirement features to obtain the sub-execution result of the attention subtask, wherein the target reply word is determined based on the sub-execution results of the multiple attention subtasks.

6. The method according to claim 5, wherein: The demand feature is determined by extracting the demand information of the target object, and the arrangement order of the multiple sub-demand features in the demand feature is determined according to the arrangement order of the multiple sub-demand information in the demand information; wherein the sub-demand information includes at least one of the following: The required words in the required text, the required speech frames in the required speech, and the required image blocks in the required image.

7. The method according to claim 1, wherein: The reply text includes multiple reply words arranged in order, and the multiple related reply words are arranged before the target reply word; the related reply word feature is obtained by fusing the features of the multiple related reply words based on the attention mechanism.

8. The method according to claim 1, wherein: The demand feature includes a demand voice recognition feature, and the demand voice recognition feature is determined based on the following operations: Extract features of the target object's required speech to obtain initial speech features; Decoding the initial speech features to obtain a plurality of initial decoded features, wherein the initial decoded features represent a required speech frame in the required speech, and the required speech frame represents a required word; Based on the attention mechanism, a plurality of the initial decoding features and the initial speech features are fused to obtain the required speech recognition features.

9. The method according to claim 8, wherein: The requirement feature is obtained by fusing the plurality of the initial decoding features and the initial speech features based on the attention mechanism, including: Based on the attention mechanism, the initial decoding feature and the initial speech feature are fused to obtain the required word audio feature corresponding to the required speech frame; Performing global feature fusion on the audio features of the plurality of required words to obtain an intermediate speech feature; The intermediate speech feature and the plurality of initial decoding features are fused based on the attention mechanism to obtain the required speech recognition feature.

10. The method according to claim 1, wherein: The requirement feature includes a requirement text feature, and the requirement text feature is determined based on the following operations: Extracting first-level semantic features from the target object's demand text to obtain first-level intermediate demand text features, wherein the first-level intermediate demand text features include a plurality of first-level intermediate sub-demand features corresponding to a plurality of demand words in the demand text; Performing n-level semantic feature extraction on the n-1-level intermediate demand text feature to obtain the n-level intermediate demand text feature including multiple n-level intermediate sub-demand features; In the case of N=n, the Nth level intermediate requirement text feature is determined as the requirement text feature, wherein a plurality of sub-requirement features in the requirement text feature correspond to a plurality of the requirement words, N≥n>1, and N and n are integers.

11. The method according to claim 1, wherein: The demand feature includes a demand image feature, and the demand image feature is determined based on the following operations: Extracting first-level demand image features from a demand image of a target object to obtain first-level intermediate demand image features, wherein the first-level intermediate demand image features include a plurality of first-level intermediate sub-demand features corresponding to a plurality of image blocks in the demand image; Extracting the n-th level demand image feature from the n-1-th level intermediate demand image feature to obtain the n-th level intermediate demand image feature including a plurality of n-th level intermediate sub-demand features; In the case of N=n, the Nth level intermediate required image feature is determined as the required image feature, wherein a plurality of sub-requirement features in the required image feature correspond to a plurality of the image blocks, N≥n>1, and N and n are integers.

12. The method according to claim 1, wherein: The method further comprises: Acquire demand information of the target object; Extracting features from the demand information to obtain the demand features; and The reply text is pushed to the target object.

13. A task execution device based on a large model, comprising: An acquisition module is used to acquire demand features that represent demand intent; The reply text obtaining module is used to use the large model to perform the target processing task according to the demand characteristics to obtain a reply text that matches the demand intention, wherein the target reply word in the reply text is determined based on the following method: Determining query features for each of the plurality of attention subtasks in the target processing task based on associated reply word features, wherein the associated reply word features are determined based on associated reply words in the reply text; Based on the requirement features read from the storage unit as value features and key features shared by multiple attention subtasks, a computing unit is used to execute multiple attention subtasks according to multiple query features, value features and key features to obtain the target reply word.

14. The device according to claim 13, wherein: The plurality of query features include a first query feature and a second query feature, and the plurality of attention subtasks include the first subtask and the second subtask; and the computing unit is configured to perform the following operations to perform the plurality of attention subtasks according to the plurality of query features, the value features, and the key features: Based on the value feature and the key feature, executing the first subtask according to the first query feature to obtain a first sub-execution result; In the case of reusing the value feature and the key feature, executing the second subtask according to the second query feature to obtain a second sub-execution result; and The first sub-execution result and the second sub-execution result are integrated to obtain the target reply word.

15. The device according to claim 13, wherein: The sub-execution results of each of the plurality of attention sub-tasks include the first sub-execution result and the second sub-execution result; the computing unit is configured to perform the following operations to fuse the first sub-execution result and the second sub-execution result to obtain the target reply word: Based on the conversion weight and the value weight read from the storage unit, weighted calculation is performed on the plurality of sub-execution results to obtain an intermediate fusion feature; as well as The target reply word is determined based on the intermediate fusion feature.

16. The device according to any one of claims 13 to 15, wherein: The computing unit is configured to perform the following operations to determine query features for each of the plurality of attention subtasks in the target processing task based on the associated reply word features: Based on the query weight and key weight for the attention subtask, the associated reply word feature is processed to obtain the query feature of the attention subtask.

17. An artificial intelligence agent, comprising: An input module, used for receiving input information; a processing module, configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and obtain output information by calling the large model to execute the method according to any one of claims 1 to 12; An output module is used to output the output information obtained by the processing module.

18. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images