Methods, apparatus, devices, media, and products for task execution in large-scale models.

Abstract

This disclosure discloses a task execution method for large-scale models, relating to the field of artificial intelligence technology, particularly deep learning technology and large-scale model technology. The specific implementation scheme includes: based on the target features to be processed, performing a modality routing task using a target computing unit to obtain modality recognition results; based on the target features to be processed and the target domain gating model parameters, performing a domain routing task using a target computing unit to obtain domain recognition results; and based on the target features to be processed and the target feedforward task model parameters, performing a feedforward task using a target computing unit to obtain task execution results. This disclosure also provides a task execution method, apparatus, electronic device, storage medium, and program product for large-scale models.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of deep learning technology and large model technology. Specifically, it relates to methods, apparatuses, electronic devices, storage media, and program products for performing tasks on large models. Background Technology

[0002] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. AI generally includes large-scale modeling techniques. These techniques can be widely applied across various areas of AI, such as text processing, semantic understanding, machine translation, and human-computer interaction. Utilizing large-scale models to perform tasks in different domains requires careful consideration of factors such as timeliness and cost. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, storage medium, and program product for performing tasks on large models.

[0004] According to one aspect of this disclosure, a task execution method for a large model is provided, comprising: performing a modal routing task using a target computing unit based on target features to be processed, to obtain a modal recognition result; performing a domain routing task using the target computing unit based on the target features to be processed and target domain gating model parameters, to obtain a domain recognition result, wherein the target domain gating model parameters are domain gating model parameters corresponding to the modal recognition result read from a target storage unit; and performing a feedforward task using the target computing unit based on the target features to be processed and target feedforward task model parameters, to obtain a task execution result, wherein the target feedforward task model parameters are feedforward task model parameters corresponding to the domain recognition result read from the target storage unit.

[0005] According to another aspect of this disclosure, a task execution apparatus for a large model is provided, comprising: a target storage unit storing multiple domain gating model parameters and multiple feedforward task model parameters; a target computing unit configured to: perform a modality routing task based on target features to be processed, thereby obtaining a modality recognition result; perform a domain routing task based on the target features to be processed and the target domain gating model parameters, thereby obtaining a domain recognition result, wherein the target domain gating model parameters are domain gating model parameters corresponding to the modality recognition result read from the target storage unit; and perform a feedforward task based on the target features to be processed and the target feedforward task model parameters, thereby obtaining a task execution result, wherein the target feedforward task model parameters are feedforward task model parameters corresponding to the domain recognition result read from the target storage unit.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods described above.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the methods described above.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described above.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 This illustration schematically shows an exemplary system architecture applicable to task execution methods and apparatus for large models according to embodiments of the present disclosure;

[0012] Figure 2 A flowchart illustrating a task execution method for a large model according to an embodiment of the present disclosure is shown schematically.

[0013] Figure 3 This is a schematic diagram of the processing layer of a large model according to an embodiment of the present disclosure;

[0014] Figure 4A A schematic diagram illustrating the determination of a large model according to an embodiment of the present disclosure is shown;

[0015] Figure 4B A schematic diagram illustrating the determination of a large model according to another embodiment of the present disclosure is shown;

[0016] Figure 5 A block diagram of a task execution apparatus for a large model according to embodiments of the present disclosure is schematically shown; and

[0017] Figure 6 A block diagram of an electronic device suitable for implementing a task execution method for large models, according to an embodiment of the present disclosure, is illustrated schematically. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] In the field of deep learning, the application of large models is constantly expanding. Large models can include, for example, large language models (LLM), large image models, and large audio models. Large models demonstrate superior task processing capabilities. However, it is often difficult to guarantee that large models achieve high performance while also possessing excellent processing efficiency.

[0020] Large models can be pre-trained large models. The architecture of a large model can be a Transformer (encoder-decoder) architecture. The Transformer architecture boasts powerful data processing capabilities and flexibility, and its applications in natural language processing are constantly expanding. A large model with a Transformer architecture can include multiple processing layers. Each processing layer can have the same structure, including modules based on Multi-Head Self-Attention (MHA) and modules based on Feed-Forward Networks (FFN).

[0021] All parameters in each processing layer process the input data. However, each parameter in a processing layer plays a different role depending on the domain type and modality of the input data. This means that during each task execution, the entire large model runs, but only a small portion of the parameters actually function. With parameter redundancy, model inference efficiency may be low.

[0022] Therefore, in order to ensure high performance of large models while possessing superior inference efficiency, this disclosure provides a task execution method for large models, which will be described below.

[0023] Figure 1 The illustration schematically shows an exemplary system architecture applicable to task execution methods and apparatus for large models according to embodiments of the present disclosure.

[0024] It is important to note that Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0025] like Figure 1 As shown, the system architecture according to this embodiment may include a terminal device 101, a network 102, and a server cluster 103. The network 102 serves as a medium for providing a communication link between the terminal device 101 and the server cluster 103. The network 102 may also serve as a medium for providing a communication link within the server cluster 103. The network 102 may include various connection types, such as wired and / or wireless communication links, etc.

[0026] Users can use terminal device 101 to interact with server cluster 103 via network 102 to receive or send messages, etc. For example, terminal device 101 can send a request to server cluster 103 for training a deep learning model via network 102.

[0027] Various communication client applications can be installed on the terminal device 101, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (for example only).

[0028] Terminal device 101 can be various electronic devices with a display screen and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0029] Server cluster 103 may be servers that provide various services, such as a back-end management server that supports requests sent by users using terminal device 101 (for example only).

[0030] Server cluster 103 can be a cloud server, also known as a cloud computing server or cloud host. It is a host product in the cloud computing service system, which solves the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"), such as high management difficulty and weak business scalability. The server can also be a server for a distributed system, or a server combined with blockchain.

[0031] Server cluster 103 includes multiple server nodes 1031, 1032, 1033, and 1034, each server node including one or more hardware devices. The server cluster 103 or the server nodes can be used to execute the task execution methods for large models provided in this disclosure, enabling the deployment, inference, or training of large models with lower computational resources.

[0032] As can be understood, the system architecture of this disclosure has been described above, and the method of this disclosure will be described below.

[0033] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0034] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0035] Figure 2 A flowchart illustrating a task execution method for a large model according to an embodiment of the present disclosure is shown schematically.

[0036] like Figure 2 As shown, the method includes operations S210~S230.

[0037] In operation S210, based on the target features to be processed, the target computing unit is used to perform a modal routing task to obtain the modal recognition result.

[0038] In this embodiment of the disclosure, the target computing unit may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), and an artificial intelligence computing unit. The artificial intelligence computing unit may include at least one of a neural network processing unit (NPU), a tensor processing unit (TPU), and a Kunlun chip.

[0039] In this embodiment, the processing layer of the large model may include a modal router. The modal router is used to perform modal recognition on the target features to be processed, obtaining a modal recognition result. The modal router can be implemented using a linear layer, such as Softmax. The target computation unit can process the target features based on the modal routing model parameters of the modal router to obtain the modal recognition result, thereby completing the modal routing task.

[0040] In this embodiment of the disclosure, the modality recognition result can characterize the modality type of the target feature to be processed. For example, the modality type of the target feature to be processed may include at least one of text, image, or audio.

[0041] In operation S220, based on the target features to be processed and the target domain gating model parameters, the target computing unit performs a domain routing task to obtain the domain identification result.

[0042] In this embodiment, the processing layer of the large model includes a domain router for performing domain identification on the target features to be processed, thereby obtaining a domain identification result. The domain router can also be implemented using a linear layer such as Softmax. The target computation unit can perform a domain routing task on the target features to be processed based on the parameters of the target domain gating model to obtain the domain identification result.

[0043] In this embodiment of the disclosure, the domain identification result can characterize the domain type of the target feature to be processed. For example, the domain type of the target feature to be processed may include at least one of translation, question answering, retrieval, text generation, and intent recognition.

[0044] In this embodiment of the disclosure, each processing layer of the large model can be configured with multiple domain gates, such as Router1, Router2, and Router3. Each domain gate corresponds to a modality type. For example, Router1 corresponds to the text modality type, Router2 corresponds to the image modality type, and Router3 corresponds to the audio modality type.

[0045] In this embodiment, the target domain gating model parameters can be the domain gating model parameters corresponding to the modality recognition result, read from the target storage unit. For example, the domain gating model parameters of Router1, Router2, and Router3 can be stored in the target storage unit. When the modality recognition result represents the modality type of the data to be processed as text, the domain gating model parameters of Router1 can be read from the target storage unit as the target domain gating model parameters.

[0046] In operation S230, based on the target features to be processed and the target feedforward task model parameters, the target computing unit is used to execute the feedforward task and obtain the task execution result.

[0047] In this embodiment of the disclosure, the processing layer of the large model may include multiple modal expert modules, each modal expert module being matched with a modality type. Each modal expert module includes multiple domain expert submodules. Each domain expert submodule is matched with a domain type.

[0048] In this embodiment of the disclosure, each domain expert submodule may include at least one feedforward neural network. Each domain expert submodule is used to process the target features to be processed, such as through vector mapping, to obtain the task execution result. The target computing unit can perform a feedforward task on the target features to be processed based on the target feedforward task model parameters of the target domain expert submodule to obtain the task execution result.

[0049] In this embodiment, each processing layer of the large model can be configured with multiple modality expert modules corresponding to each modality type, such as modality expert modules Expert1, Expert1, and Expert1. Each modality expert module can include multiple domain expert sub-modules; for example, modality expert module Expert1 includes domain expert sub-modules Expert1-1, Expert1-2, and Expert1-3. Each domain expert sub-module corresponds to a domain type. For example, domain expert sub-module Expert1-1 corresponds to the question-answering domain type, domain expert sub-module Expert1-2 corresponds to the retrieval domain type, and domain expert sub-module Expert1-3 corresponds to the translation domain type.

[0050] In this embodiment, the target feedforward task model parameters are the target feedforward task model parameters corresponding to the domain recognition result, read from the target storage unit. For example, the feedforward task model parameters of the domain expert submodules Expert1-1, Expert1-2, and Expert1-3 can be stored in the target storage unit. When the modality recognition result represents the modality type of the data to be processed as a question-and-answer domain type, the target feedforward task model parameters can be the feedforward task model parameters of the domain expert submodule Expert1-1.

[0051] In this embodiment, the processing layer of a large model combines multiple domain expert submodules using ensemble learning to form multiple modal expert modules, enabling the processing of data of different modalities. This improves the application capability and scope of the large model, enhancing its versatility. Furthermore, each domain expert submodule focuses on solving feedforward tasks of a specific domain type within a specific modality, thereby increasing the granularity of the large model's application and its specificity. Additionally, when executing feedforward tasks of a specific domain type within a modality, only the target computing unit needs to utilize the target feedforward task model parameters, eliminating the need to use all feedforward task model parameters from all domain expert submodules. This improves the processing efficiency of the target computing unit while reducing its hardware resource requirements and energy consumption. Finally, the modal gating system, multiple domain gating systems, and multiple modal expert modules are combined to form a complete processing layer. The system can dynamically adapt the target domain gate based on the data characteristics of the target features to be processed through a modal gating mechanism. Furthermore, it can dynamically adapt the target domain expert sub-module under the target modal expert module based on the data characteristics of the target features to be processed, thereby improving the inference performance of the activated target feedforward task model parameters. This enhances inference performance while maintaining the inference efficiency of the large model.

[0052] As can be understood, the above provides an overview of the method disclosed herein, and the following will further explain the task execution method of this disclosure for large models.

[0053] Figure 3 This is a schematic diagram of the processing layer of a large model according to an embodiment of the present disclosure.

[0054] like Figure 3 As shown, a single processing layer of the large model includes a modal gate Router0, a first-domain gate Router1, a second-domain gate Router2, ..., an nth-domain gate Routern, a first-modal expert module Expert1, a second-modal expert module Expert2, ..., and an nth-modal expert module Expertn. The first-modal expert module Expert1, corresponding to the first-domain gate Router1, includes first-domain expert submodules Expert1-1, second-domain expert submodules Expert1-2, ..., and an i-th-domain expert submodule Expert1-i. The second-modal expert module Expert2, corresponding to the second-domain gate Router2, includes first-domain expert submodules Expert2-1, second-domain expert submodules Expert2-2, ..., and an i-th-domain expert submodule Expert2-i. The nth modal expert module Expertn, corresponding to the nth domain gate Router, includes the first domain expert submodule Expertn-1, the second domain expert submodule Expertn-2, ..., the ith domain expert submodule Expertn-i.

[0055] The domain gates Router1 (first domain), Router2 (second domain), ..., Routern (nth domain) have different modal types corresponding to their respective domain gate model parameters.

[0056] The feedforward task model parameters of multiple domain expert submodules corresponding to the same domain gate are different from each other's corresponding domain types.

[0057] Modal gating model parameters, multiple domain gating model parameters, and multiple feedforward task model parameters can be pre-stored in the target storage unit.

[0058] like Figure 3As shown, based on the target feature to be processed 310 and the modal gating model parameters of the modal gating system stored in the target storage unit, a modal routing task can be performed using the target computing unit to obtain modal recognition results. For example, the modal recognition results indicate that the modal type of the target feature to be processed corresponds to the first domain gating system Router1. Based on the target feature to be processed and the target domain gating model parameters of the first domain gating system Router1, a domain routing task can be performed using the target computing unit to obtain domain recognition results. For example, the domain recognition results indicate that the domain type of the target feature to be processed corresponds to the first domain expert submodule Expert1-1. Based on the target feature to be processed and the target feedforward task model parameters of the first domain expert submodule Expert1-1, a feedforward task can be performed using the target computing unit to obtain task execution results 320.

[0059] According to embodiments of this disclosure, the task execution method for large models may further include: obtaining target domain gating model parameters from multiple domain gating model parameters stored in the target storage unit based on modality recognition results; and performing a domain routing task using the target computing unit based on the target features to be processed and the target domain gating model parameters to obtain domain recognition results.

[0060] According to embodiments of this disclosure, the task execution method for large models may further include: obtaining target feedforward task model parameters from multiple feedforward task model parameters stored in the target storage unit based on the domain identification result; and executing the feedforward task using the target computing unit based on the target features to be processed and the target feedforward task model parameters to obtain the task execution result.

[0061] In this embodiment, the input data of the large model can be text-based modality and question-and-answer domain data to be processed. After processing by the preprocessing layer of the processing layer, the target features to be processed are obtained. After the task execution method provided in this embodiment, the task execution result can be used to obtain the output result of the large model. The output result can be the answer corresponding to the data to be processed.

[0062] According to embodiments of this disclosure, modal types may include at least one of the following: images, text, and audio. Domain types may include at least one of the following: translation data, question-answering data, retrieval data, text generation data, and intent recognition data.

[0063] In other embodiments, the processing layer of the large model can consist of two parts: a gating network and expert modules. The expert modules may include: a first expert module Expert1-1, Expert2-1, ..., Expertn-1; a second expert module Expert1-2, Expert2-2, ..., Expertn-2; and an i-th expert module Expertti-1, Expertti-2, ..., Expertti-n. The gating network is responsible for dynamically determining which expert module should be activated to generate the best prediction based on the characteristics of the target feature to be processed. Based on the output of the gating network, the target expert module is determined from among the multiple expert modules. Then, the target computational unit processes the target feature based on the model parameters of the target expert module to obtain the task execution result.

[0064] Compared to task execution methods utilizing a single gated network, the task execution method provided in this disclosure can perform modal classification of target features by executing modal routing tasks and mapping them to target domain gated model parameters. It can also perform domain classification of target features by executing domain routing tasks and mapping them to target feedforward task model parameters. Therefore, through two-level type recognition, the target feedforward task model parameters obtained from multiple feedforward task model parameters are accurate and effective, thereby improving the inference accuracy and efficiency of task execution methods used for large models.

[0065] As we have explained above, the network structure of the processing layer in a large model is now being discussed. The following section will further explain how the model parameters for the processing layer are obtained.

[0066] In the embodiments of this disclosure, the task execution method for large models can be applied to any service node of the server, but is not limited thereto, and can also be applied to terminal devices.

[0067] In this embodiment of the disclosure, the task execution method for large models may further include operations for obtaining model parameters.

[0068] For example, a model retrieval request is sent to the server using an interface. In response to receiving multiple feedforward task model parameters and multiple domain-gated model parameters corresponding to the model retrieval request, the multiple feedforward task model parameters and multiple domain-gated model parameters are stored in the target storage unit.

[0069] In this embodiment, the terminal device can send a model acquisition request to the server via an interface. Upon receiving the request, the server can send model parameters of a large model, such as modal gating model parameters, multiple feedforward task model parameters, and multiple domain gating model parameters, to the terminal device via the interface. In response to receiving the model parameters, the terminal device stores the modal gating model parameters, multiple feedforward task model parameters, and multiple domain gating model parameters in a target storage unit. This allows the terminal device to retrieve the corresponding model parameters from the target storage unit during task execution using the target computing unit.

[0070] In this embodiment of the disclosure, the model acquisition request may include model performance requirements. Model performance requirements may include at least one of the following: model accuracy, model latency, power consumption of the target computing unit, and number of model parameters. However, it is not limited to these. It may also include the domain type of the domain expert submodule and / or the modality type of the domain gating.

[0071] In this embodiment of the disclosure, the multiple feedforward task model parameters corresponding to the model acquisition request may include feedforward task model parameters that meet the model performance requirements and multiple domain-gated model parameters.

[0072] The task execution method for large models provided in this disclosure can generate a model acquisition request based on the hardware of the terminal device, such as the resource configuration of the target computing unit and user requirements. Based on the model acquisition request, model parameters adapted to the hardware resources of the terminal device or user requirements are obtained and successfully applied to the terminal device. This improves the flexibility and targeting of the task execution method for large models and enhances the satisfaction of personalized needs.

[0073] The methods for obtaining model parameters have been explained above. The following section will further explain the acquisition and optimization of model parameters for feedforward tasks.

[0074] According to embodiments of this disclosure, multiple feedforward task model parameters can be obtained in the following manner.

[0075] For example, based on the sample set and the pre-trained large model, multiple target compression large models are obtained. Based on the multiple target compression large models, multiple feedforward task model parameters are obtained.

[0076] In this embodiment of the disclosure, the pre-trained large model may include a pre-trained large model containing a Transformer structure. The pre-trained large model may include model parameters to be compressed that have the same function as the feedforward task model parameters. These model parameters to be compressed may, for example, include model parameters used to perform the feedforward task, such as the model parameters of FFN. Compared to the feedforward task model parameters, the model parameters to be compressed have a larger number of parameters.

[0077] In embodiments of this disclosure, the sample set may include multiple sample subsets of different domain types. For example, the sample set may include a text-based question-and-answer sample subset, a text-based retrieval sample subset, a text-based translation sample subset, an audio-based question-and-answer sample subset, an audio-based retrieval sample subset, an audio-based translation sample subset, an image-based question-and-answer sample subset, an image-based retrieval sample subset, and an image-based translation sample subset.

[0078] In this embodiment of the disclosure, a compression task can be performed using a computing unit based on multiple sample subsets and the same pre-trained large model to obtain multiple target compressed large models that correspond one-to-one with the multiple sample subsets.

[0079] In this embodiment of the disclosure, the compression task may refer to using a sample set to compress predetermined parameters in a pre-trained large model, such as the parameters of the model to be compressed, for example, by pruning or lightweighting, to obtain a target compressed large model that inherits the knowledge and capabilities of the pre-trained large model and whose model parameters are smaller than those of the pre-trained large model.

[0080] In this embodiment of the disclosure, each of the multiple target compression large models maintains the same backbone network as the pre-trained large model, with only the model parameters of the FFN used to perform the feedforward task being different. For example, the model parameters of the FFN used to perform the feedforward task in the target compression large model correspond to the corresponding sample subset.

[0081] In this embodiment of the disclosure, the FFN model parameters of each of the multiple target compression large models can be used as the model parameters of multiple feedforward tasks.

[0082] The compression method of the target compressed large model provided in this embodiment compresses the parameters of the model to be compressed, resulting in a smaller number of parameters in the compressed target compressed large model while effectively inheriting the knowledge and reasoning capabilities of the model to be compressed. Furthermore, by using multiple sample subsets of different domain types, the target compressed large model inherits the reasoning capabilities and knowledge of the pre-trained large model across different domain types and modalities.

[0083] In this embodiment of the disclosure, multiple target compression large models are obtained based on a sample set and a pre-trained large model, which may include the following operations.

[0084] For example, for each subset of samples, a compressed large model is obtained based on the pruning matrix and the pre-trained large model. The target compressed large model is then obtained based on the compressed subset of samples, the pre-trained large model, and the compressed large model.

[0085] In this embodiment of the disclosure, the clipping matrix can be used to indicate the clipping method of the model parameters to be compressed. The clipping matrix Z can include multiple clipping elements, each of which can be represented by 0 or 1. Each clipping element corresponds to a model parameter of the model parameters to be compressed. 0 indicates that the model parameter is clipped, and 1 indicates that the model parameter is retained.

[0086] In this embodiment, a compressed large model can be obtained by vector multiplication of the pruning matrix with the model parameters of the pre-trained large model. Based on the sample subset, the pre-trained large model, and the compressed large model, the target compressed large model is obtained.

[0087] The target compressed large model is obtained based on a sample subset, a pre-trained large model, and a compressed large model. This can include the following operations: based on the sample subset, the pre-trained large model, and the compressed large model, a cyclic compression task is performed using a computing unit to obtain the target compressed large model.

[0088] In this embodiment of the disclosure, the iterative compression task may include: if, based on a subset of samples, the performance of the pre-trained large model matches the performance of the compressed large model, then the compressed large model is selected as the target compressed large model. If, based on a subset of samples, the performance of the pre-trained large model does not match the performance of the compressed large model, the pruning matrix is ​​updated to obtain an updated compressed large model. The iterative compression task is then repeatedly executed using the updated compressed large model, the subset of samples, and the pre-trained large model until the performance of the pre-trained large model matches the performance of the compressed large model.

[0089] Compression of the pre-trained large model using the pruning matrix provided in this embodiment makes the entire compression process analyzable and improves compression efficiency. Furthermore, compressing the pre-trained large model using multiple sample subsets from different domain types separately increases the variety of the resulting target compressed large model, thereby further improving compression efficiency.

[0090] In this embodiment of the disclosure, obtaining a target compressed large model based on a sample subset, a pre-trained large model, and a compressed large model may include the following operations: determining the reference inference capability of the pre-trained large model and the verification inference capability of the compressed large model based on the sample subset; and determining the target compressed large model based on the compressed large model if the verification inference capability matches the reference inference capability.

[0091] For example, a pre-trained large model and a compressed large model can be used to process a subset of sample data within a sample subset, respectively, to obtain a first prediction result corresponding to the pre-trained large model and a second prediction result corresponding to the compressed large model. Based on the first prediction result and the label subset corresponding to the sample data subset, a reference inference capability is obtained. Based on the second prediction result and the label subset corresponding to the sample data subset, a verification inference capability is obtained. If the similarity between the verification inference capability and the reference inference capability is greater than a threshold, the verification inference capability is determined to match the reference inference capability.

[0092] If the similarity between the verification reasoning ability and the reference reasoning ability is less than or equal to a threshold, it is determined that the verification reasoning ability and the reference reasoning ability are mismatched. Based on the reference reasoning ability and the verification reasoning ability, the model parameters of the pre-trained large model and the pruning elements of the pruning matrix can be adjusted to obtain an updated pre-trained large model and an updated pruning matrix.

[0093] Based on the sample subset, the updated pre-trained large model, and the pruning elements of the updated pruning matrix, the compression task is re-executed until the reference inference capability and the verification inference capability match. The compressed large model whose reference inference capability and verification inference capability match is then used as the target compressed large model.

[0094] In another embodiment of this disclosure, a predetermined sparsity can also be set. The predetermined sparsity is used to indicate a predetermined pruning granularity. The predetermined sparsity can be determined based on information carried in the model acquisition request.

[0095] Based on reference inference and validation inference capabilities, adjusting the model parameters of the pre-trained large model and the pruning elements of the pruning matrix to obtain the updated pre-trained large model and the updated pruning matrix elements may include: determining sparsity based on the pruning matrix; determining the sparse loss value based on the sparsity and a predetermined sparsity; and adjusting the model parameters of the pre-trained large model and the pruning elements of the pruning matrix based on reference inference capability, validation inference capability, and the sparse loss value to obtain the updated pre-trained large model and the updated pruning matrix elements.

[0096] In this embodiment, the sparse loss value can be determined using a loss function based on sparsity and a predetermined sparsity. Using the loss function, a reference inference capability is obtained based on a first prediction result and a label subset corresponding to a subset of sample data. Using the loss function, a verification inference capability is obtained based on a second prediction result and a label subset corresponding to a subset of sample data. The loss function can be the cross-entropy loss function, but is not limited to it; other functions that can characterize the matching degree between two sets of data to be evaluated can also be used.

[0097] Based on the sample subset, the updated pre-trained large model, and the pruning elements of the updated pruning matrix, the compression task is re-executed until the reference inference capability and the verification inference capability match and the sparsity of the compressed large model meets the predetermined sparsity. The compressed large model whose reference inference capability and verification inference capability match and whose sparsity meets the predetermined sparsity is selected as the target compressed large model.

[0098] In this embodiment of the disclosure, the compressed model parameters corresponding to the parameters of the model to be compressed in the target compressed large model can be used as the feedforward task model parameters.

[0099] The compression method provided in this embodiment yields a target compressed large model, where the model parameters are pruned and lightweight compared to the pre-trained large model, thus reducing the number of parameters in the target compressed large model. Furthermore, by using a subset of samples as compression samples, the pre-trained large model is trained simultaneously with compression, improving the inference ability of the target compressed large model for domain-specific data within this modality, thereby enhancing its overall inference capability.

[0100] In this embodiment of the disclosure, the multiple feedforward model parameters obtained by the above method can be directly used as the feedforward model parameters of the large model.

[0101] In another embodiment of this disclosure, the multiple feedforward model parameters obtained in the above manner can also be used as multiple initial feedforward model parameters. An initial large model is obtained based on the pre-trained large model, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters. The initial modal-gated model parameters have the same function as the model parameters used to perform the modal routing task, each initial domain-gated model parameter has the same function as the domain-gated model parameters, and each initial feedforward task model parameter has the same function as the feedforward task model parameters.

[0102] For example, based on a pre-trained large model, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters, an initial large model can be obtained. This can include replacing the parameters of the model to be compressed in the pre-trained large model with the parameters of the multiple initial domain-gated model, multiple initial feedforward task model parameters, and initial modal-gated model parameters. For example, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters can be used as a hybrid expert architecture to replace the FFN layer in the pre-trained large model corresponding to the parameters of the model to be compressed, thus obtaining the initial large model. The hybrid expert architecture contains multiple layers of networks, which can utilize multi-level routing in multiple layers of networks to accurately locate the initial feedforward task model parameters. Using the initial feedforward task model parameters, inference performance can be guaranteed while inference efficiency is improved through a small number of model parameters.

[0103] The initial large model can be optimized and trained to obtain a larger model. This optimization training can further improve the model combination suitability of the hybrid expert architecture, thereby improving the inference performance of the larger model.

[0104] The following section will further explain how the optimized training of the initial large model yields a larger model.

[0105] According to embodiments of this disclosure, optimizing and training an initial large model to obtain a large model may include: obtaining a model output result set based on a sample dataset and the initial large model; obtaining multiple loss values ​​based on the model output result set and a label set; obtaining a large model based on the multiple loss values ​​and the initial large model; and using model parameters in the large model that have the same function as the feedforward task model parameters as the feedforward task model parameters.

[0106] The sample set may also include a set of labels that match the sample dataset. The sample dataset includes multiple subsets of sample data with different domain types. The initial large model includes model parameters that have the same function as the model parameters of the feedforward task.

[0107] In this embodiment, the sample dataset of the sample set can be input into an initial large model to obtain the model output result set. The model output result set and the label set are input into a loss function to obtain multiple loss values. Based on the multiple loss values ​​and the initial large model, a large model is obtained. The multiple loss values ​​correspond one-to-one with multiple subsets of sample data.

[0108] There are no restrictions on the type of loss function. It can include cross-entropy loss functions. Any loss function that can determine the loss value is acceptable.

[0109] According to embodiments of this disclosure, obtaining feedforward task model parameters based on multiple loss values ​​and an initial large model may include: obtaining a target loss value based on multiple loss values; and obtaining feedforward task model parameters based on the target loss value and the initial large model.

[0110] Obtaining the target loss value based on multiple loss values ​​can include: weighted summation of the multiple loss values; adjusting the initial large model if the target loss value does not converge; and using the initial large model where the target loss value converges as the large model if the target loss value converges. The model parameters in the large model corresponding to the initial feedforward task model parameters are then used as the feedforward task model parameters.

[0111] By employing the aforementioned hybrid training method, the entire hybrid expert architecture can be fine-tuned using the sample set to improve overall adaptability. This, in turn, enhances the inference efficiency and performance of the large model's feedforward task parameters, while simultaneously improving the classification performance of the modal routing task parameters and domain routing task parameters, thereby improving the overall performance of the large model.

[0112] Figure 4A A schematic diagram illustrating the determination of a large model according to an embodiment of the present disclosure is shown.

[0113] like Figure 4A As shown, the pre-trained large model can include multiple processing layers, each of which includes a multi-head attention module (MHA), an addition and normalization module (Add&Norm), and a feedforward module (FFN). The model parameters of the FFN of the pre-trained large model can be compressed using a sample set to obtain multiple target compressed large models. Based on these multiple target compressed large models, multiple initial feedforward task model parameters FFN1, ..., FFNX are obtained.

[0114] like Figure 4A As shown, the model parameters of FFN in the pre-trained large model are replaced by multiple initial domain gated model parameters Router1, ..., RouterX, multiple initial feedforward task model parameters FFN1, ..., FFNX, and the initial modal gated model parameter Router0 to obtain the initial large model M410.

[0115] like Figure 4A As shown, the initial large model 410 is optimized and trained using the sample set to obtain the large model M420.

[0116] Figure 4B A schematic diagram illustrating the determination of a large model according to another embodiment of the present disclosure is shown.

[0117] like Figure 4B As shown, the task execution method is similar to... Figure 4AThe task execution methods shown are similar, with the only difference being that the model parameters of the FFN of the pre-trained large model are compressed using a subset of samples from the sample set to obtain a lightweight large model that matches the sample subset. The FFN parameters in the lightweight large model are then copied to obtain multiple initial feedforward task parameters FFN'.

[0118] The initial large model M410' is obtained by replacing the FFN model parameters in the pre-trained large model with multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters. The initial large model M410' is then trained using the sample set to obtain the large model M420'.

[0119] With Figure 4B Compared to the method shown for determining the large model, using methods such as Figure 4A The method of determining the large model can be achieved by using different types of sample subsets in the sample set, so that the parameters of multiple initial feedforward task models can inherit the knowledge of the pre-trained large model before pruning. The parameters of multiple initial feedforward task models have natural differentiation from each other, which is conducive to improving the training efficiency and training accuracy of hybrid training.

[0120] Figure 5 A block diagram of a task execution apparatus for a large model is shown schematically according to an embodiment of the present disclosure.

[0121] like Figure 5 As shown, the task execution device 500 for large models includes: a target storage unit 510 and a target computing unit 520.

[0122] The target storage unit 510 stores multiple domain gating model parameters and multiple feedforward task model parameters.

[0123] The target computing unit 520 is configured as follows:

[0124] Based on the target features to be processed, the modal routing task is performed using the target computing unit to obtain the modal recognition result.

[0125] Based on the target features to be processed and the target domain gating model parameters, a domain routing task is performed using the target computing unit to obtain the domain identification result. The target domain gating model parameters are the domain gating model parameters corresponding to the modality identification result read from the target storage unit.

[0126] Based on the target features to be processed and the target feedforward task model parameters, the target computing unit is used to execute the feedforward task to obtain the task execution result. The target feedforward task model parameters are the feedforward task model parameters corresponding to the domain recognition result read from the target storage unit.

[0127] According to embodiments of this disclosure, the task execution device 500 for large models further includes an actuator.

[0128] The actuator is configured as follows:

[0129] Based on the modality recognition results, target domain gated model parameters are obtained from multiple domain gated model parameters stored in the target storage unit. Among them, the modality types corresponding to the multiple domain gated model parameters are different.

[0130] According to embodiments of this disclosure, the actuator is further configured to:

[0131] Based on the domain identification results, the target feedforward task model parameters are obtained from multiple feedforward task model parameters stored in the target storage unit. Among them, the multiple feedforward task model parameters correspond to different domain types.

[0132] According to embodiments of this disclosure, multiple feedforward task model parameters are obtained in the following manner.

[0133] According to embodiments of this disclosure, the target computing unit is further configured as follows:

[0134] Based on the sample set and the pre-trained large model, multiple target compression large models are obtained. The sample set includes multiple sample subsets of different domain types, and the pre-trained large model includes parameters of the model to be compressed that have the same function as the parameters of the feedforward task model.

[0135] Based on a large model with multiple objectives, parameters for multiple feedforward task models are obtained.

[0136] According to embodiments of this disclosure, multiple target compression large models are obtained based on a sample set and a pre-trained large model, including:

[0137] For each subset of samples, a compressed large model is obtained based on the pruning matrix and the pre-trained large model, where the pruning matrix indicates how the parameters of the model to be compressed are pruned; and

[0138] The target compressed large model is obtained based on a subset of samples, a pre-trained large model, and a compressed large model.

[0139] According to embodiments of this disclosure, a target compressed large model is obtained based on a sample subset, a pre-trained large model, and a compressed large model, including:

[0140] Based on a subset of samples, determine the reference inference capability of the pre-trained large model and the verification inference capability of the compressed large model; and

[0141] If the verification reasoning ability matches the reference reasoning ability and the sparsity of the compressed large model meets the predetermined sparsity, then the target compressed large model is determined based on the compressed large model.

[0142] According to embodiments of this disclosure, the feedforward task model parameters are obtained in the following manner:

[0143] The target computing unit is also configured as follows:

[0144] Based on the sample dataset and the initial large model, the model output result set is obtained. The sample dataset also includes a label set that matches the sample dataset. The sample dataset includes multiple subsets of sample data with different domain types. The initial large model includes model parameters that have the same function as the model parameters of the feedforward task.

[0145] Based on the model output result set and label set, multiple loss values ​​are obtained, and each loss value corresponds one-to-one with a subset of sample data.

[0146] Based on multiple loss values ​​and an initial large model, the parameters of the feedforward task model are obtained.

[0147] According to embodiments of this disclosure, feedforward task model parameters are obtained based on multiple loss values ​​and an initial large model, including:

[0148] The target loss value is obtained based on multiple loss values.

[0149] Based on the target loss value and the initial large model, the parameters of the feedforward task model are obtained.

[0150] According to embodiments of this disclosure, the initial large model is obtained in the following manner:

[0151] The target computing unit is also configured as follows:

[0152] Based on the pre-trained large model, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters, an initial large model is obtained. Among them, the initial modal-gated model parameters have the same function as the model parameters used to perform modal routing tasks, each initial domain-gated model parameter has the same function as the domain-gated model parameters, and each initial feedforward task model parameter has the same function as the feedforward task model parameters.

[0153] According to embodiments of this disclosure, the modal type includes at least one of the following: image, text, and audio.

[0154] The domain type includes at least one of the following: translation, question answering, retrieval, text generation, and intent recognition.

[0155] According to embodiments of this disclosure, the target computing unit is further configured as follows:

[0156] Use the interface to send a model retrieval request to the server.

[0157] In response to receiving multiple feedforward task model parameters and multiple domain-gated model parameters corresponding to the model acquisition request, the multiple feedforward task model parameters and multiple domain-gated model parameters are stored in the target storage unit.

[0158] According to embodiments of this disclosure, the model acquisition request includes model performance requirements.

[0159] The multiple feedforward task model parameters corresponding to the model acquisition request include feedforward task model parameters that meet the model performance requirements.

[0160] According to embodiments of this disclosure, the model performance requirements include at least one of the following:

[0161] Model accuracy, model latency, energy consumption of the target computing unit, and number of model parameters.

[0162] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0163] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0164] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the method described above.

[0165] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0166] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0167] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0168] Multiple components in device 600 are connected to input / output (I / O) interface 605, including: input unit 606, such as a keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as a disk, optical disk, etc.; and communication unit 609, such as a network interface card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0169] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the task execution method for large models. For example, in some embodiments, the task execution method for large models may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the task execution method for large models described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the task execution method for large models by any other suitable means (e.g., by means of firmware).

[0170] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0171] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0172] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0173] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0174] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0175] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0176] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0177] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task execution method for large models, comprising: Based on the target features to be processed, the modal routing task is performed using the target computing unit to obtain the modal recognition result; Based on the target features to be processed and the target domain gating model parameters, a domain routing task is performed using the target computing unit to obtain a domain identification result. The target domain gating model parameters are domain gating model parameters corresponding to the modality identification result, read from the target storage unit. The modality identification result characterizes the modality type of the target features to be processed, and the modality type of the target features to be processed includes at least one of the following: image, text, audio; and Based on the target features to be processed and the target feedforward task model parameters, the target computing unit is used to execute the feedforward task to obtain the task execution result. The target feedforward task model parameters are feedforward task model parameters corresponding to the domain recognition result read from the target storage unit. The domain recognition result represents the domain type of the target features to be processed. The domain type includes at least one of the following: translation, question answering, retrieval, text generation, and intent recognition. The processing layer of the large model includes multiple modal expert modules, each modal expert module is matched with a modality type, each modal expert module includes multiple domain expert sub-modules, each domain expert sub-module is matched with a domain type, and each domain expert sub-module includes at least one feedforward task model parameter.

2. The method according to claim 1, further comprising: Based on the modality recognition result, the target domain gated model parameters are obtained from multiple domain gated model parameters stored in the target storage unit, wherein the multiple domain gated model parameters correspond to different modality types.

3. The method according to claim 1, further comprising: Based on the domain identification result, the target feedforward task model parameters are obtained from multiple feedforward task model parameters stored in the target storage unit, wherein the multiple feedforward task model parameters correspond to different domain types.

4. The method according to claim 1, wherein, The parameters of multiple feedforward task models were obtained in the following way: Based on the sample set and the pre-trained large model, multiple target compression large models are obtained. The sample set includes multiple sample subsets of different domain types, and the pre-trained large model includes parameters of the model to be compressed that have the same function as the parameters of the feedforward task model. Based on the multiple target compression models, the parameters of the multiple feedforward task models are obtained.

5. The method according to claim 4, wherein, Based on the sample set and pre-trained large model, multiple target compression large models are obtained, including: For each of the sample subsets, a compressed large model is obtained based on the pruning matrix and the pre-trained large model, wherein the pruning matrix indicates the pruning method for the parameters of the model to be compressed; and The target compressed large model is obtained based on the sample subset, the pre-trained large model, and the compressed large model.

6. The method according to claim 5, wherein, The process of obtaining the target compressed large model based on the sample subset, the pre-trained large model, and the compressed large model includes: Based on the sample subset, the reference inference capability of the pre-trained large model and the verification inference capability of the compressed large model are determined; and If the verification reasoning capability matches the reference reasoning capability and the sparsity of the compressed large model satisfies a predetermined sparsity, the target compressed large model is determined based on the compressed large model.

7. The method according to any one of claims 1 to 6, wherein, The parameters of the feedforward task model are obtained in the following way: Based on the sample dataset and the initial large model, a model output result set is obtained. The sample dataset also includes a label set that matches the sample dataset. The sample dataset includes multiple sample data subsets with different domain types. The initial large model includes model parameters that have the same function as the feedforward task model parameters. Based on the model output set and the label set, multiple loss values ​​are obtained, wherein each of the multiple loss values ​​corresponds one-to-one with a subset of the sample data; and Based on the multiple loss values ​​and the initial large model, the parameters of the feedforward task model are obtained.

8. The method according to claim 7, wherein, The parameters of the feedforward task model are obtained based on the multiple loss values ​​and the initial large model, including: Based on the multiple loss values, the target loss value is obtained; and Based on the target loss value and the initial large model, the parameters of the feedforward task model are obtained.

9. The method according to claim 7, wherein, The initial large model was obtained in the following way: Based on the pre-trained large model, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters, the initial large model is obtained. The initial modal-gated model parameters have the same function as the model parameters used to perform the modal routing task. Each initial domain-gated model parameter has the same function as the domain-gated model parameters, and each initial feedforward task model parameter has the same function as the feedforward task model parameters.

10. The method according to any one of claims 1 to 6, further comprising: Use the interface to send a model retrieval request to the server; In response to receiving multiple feedforward task model parameters and multiple domain-gated model parameters corresponding to the model acquisition request, the multiple feedforward task model parameters and the multiple domain-gated model parameters are stored in the target storage unit.

11. The method according to claim 10, in, The model acquisition request includes model performance requirements. The plurality of feedforward task model parameters corresponding to the model acquisition request include feedforward task model parameters that meet the model performance requirements; The model performance requirements include at least one of the following: Model accuracy, model latency, energy consumption of the target computing unit, and number of model parameters.

12. A task execution device for a large model, comprising: The target storage unit stores parameters for multiple domain-gated models and parameters for multiple feedforward task models. The target computation unit is configured as follows: Based on the target features to be processed, the modal routing task is performed using the target computing unit to obtain the modal recognition result; Based on the target features to be processed and the target domain gating model parameters, a domain routing task is performed using the target computing unit to obtain a domain recognition result. The target domain gating model parameters are domain gating model parameters corresponding to the modality recognition result, read from the target storage unit. The modality recognition result characterizes the modality type of the target features to be processed, and the modality type of the target features to be processed includes at least one of the following: image, text, audio; and Based on the target features to be processed and the target feedforward task model parameters, the target computing unit is used to execute the feedforward task to obtain the task execution result. The target feedforward task model parameters are feedforward task model parameters corresponding to the domain recognition result read from the target storage unit. The domain recognition result represents the domain type of the target features to be processed. The domain type includes at least one of the following: translation, question answering, retrieval, text generation, and intent recognition. The processing layer of the large model includes multiple modal expert modules, each modal expert module is matched with a modality type, each modal expert module includes multiple domain expert sub-modules, each domain expert sub-module is matched with a domain type, and each domain expert sub-module includes at least one feedforward task model parameter.

13. The apparatus of claim 12, further comprising: The actuator is configured as follows: Based on the modality recognition result, the target domain gated model parameters are obtained from multiple domain gated model parameters stored in the target storage unit, wherein the multiple domain gated model parameters correspond to different modality types.

14. The apparatus according to claim 12, wherein, The actuator is also configured to: Based on the domain identification result, the target feedforward task model parameters are obtained from multiple feedforward task model parameters stored in the target storage unit, wherein the multiple feedforward task model parameters correspond to different domain types.

15. The apparatus according to claim 12, wherein, The parameters of multiple feedforward task models were obtained in the following way: The target computing unit is further configured as follows: Based on the sample set and the pre-trained large model, multiple target compression large models are obtained. The sample set includes multiple sample subsets of different domain types, and the pre-trained large model includes parameters of the model to be compressed that have the same function as the parameters of the feedforward task model. Based on the multiple target compression models, the parameters of the multiple feedforward task models are obtained.

16. The apparatus according to claim 15, wherein, Based on the sample set and pre-trained large model, multiple target compression large models are obtained, including: For each of the sample subsets, a compressed large model is obtained based on the pruning matrix and the pre-trained large model, wherein the pruning matrix indicates the pruning method for the parameters of the model to be compressed; and The target compressed large model is obtained based on the sample subset, the pre-trained large model, and the compressed large model.

17. The apparatus according to claim 16, wherein, The process of obtaining the target compressed large model based on the sample subset, the pre-trained large model, and the compressed large model includes: Based on the sample subset, the reference inference capability of the pre-trained large model and the verification inference capability of the compressed large model are determined; and If the verification reasoning capability matches the reference reasoning capability and the sparsity of the compressed large model satisfies a predetermined sparsity, the target compressed large model is determined based on the compressed large model.

18. The apparatus according to any one of claims 12 to 17, wherein, The parameters of the feedforward task model are obtained in the following way: The target computing unit is further configured as follows: Based on the sample dataset and the initial large model, a model output result set is obtained. The sample dataset also includes a label set that matches the sample dataset. The sample dataset includes multiple sample data subsets with different domain types. The initial large model includes model parameters that have the same function as the feedforward task model parameters. Based on the model output set and the label set, multiple loss values ​​are obtained, wherein each of the multiple loss values ​​corresponds one-to-one with a subset of the sample data; and Based on the multiple loss values ​​and the initial large model, the parameters of the feedforward task model are obtained.

19. The apparatus according to claim 18, wherein, The parameters of the feedforward task model are obtained based on the multiple loss values ​​and the initial large model, including: Based on the multiple loss values, the target loss value is obtained; and Based on the target loss value and the initial large model, the parameters of the feedforward task model are obtained.

20. The apparatus according to claim 18, wherein, The initial large model was obtained in the following way: The target computing unit is further configured as follows: Based on the pre-trained large model, multiple initial domain-gated model parameters, multiple initial feedforward task model parameters, and initial modal-gated model parameters, the initial large model is obtained. The initial modal-gated model parameters have the same function as the model parameters used to perform the modal routing task. Each initial domain-gated model parameter has the same function as the domain-gated model parameters, and each initial feedforward task model parameter has the same function as the feedforward task model parameters.

21. The apparatus according to any one of claims 12 to 17, wherein the target computing unit is further configured to: Use the interface to send a model retrieval request to the server; and In response to receiving multiple feedforward task model parameters and multiple domain-gated model parameters corresponding to the model acquisition request, the multiple feedforward task model parameters and the multiple domain-gated model parameters are stored in the target storage unit.

22. The apparatus according to claim 21, wherein, The model acquisition request includes model performance requirements. The plurality of feedforward task model parameters corresponding to the model acquisition request include feedforward task model parameters that meet the model performance requirements; The model performance requirements include at least one of the following: Model accuracy, model latency, energy consumption of the target computing unit, and number of model parameters.

23. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

25. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images