Task execution methods, apparatus, electronic devices, storage media, and programs used in large-scale models.

Abstract

To provide a task execution method and apparatus for a large scale model, an electronic device, a storage medium, and a program.SOLUTION: A method executes, according to a target feature to be processed, a collaborative computing task using a target computing unit, to obtain a target collaborative feature. The collaborative computing task includes a first collaborative task for processing the target feature to be processed and a first collaborative sub-weight to obtain an intermediate collaborative feature, and a second collaborative task for processing the intermediate collaborative feature and a second collaborative sub-weight to obtain a target collaborative feature. The method further processes collaborative weights according to a common matrix multiplication mechanism to determine the first collaborative sub-weight and the second collaborative-sub weight, and fuses a target basic feature obtained by executing a basic computing task using the target computing unit and the target collaborative feature to obtain a next target feature to be processed.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to the fields of deep learning technology and large model technology.

Background Art

[0002] Due to the rapid development of artificial intelligence technology, in scenarios such as intelligent customer service and knowledge question answering, large models can be used to process data in various scenarios.

Summary of the Invention

[0003] The present disclosure provides a task execution method, apparatus, task execution device, electronic device, storage medium and program for use in large models.

[0004] According to one aspect of the present disclosure, according to the features to be processed, a target computing unit is used to execute a cooperative computing task to obtain a target cooperative feature. Here, the cooperative computing task includes a first cooperative task for processing the features to be processed and a first cooperative sub-weight to obtain an intermediate cooperative feature, and a second cooperative task for processing the intermediate cooperative feature and a second cooperative sub-weight to obtain the target cooperative feature. The first cooperative sub-weight and the second cooperative sub-weight are determined by processing the cooperative weights according to the matrix multiplication mechanism of a general matrix. The target basic feature and the target cooperative feature are fused to obtain the next feature to be processed. Here, the target basic feature is obtained by using the target computing unit to execute a basic computing task, and the basic computing task is used to process the basic weight and the features to be processed. A task execution method for use in large models is provided.

[0005] In other parts of this disclosure, a task execution device is provided for use in large-scale models, comprising: a target storage unit for storing co-computation tasks; and a target computing unit arranged to execute co-computation tasks and obtain target co-features according to a target feature to be processed, wherein the co-computation task includes a first co-task for obtaining intermediate co-features by processing the target feature and a first co-subweight, and a second co-task for obtaining target co-features by processing the intermediate co-features and a second co-subweight, the first and second co-subweights being determined by processing the co-weights according to a matrix multiplication mechanism of a general matrix; and a target basic feature and the target co-features being merged to obtain the next target feature to be processed, wherein the target basic feature is obtained by executing a basic computation task in the target computing unit, the basic computation task being used to process the basic weight and the target feature.

[0006] In other parts of this disclosure, task execution devices for use in large-scale models are provided, including a task execution device used in a large-scale model provided by an embodiment of this disclosure.

[0007] In other parts of the present disclosure, an electronic device is provided, comprising at least one processor and memory communicably connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor in such a manner as provided by embodiments of the present disclosure.

[0008] In other aspects of this disclosure, a non-temporary computer-readable storage medium is provided that stores computer instructions causing a computer to perform the methods provided by embodiments of this disclosure.

[0009] According to other aspects of this disclosure, a computer program is provided that, when executed by a processor, implements the method provided by the embodiments of this disclosure.

[0010] It should be understood that the content described in this section is not intended to represent key points or important features of the embodiments of this disclosure, nor does it limit the scope of this disclosure. Other features of this disclosure will be readily apparent from the following description. [Brief explanation of the drawing]

[0011] The drawings are provided to better understand this technical proposal and do not limit this application.

[0012] [Figure 1] Figure 1 schematically shows an example of a system architecture to which the task execution method and apparatus according to the embodiments of this disclosure can be applied. [Figure 2] Figure 2 schematically shows a flowchart of the task execution method used in the large-scale model according to the embodiment of this disclosure. [Figure 3] Figure 3 schematically shows a schematic diagram of the principle of the task execution method used in the large-scale model according to the embodiment of this disclosure. [Figure 4] Figure 4 schematically shows a schematic diagram of the principle of a task execution method used in a large-scale model according to other embodiments of this disclosure. [Figure 5] Figure 5 schematically shows a flowchart of a task execution method used in a large-scale model according to other embodiments of this disclosure. [Figure 6] Figure 6 schematically shows an application scenario diagram of a task execution method used in a large-scale model according to the embodiment of this disclosure. [Figure 7] Figure 7 schematically shows a schematic diagram of the principle of a task execution method used in a large-scale model according to another embodiment of this disclosure. [Figure 8] Figure 8 schematically shows a block diagram of a task execution device used in a large-scale model according to the embodiment of this disclosure. [Figure 9] Figure 9 schematically shows a block diagram of a task execution device used in a large-scale model according to the embodiment of this disclosure. [Figure 10]Figure 10 schematically shows a block diagram of electronic equipment suitable for implementing the task execution method used in the large-scale model according to the embodiment of this disclosure. [Modes for carrying out the invention]

[0013] Illustrative embodiments of the present disclosure will be described below with reference to the drawings. Various details of the embodiments of the present disclosure are included herefor the sake of clarity, and should be considered as illustrative. Therefore, those skilled in the art will understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and configurations will be omitted in the following description.

[0014] In the proposed technology disclosed herein, the acquisition, storage, and application of such user personal information all comply with the provisions of relevant laws and regulations, employ necessary confidentiality measures, and do not violate public order and morals.

[0015] In the field of deep learning, the application of large-scale models is constantly expanding. However, the cost of training and inference for large-scale models is high, making their deployment difficult. For example, large-scale models may include large-language models (LLMs), large-scale image models, and large-scale speech models. Large-language models exhibit powerful text processing capabilities. However, because the scale of model parameters in large-scale models is enormous, potentially reaching hundreds of millions or even billions, large-scale models typically require a large amount of computational resources to perform the computational tasks.

[0016] Embodiments of this disclosure provide a task execution method, apparatus, task execution device, electronic device, storage medium, and program used in large-scale models. The task execution method used in large-scale models includes, in accordance with the features to be processed, executing a co-computation task using a target computation unit to obtain a target co-computation feature, wherein the co-computation task includes a first co-computation task for obtaining an intermediate co-computation feature by processing the features to be processed and a first co-computation subweight, and a second co-computation task for obtaining a target co-computation feature by processing the intermediate co-computation feature and a second co-computation subweight, wherein the first and second co-computation subweights are determined by processing the co-computation weights according to a matrix multiplication mechanism of a general matrix, and merging the target basic feature and the target co-computation feature to obtain the next target features to be processed, wherein the target basic feature is obtained by executing a basic computation task using a target computation unit, and the basic computation task is used to process the basic weight and the features to be processed.

[0017] According to embodiments of this disclosure, by splitting the weight parameters of a large-scale model into base weights and cooperative weights, the base weights of a general base model and cooperative weights for a specified personalization task are used to perform the computational tasks that the large-scale model needs to perform, thereby improving the flexibility and adaptability of the large-scale model in the inference process for performing personalization needs during the training or application process using a general base model. At the same time, the cooperative weights are determined as first cooperative subweights and second cooperative subweights according to a matrix multiplication mechanism based on a general matrix, so that the matrix dimensions of both the first cooperative subweights and the second cooperative subweights are smaller than the matrix dimensions of the cooperative weights, so that the sum of the computational complexity of the first and second cooperative tasks is smaller than the computational complexity of performing the cooperative computation task directly using the cooperative weights, thereby reducing the computational overhead of the target computation unit during the process in which the large-scale model performs the computational tasks, reducing computational energy consumption, and improving the computational efficiency of the large-scale model.

[0018] FIG. 1 schematically shows an example of a system architecture to which the task execution method and apparatus according to embodiments of the present disclosure can be applied.

[0019] It should be noted that FIG. 1 is only an example of a system architecture to which embodiments of the present disclosure can be applied to help those skilled in the art understand the technical content of the present disclosure, and it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenarios.

[0020] As shown in FIG. 1, the system architecture according to this embodiment can include a terminal device 101, a network 102, and a server cluster 103. The network 1o2 is used as a medium to provide a communication link between the terminal device 101 and the server cluster 103. The network 102 can also be used as a medium to provide a communication link within the server cluster 103. The network 102 can include various connection types such as wired and / or wireless communication links.

[0021] The user can interact with the server cluster 103 via the network 102 using the terminal device 101 to send and receive messages, etc. For example, the terminal device 101 may send a request to train a deep learning model to the server cluster 103 via the network 102.

[0022] Various communication client applications such as a knowledge reading application, a web browser application, a search application, an instant messaging tool, an email client, and / or social platform software (mere examples) can be installed on the terminal device 101.

[0023] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, and may include, but is not limited to, smartphones, tablet computers, laptop computers, and desktop computers.

[0024] The server cluster 103 may also be a server that provides various services, such as a background management server that supports requests sent by users using terminal devices 101 (this is just one example).

[0025] Server cluster 103 is a cloud server, also known as a cloud computing server or cloud host. It is a host product for cloud computing service systems, addressing the management difficulties and limited business scalability inherent in traditional physical hosts and VPS services ("Virtual Private Server," or simply "VPS"). The server can also be a distributed system server or a server combined with blockchain technology.

[0026] Server cluster 103 includes multiple server nodes 1031, 1032, 1033, and 1034, each server node including one or more hardware devices. The server cluster 103 or the server nodes can be used to perform the task execution methods used for the large-scale models provided in this disclosure, thereby enabling the deployment, inference, or training of the large-scale models with fewer computing and memory resources.

[0027] The system architecture of this disclosure is described above, and the methods of this disclosure are described below.

[0028] Please understand that the number of terminal devices, networks, and servers shown in Figure 1 are merely illustrative. You can have any number of terminal devices, networks, and servers depending on your implementation needs.

[0029] Figure 2 schematically shows a flowchart of the task execution method used in the large-scale model according to the embodiment of this disclosure.

[0030] As shown in Figure 2, the task execution method used for the large-scale model includes operations S210 to S220.

[0031] In operation S210, a collaborative computation task is performed using the target computation unit according to the features to be processed, and target collaborative features are obtained.

[0032] In operation S220, the target basic features and target co-features are merged to obtain the next feature to be processed.

[0033] According to embodiments of this disclosure, the target computing unit may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), and an artificial intelligence computing unit. The artificial intelligence computing unit may include at least one of a neural network processing unit (NPU), a tensor processing unit (TPU), and a Kunlun core.

[0034] According to embodiments of this disclosure, the computational task may include neuron computational tasks performed by the large-scale model, or it may include computational tasks performed by the processing layer of the large-scale model, such as attention tasks that the large-scale model needs to perform.

[0035] According to embodiments of this disclosure, computational tasks for large-scale models may include co-computation tasks and base computation tasks. A base computation task can be understood as a computation task performed based on the base weights of a base model (also called a base model) within a large-scale model. A co-computation task can be understood as a computation task performed based on the co-weights of a co-model in a large-scale model.

[0036] According to the embodiments of this disclosure, the collaborative weights can be model parameters obtained by fine-tuning a large-scale model including a base model, and computational tasks can be performed based on the base weights and the fine-tuned collaborative weights, thereby enabling inference tasks such as prediction and image generation of the large-scale model.

[0037] According to embodiments of this disclosure, the collaborative weights may be model parameters to be fine-tuned in a large-scale model built on a base model. The process of fine-tuning the large-scale model is achieved by adjusting only the collaborative weights, and inference tasks such as text prediction and image generation are performed on the large-scale model based on the fine-tuned collaborative weights and base weights.

[0038] According to embodiments of the present disclosure, a cooperative computation task includes a first cooperative task for obtaining an intermediate cooperative feature by processing a target feature and a first cooperative subweight, and a second cooperative task for obtaining a target cooperative feature by processing the intermediate cooperative feature and a second cooperative subweight.

[0039] According to embodiments of the present disclosure, the first and second cooperative subweights are determined by processing the cooperative weights according to a matrix multiplication mechanism of a general matrix.

[0040] According to embodiments of this disclosure, the first and second cooperative subweights are determined according to a matrix multiplication mechanism of general matrices, by dividing the cooperative weight W_LoRA into a first cooperative subweight W_LoRA_A and a second cooperative subweight W_LoRA_B based on the GEMM (General Matrix to Matrix Multiplication) mechanism. The matrix dimensions of the first cooperative subweight W_LoRA_A and the second cooperative subweight W_LoRA_B are both smaller than the matrix dimension of the cooperative subweight W_LoRA. Therefore, the computational task of multiplying the cooperative weight W_LoRA by the target feature is determined as a first cooperative task of multiplying the first cooperative subweight W_LoRA_B by the target feature, and a second cooperative task of multiplying the intermediate cooperative feature by the second cooperative subweight W_LoRA_B.

[0041] Cooperative computing tasks can be understood as LoRA (Low-Rank Adaptation) computing tasks. Therefore, cooperative computing tasks can be used to support basic computing tasks, accelerating the computational efficiency of large models performing the computing tasks and saving computational overhead on target computing units.

[0042] According to embodiments of this disclosure, target basic features can be obtained by performing a basic computation task using a target computation unit, which is used to process basic weights and target features to be processed.

[0043] According to embodiments of this disclosure, the basic computation task may include performing matrix multiplication operations on basic weights and target features to be processed to obtain target basic features. The fusion result between the target basic features and target co-features can represent the computational result of the combination of associated basic weights and co-features in the current large-scale model.

[0044] According to embodiments of this disclosure, a target basic feature and a target co-feature are merged to obtain the next feature to be processed. This may include adding the target basic feature and the target co-feature to obtain the next feature to be processed. It should be understood that the next feature to be processed can also be used to perform the next co-computation and basic computation tasks of a large-scale model.

[0045] In the embodiments of this disclosure, collaborative tasks can be understood as collaborative computation tasks, and basic tasks can be understood as basic computation tasks.

[0046] According to the embodiments of this disclosure, the features to be processed can be determined based on initial features. For example, if the collaborative computing task and the basic computing task are the first computing tasks of a large-scale model, the features to be processed can be obtained based on initial features. Initial features are obtained based on input data. The input data may be text data. The input text data can be processed by tokenization or embedding to obtain initial features.

[0047] According to embodiments of this disclosure, or the feature to be processed as a target, the target computation unit may obtain it by executing the preceding co-computation task and the preceding basic computation task. For example, if the co-computation task and the basic computation task are the nth computation task out of N computation tasks in a large model, the feature to be processed as a target may obtain it by executing the (n-1)th computation task (the (n-1)th co-computation task and the (n-1)th basic computation task), where n is an integer greater than 1 and less than N.

[0048] To facilitate the explanation of the task execution method provided by the embodiments of this disclosure, in these embodiments, the first cooperative subweight W_LoRA_A can be represented as matrix A, with dimension m*LoRA_rank, the second cooperative subweight W_LoRA_B can be represented as matrix B, with dimension n*LoRA_rank, where LoRA_rank can be represented as the rank matrix for the cooperative weights. It should be understood that the cooperative weight W_LoRA = A*B. m*n can represent the dimension of the cooperative weight matrix. The target feature to be processed can be represented by matrix MB1, with dimension m*n, and the base weight can be represented by W_Base.

[0049] According to embodiments of this disclosure, the target calculation unit can obtain intermediate cooperative features by performing a matrix multiplication operation on the first cooperative subweights and the feature to be processed. For example, the first cooperative subweights (dimension m*LoRA_rank) can be matrix multiplied by the feature to be processed (matrix dimension m*k), and the resulting intermediate cooperative feature dimension can be m*lora_rank.

[0050] According to embodiments of this disclosure, a matrix multiplication operation can be performed on the target calculation unit, the second cooperative subweight, and the intermediate cooperative feature to obtain the next feature to be processed. For example, the second cooperative subweight B (matrix dimension n*lora_rank) can be matrix multiplied with the intermediate cooperative feature (matrix dimension m*lora_rank), and the dimension of the obtained next feature to be processed may be m*n.

[0051] According to the embodiments of this disclosure, since the rank dimension lora_rank of the first and second cooperative subweights may be less than m or n, the cooperative tasks are determined to be performed sequentially as a first cooperative task and a second cooperative task, and the dimension of the matrix multiplication between the first and second cooperative tasks is reduced, thereby reducing the computational overhead when the target computing unit performs the cooperative tasks, improving the computational efficiency of the target computing unit, and reducing the energy consumption level generated in the process by which the target computing unit performs computation tasks on large models. This makes it possible to load large models on electronic devices with low computational performance.

[0052] Figure 3 schematically shows a schematic diagram of the principle of the task execution method used in the large-scale model according to the embodiment of this disclosure.

[0053] As shown in Figure 3, the i-th target feature to be processed may be a target feature that has already been processed by a large-scale model. The target calculation unit may obtain a calibrated i-th target feature by multiplying the i-th target feature by a first co-calibration parameter. The calibrated i-th target feature and the calibrated first co-subweight W_Lora_A can be input to the first co-task module 311. The first co-task module 311 can use the target calculation unit to perform the first co-task and obtain an intermediate co-feature. The calibrated first co-subweight W_Lora_A may be obtained offline after the initial first co-subweight W_Lora_A has been calculated, based on the result of dividing the initial first co-subweight by the first co-calibration parameter.

[0054] As shown in Figure 3, the calibrated intermediate co-function features can be obtained by multiplying the intermediate co-function features by a second co-function calibration parameter. The calibrated intermediate co-function features and the calibrated second co-function subweights W_LoRA_B can be input to the second co-function task module 312. The second co-function task module 312 can perform the second co-function task using the target calculation unit to obtain the calibrated target co-function features. The calibrated second co-function subweights W_LoRA_B can be obtained offline after the initial second co-function subweights have been calculated, based on the division result between the initial second co-function subweights and the second co-function calibration parameter.

[0055] As shown in Figure 3, the target calculation unit can also obtain a calibrated i-th basic calibration feature by multiplying the i-th target feature to be processed by a basic calibration parameter. The i-th basic calibration feature and the calibrated basic weight W_Base are input to the basic calculation task module 320, which can use the target calculation unit to perform a basic calculation task and obtain a calibrated target basic feature. The i-th target feature to be processed is obtained by adding the calibrated target co-feature feature and the calibrated target basic feature.

[0056] According to embodiments of this disclosure, a first collaborative task includes a plurality of first collaborative subtasks.

[0057] According to embodiments of the present disclosure, performing a collaborative computation task using a target computation unit, depending on the features to be processed, may include using the target computation unit to read a first collaborative sub-subtask, a first collaborative sub-weight, and a sub-features to be processed from a target storage unit.

[0058] According to embodiments of this disclosure, sub-features to be processed can be obtained by partitioning the feature to be processed. For example, the feature to be processed can be partitioned row by row and / or column by column by row by row and / or column by column according to a pre-defined partitioning strategy to obtain the sub-features to be processed. For example, the feature to be processed may be a 1000*1000-dimensional feature matrix, and the sub-matrix to be processed may be a 1000*100-dimensional matrix.

[0059] According to embodiments of this disclosure, a first cooperative subtask is performed using a target calculation unit based on a first cooperative subweight and a subfeature to be targeted for processing, and a first intermediate cooperative subfeature is obtained.

[0060] According to embodiments of this disclosure, multiple first cooperative subtasks are associated with the same first cooperative subweight. For example, the i-th first cooperative subtask uses a target computation unit to perform a matrix multiplication operation on the same first cooperative subweight and the i-th target subfeature to be processed to obtain the i-th first intermediate cooperative subfeature, where i is a positive integer greater than 0.

[0061] According to embodiments of this disclosure, when a target feature is divided into N target sub-features, a target calculation unit is used to obtain N first intermediate cooperative sub-features by matrix multiplying each of the N target sub-features by a first cooperative subweight. Each of the N first intermediate cooperative subweights may correspond to one of the N first cooperative subtasks.

[0062] In one example, the goal computation unit may contain N units, and the i-th goal computation unit can perform the i-th first cooperative subtask. For example, the i-th goal computation unit can perform a matrix multiplication operation on the first cooperative subweight and the i-th goal subfeature to be processed to obtain the i-th first intermediate cooperative subweight.

[0063] According to embodiments of this disclosure, the first intermediate cooperative sub-features acquired by each of the multiple target computing units are written to a target storage unit, and the intermediate cooperative features can be acquired without communication interaction between the multiple target computing units. This avoids the generation of communication overhead by the target computing units by transferring intermediate values ​​of the computed intermediate cooperative features between the multiple target computing units. At the same time, by dividing the target feature to be processed, which has a large tensor dimension, into the target sub-features to be processed, which have a small tensor dimension, and by using multiple target computing units to each execute a part of the first cooperative task, multiple first cooperative subtasks can be executed in parallel using multiple target computing units to acquire intermediate cooperative features. This reduces the impact of computational performance, such as the number of threads and memory space of the target computing units, on the computational efficiency of executing the first cooperative task during the task execution process of large models, thereby improving the overall task execution efficiency of large models.

[0064] According to embodiments of this disclosure, intermediate collaborative features can be determined based on first intermediate collaborative sub-features corresponding to each of a plurality of first collaborative subtasks. A plurality of first intermediate collaborative sub-features can be merged, for example, by all-reducing a plurality of first intermediate collaborative sub-features according to the division dimension to obtain an intermediate collaborative feature. The intermediate collaborative feature can be stored in a target memory unit.

[0065] According to embodiments of this disclosure, a second collaborative task may include a plurality of second collaborative subtasks.

[0066] According to embodiments of the present disclosure, performing a collaborative computation task using a target computation unit, depending on the features to be processed, includes using the target computation unit to read a second intermediate collaborative sub-features corresponding to a second collaborative subtask from a target storage unit, and using the target computation unit to perform a second collaborative subtask based on the second intermediate collaborative sub-features and the second collaborative sub-weights to obtain target collaborative sub-features.

[0067] According to embodiments of this disclosure, a second intermediate co-feature sub-feature is determined based on the intermediate co-feature. For example, the intermediate co-feature can be partitioned into rows and / or columns based on a pre-configured partitioning strategy to obtain the second intermediate co-feature sub-feature. For example, the intermediate co-feature may be a 1000*1000-dimensional feature matrix, and the second intermediate co-feature sub-feature may be a 1000*100-dimensional matrix.

[0068] According to embodiments of this disclosure, multiple second cooperative subtasks are associated with the same second cooperative subweight. For example, for the j-th second cooperative subtask, the target computation unit can be used to perform a matrix multiplication operation on the same second cooperative subweight and the j-th second intermediate cooperative subweight to obtain the j-th target cooperative subfeature.

[0069] In one example, the goal computation unit may contain N units, and the j-th goal computation unit can perform the j-th second cooperative subtask. For example, the j-th goal computation unit can perform a matrix multiplication operation on the second cooperative subweight and the j-th second second intermediate cooperative subfeature to obtain the j-th goal cooperative subfeature.

[0070] According to embodiments of this disclosure, target co-communication sub-features acquired by each of multiple target computation units are written to a target storage unit, and the target co-communication features can be acquired without communication interaction between the multiple target computation units. This avoids the generation of communication overhead by the target computation units by transferring intermediate values ​​of the computationally obtained intermediate co-communication features between the multiple target computation units. At the same time, multiple target computation units can be used to execute multiple second co-communication subtasks in parallel and acquire target co-communication features by dividing the intermediate co-communication features of large tensor dimensions into second co-communication sub-features of small tensor dimensions and executing a portion of the second co-communication task using multiple target computation units. This reduces the impact of computational performance, such as the number of threads and memory space of the target computation units, on the computational efficiency of executing the second co-communication task during the task execution process of large models, thereby improving the overall task execution efficiency of large models.

[0071] According to embodiments of this disclosure, the target co-feature may be determined based on a second intermediate co-feature corresponding to each of a plurality of second co-feature subtasks. For example, the target co-feature may be obtained by calling a fusion function and matrix-adding the plurality of second intermediate co-feature subtasks.

[0072] Figure 4 schematically shows a schematic diagram of the principle of a task execution method used in a large-scale model according to other embodiments of this disclosure.

[0073] As shown in Figure 4, the electronic device 400 may include a plurality of target calculation units and target storage units 440. A plurality of target calculation units 411, 412, 421, 422, 431, and 432. Target calculation units 411 and 412 may be used to perform a first cooperative subtask, target calculation units 421 and 422 may be used to perform a second cooperative subtask, and target calculation units 431 and 432 may be used to perform a basic calculation task. The target storage unit 440 may include a first target storage area 441 and a second target storage area 442.

[0074] As shown in Figure 4, the target calculation unit 411 can read the first cooperative subweight W_Lora_A and the first target subfeature X11 to be processed from the target storage unit 440, and performs a matrix multiplication operation on the first cooperative subweight W_Lora_A and the first target subfeature X11 to be processed to obtain the first intermediate cooperative subfeature. The target calculation unit 412 reads the first cooperative subweight W_Lora_A and the second target subfeature X12 to be processed from the target storage unit 440, and performs a matrix multiplication operation on the first cooperative subweight W_Lora_A and the second target subfeature X12 to be processed to obtain the second intermediate cooperative subfeature. The first and second intermediate cooperative subfeatures can be written to the first target storage area 441. The fusion function can be called to process the first and second intermediate co-function sub-features to obtain the intermediate co-function feature, and then the intermediate co-function feature can be split based on a pre-configured splitting strategy to obtain the first second intermediate co-function sub-features X21 and the second second intermediate co-function sub-features X22.

[0075] As shown in Figure 4, the target calculation unit 421 can read the second cooperative subweight W_Lora_B and the first second intermediate cooperative subfeature X21 from the target storage unit 440, and performs a matrix multiplication operation on the second cooperative subweight W_Lora_B and the first second intermediate cooperative subfeature X21 to obtain the first target cooperative subfeature. The target calculation unit 422 can read the second cooperative subweight W_Lora_B and the second second intermediate cooperative subfeature X22 from the target storage unit 440, and performs a matrix multiplication operation on the second cooperative subweight W_Lora_B and the second second intermediate cooperative subfeature X22 to obtain the second target cooperative subfeature. The first and second target cooperative subfeatures can be written to the second target storage area 442. The fusion function can be called to process the first and second target cooperative subfeatures and obtain the target cooperative feature.

[0076] As shown in Figure 4, the target calculation unit 431 may read the first basic subweight W_Base1 and the feature MB1 to be processed from the target storage unit 440. The target calculation unit 431 can perform a matrix multiplication operation on the first basic subweight W_Base1 and the feature MB1 to be processed to obtain an intermediate basic feature MB2. The intermediate basic feature can be written to the first target storage area 441. The target calculation unit 432 can read the second basic subweight W_Base2 and the intermediate basic feature MB2 from the first target storage area 441. The target calculation unit 432 can perform a matrix multiplication operation on the second basic subweight W_Base2 and the intermediate basic feature MB2 to obtain a target basic feature. The target basic feature can be written to the second target storage area 442. A fusion function can be called to process the target basic feature and the target co-features to obtain the next feature to be processed.

[0077] According to the embodiments of this disclosure, the basic weight W_Base can be processed based on the matrix multiplication mechanism of a general matrix to obtain a first basic subweight W_Base1 and a second basic subweight W_Base2.

[0078] Figure 5 schematically shows a flowchart of a task execution method used in a large-scale model according to other embodiments of this disclosure.

[0079] As shown in Figure 5, the task execution method may include operations S501 to S506.

[0080] In operation S501, the feature to be processed is obtained. The tensor dimension of the feature to be processed can be m*k.

[0081] In operation S502, the target basic feature calculation is performed. For example, the target calculation unit is used to process the basic weights and the features to be processed to obtain the target basic features.

[0082] In operation S503, it is determined whether the dimension of the feature to be processed is greater than a predetermined dimensional threshold. For example, it is possible to determine whether the dimension m of the feature to be processed is greater than a predetermined dimensional threshold.

[0083] If the result of operation S503 is "yes", operation S504 can be executed, and the Tensor Core will perform the computation. For example, the Tensor Core (also known as the Tensor Computation Core) in the target computation unit will perform the first and second collaborative tasks to obtain the target collaborative features.

[0084] In operation S506, a feature fusion operation is performed, for example, by adding the target co-features and target basic features to obtain the next feature to be processed.

[0085] If the result of operation S503 is "none", operation S505 is executed, and calculations can be performed using the CUDA (Compute Unified Device Architecture) computing core. For example, the first and second collaborative tasks can be performed using the CUDA core in the target computing unit to obtain the target collaborative features. Next, operation S506 is executed to obtain the next target feature to be processed.

[0086] According to the embodiments of this disclosure, by determining the dimensions of the features to be processed and calling the corresponding type of computing core within the target computing unit to execute the collaborative computing task according to the determination result, it is possible to maximize the use of the architecture of the computing cores of the target computing unit (such as a GPU) and improve the execution efficiency of large-scale model tasks.

[0087] Figure 6 schematically shows an application scenario diagram of a task execution method used in a large-scale model according to the embodiment of this disclosure.

[0088] As shown in Figure 6, the task execution method used in large-scale models can be realized by setting up a task management process 610 and a task execution process 620. The task execution process 620 can execute operation S601 to request video memory space from the task management process 610.

[0089] The task management process 620 may perform operation S602 to share the weight storage address of the cooperative weight with the task execution process 620. The task execution process 620 may perform operation S603 to execute the cooperative computation task. For example, the first cooperative subweight or the second cooperative subweight can be called from the video memory of the target computation unit via the shared weight storage address to execute the first cooperative subtask or the second cooperative subtask.

[0090] The task management process 610 may also perform operation S604 to update the cooperative weights. For example, at least one of the weight matrices of the first cooperative subweights or the second cooperative subweights may be updated. When the task execution process 620 performs operation S605 and executes a subsequent cooperative computation task, it may retrieve the updated first cooperative subweights or the updated second cooperative subweights from the video memory of the target computation unit through the weight storage address previously shared based on the task management process, thereby executing the first cooperative subtask or the second cooperative subtask.

[0091] According to embodiments of this disclosure, by sharing weight storage addresses in a task management process and updating the first and second collaborative subweights, the task execution process can, under unrecognized conditions, asynchronously load the updated first collaborative subweights to execute a collaborative computation task, or load the updated second collaborative subweights to execute a collaborative computation task, thereby enabling hot updates of the first and second collaborative subweights while a large model is executing a task, and reducing the computation time generated for updating weight storage.

[0092] Figure 7 schematically shows a schematic diagram of the principle of a task execution method used in a large-scale model according to another embodiment of this disclosure.

[0093] As shown in Figure 7, the features to be processed in the large-scale model can be input to the basic task execution module 711 and the first cooperative task execution module 721, respectively. The basic task execution module 711 processes the features to be processed by executing a basic calculation task, and the obtained calculation results can be input to the basic alignment module 712 and merged to obtain the target basic features. The first cooperative task execution module 721 can process the features to be processed by executing multiple first cooperative subtasks and obtain multiple first intermediate cooperative features. Multiple first intermediate cooperative subfeatures can be input to the cooperative alignment module 722 and merged to obtain multiple second intermediate cooperative subfeatures. Multiple second intermediate cooperative subfeatures are input to the second cooperative task execution module 723. The second cooperative task execution module 723 can execute multiple second cooperative subtasks and obtain multiple second intermediate cooperative subfeatures. The target basic features and multiple second intermediate subfeatures can be transmitted to the target alignment module 730 to obtain the next features to be processed.

[0094] As shown in Figure 7, a basic task flow can be realized in which the basic model executes basic computation tasks based on the basic task execution module 711 and the basic alignment module 712. Furthermore, a collaborative task flow can be implemented in which collaborative computation tasks are executed based on the first collaborative task execution module 721, the collaborative alignment module 722, and the second collaborative task execution module 723.

[0095] According to embodiments of this disclosure, the feature to be processed as a target includes the text feature to be processed as a target, the initial feature is determined based on the initial text, and the execution result of the target computation unit performing the basic computation task and the cooperative settlement task is the output text corresponding to the initial text.

[0096] For example, the initial text may be the question text entered by the user, and the output text may be the answer text corresponding to the question text.

[0097] This disclosure will be understood by taking the input data of a large-scale model as text, as an example. However, this disclosure is not limited to this, and the input data of a large-scale model may also be images or audio.

[0098] In some embodiments, the target features to be processed are image features to be processed, the initial features are obtained based on the initial image, and the target computation unit performs multiple basic computation tasks and multiple cooperative settlement tasks. The execution result is an output result corresponding to the initial image. As a result, it may be an adjusted image or text. If the input data is an image, the input image can be processed based on a patch embedding operation to obtain the image features to be processed. The above-mentioned target features may be edges or colors in the image.

[0099] Figure 8 schematically shows a block diagram of a task execution device used in a large-scale model according to the embodiment of this disclosure.

[0100] As shown in Figure 8, the task execution unit 80 for large-scale models may include a target storage unit 810 and a target calculation unit 820.

[0101] The target memory unit 810 stores the cooperative computing task.

[0102] The target computation unit 820 performs a cooperative computation task according to the target feature to be processed and obtains a target cooperative feature, where the cooperative computation task includes a first cooperative task for obtaining an intermediate cooperative feature by processing the target feature and a first cooperative subweight, and a second cooperative task for obtaining a target cooperative feature by processing the intermediate cooperative feature and a second cooperative subweight, the first and second cooperative subweights being determined by processing the cooperative weights according to a matrix multiplication mechanism of a general matrix, and the target basic feature and target cooperative feature are merged to obtain the next target feature to be processed, where the target basic feature is obtained by performing a basic computation task in the target computation unit, and the basic computation task is used to process the basic weight and the target feature to be processed.

[0103] According to embodiments of this disclosure, a first collaborative task includes a plurality of first collaborative subtasks.

[0104] According to embodiments of the present disclosure, the target calculation unit is configured to perform a cooperative calculation task according to the features to be processed, by reading a first cooperative subweight and a subfeature to be processed from the target storage unit, which corresponds to a first cooperative subtask, and which is obtained by dividing the features to be processed; and by using the target calculation unit to perform a first cooperative subtask based on the first cooperative subweight and the subfeature to be processed, to obtain a first intermediate cooperative subfeature, where the intermediate cooperative feature is determined based on the first intermediate cooperative subfeature corresponding to each of the multiple first cooperative subtasks.

[0105] According to embodiments of this disclosure, multiple first cooperative subtasks are associated with the same first cooperative subweight.

[0106] According to embodiments of this disclosure, the second collaborative task includes a plurality of second collaborative subtasks.

[0107] According to embodiments of the present disclosure, the target computation unit is configured to perform a cooperative computation task according to the features to be processed, by reading a second intermediate cooperative sub-feature corresponding to a second cooperative subtask from the target storage unit, the second intermediate cooperative sub-feature being determined based on the intermediate cooperative feature, and by performing a second cooperative subtask based on the second intermediate cooperative sub-feature and the second cooperative sub-weight to obtain a target cooperative sub-feature, where the target cooperative feature is determined according to the target cooperative sub-feature corresponding to each of the multiple second cooperative subtasks.

[0108] According to embodiments of this disclosure, multiple second cooperative subtasks are associated with the same second cooperative subweight.

[0109] According to the embodiments of this disclosure, the features to be targeted for processing are determined based on the initial features.

[0110] According to embodiments of this disclosure, the features to be processed as targets may be obtained by the target computation unit performing the previous cooperative computation task and the previous basic computation task.

[0111] According to embodiments of this disclosure, the feature to be processed as a target includes the text feature to be processed as a target and is determined based on the initial feature, and the execution result of the target computation unit performing the basic computation task and the cooperative settlement task is the output text corresponding to the initial text.

[0112] Figure 9 schematically shows a block diagram of a task execution device used in a large-scale model according to an embodiment of this disclosure.

[0113] As shown in Figure 9, the task execution device 9000 used in the large-scale model may include the task execution unit 80 used in the large-scale model.

[0114] In the proposed technology described herein, the collection, storage, use, processing, transfer, provision, disclosure, and application of such user personal information will all comply with the provisions of relevant laws and regulations, employ necessary confidentiality measures, and will not violate public order and morals.

[0115] According to embodiments of the present disclosure, the present disclosure also provides electronic devices, readable storage media, and computer programs.

[0116] According to embodiments of the present disclosure, an electronic device includes at least one processor and a memory communicably connected to the at least one processor, the memory storing instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the method.

[0117] According to embodiments of the present disclosure, a non-temporary computer-readable storage medium stores computer instructions, and the computer instructions cause a computer to execute the method.

[0118] According to embodiments of this disclosure, a computer program achieves the above method when executed by a processor.

[0119] Figure 10 schematically shows a block diagram of electronic equipment suitable for implementing a code generation method based on a large-scale model according to one embodiment of the present disclosure. The electronic equipment may represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, large computers, and other suitable computers. The electronic equipment may also represent various forms of mobile devices, such as personal digital processes, mobile phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are illustrative and do not limit the implementation of the present disclosure as described herein and / or requested.

[0120] As shown in Figure 10, the device 1000 includes a computing unit 1001 and can perform various appropriate operations and processes based on computer programs stored in read-only memory (ROM) 1002 or computer programs loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 can further store various programs and data necessary for the operation of the device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected by a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0121] Multiple components in device 1000 are connected to the I / O interface 1005 and include, for example, an input unit 1006 such as a keyboard or mouse, an output unit 1007 such as various types of displays or speakers, a storage unit 1008 such as a magnetic disk or optical disk, and a communication unit 1009 such as a network card, modem, or wireless communication transceiver. The communication unit 1009 allows device 1000 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0122] The computing unit 1001 can be various general-purpose and / or dedicated processing modules having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various operational machine learning model algorithm computing units, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs each of the methods and processes described above, for example, a code generation method based on a large model. For example, in some embodiments, the code generation method based on a large model is implemented as a computer software program and is tangibly contained in a machine-readable medium, for example, a memory unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed in the device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the code generation method based on a large model described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a code generation method based on a large model by any other suitable method (e.g., firmware).

[0123] Various embodiments of the systems and technologies described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), dedicated integrated circuits (ASICs), dedicated standard products (ASSPs), systems of on-chip (SOCs), load-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can be implemented in one or more computer programs that run and / or interpret on a programmable system including at least one programmable processor, the programmable processor may be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] Program code for carrying out the methods of this disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations defined in the flowcharts and / or block diagrams are performed. The program code may be fully executed by a machine, partially executed by a machine, partially executed by an instrument and partially executed by a remote machine as a standalone software package, or fully executed by a remote machine or server.

[0125] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program used by or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatus, or any appropriate combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, convenient compact read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any appropriate combination of the above.

[0126] To provide user interaction, a computer may be equipped with the systems and technologies described herein, the computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor), a keyboard and a pointing device (e.g., a mouse or trackball), the user providing input to the computer via the keyboard and the pointing device. Other types of devices may further provide user interaction. For example, the feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form (including voice input, speech input, or tactile input).

[0127] The systems and technologies described herein can be implemented in a computing system including background components (e.g., a data server), a computing system including middleware components (e.g., an application server), or a computing system including front-end components (e.g., a user computer having a graphical user interface or a web browser, through which the user can interact with embodiments of the systems and technologies described herein), or in a computing system including any combination of such background components, middleware components, or front-end components. Components of the system can be connected to one another by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0128] A computer system can include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship is generated by computer programs running on corresponding computers that have a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers combined with blockchain technology.

[0129] It should be understood that various forms of flows as shown above may be used, and each operation may be sorted, added, or deleted. For example, each step described herein may be performed in parallel, sequentially, or in a different order, as long as this disclosure can achieve the desired results of the disclosed invention.

[0130] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. The process involves performing a collaborative computation task using a target computation unit according to the target features to be processed, thereby obtaining target collaborative features, wherein the collaborative computation task includes a first collaborative task for obtaining intermediate collaborative features by processing the target features and a first collaborative subweight, and a second collaborative task for obtaining the target collaborative features by processing the intermediate collaborative features and a second collaborative subweight, the first and second collaborative subweights being determined by processing collaborative weights according to a general matrix multiplication mechanism. The process involves fusing the target basic feature and the target co-features to obtain the next feature to be processed, wherein the target basic feature is obtained by performing a basic computation task using the target computation unit, and the basic computation task is used to process the basic weights and the feature to be processed. The first collaborative task includes a plurality of first collaborative subtasks, Executing a collaborative computing task using a target computing unit in accordance with the aforementioned characteristics to be processed is: Using the target calculation unit, read the first cooperative subweight and the sub-features to be processed for the target, corresponding to the first cooperative subtask, wherein the sub-features to be processed for the target are obtained by dividing the features to be processed for the target. The process includes performing the first cooperative subtask using the target calculation unit based on the first cooperative subweight and the sub-features to be processed, thereby obtaining a first intermediate cooperative sub-feature, wherein the intermediate cooperative feature is determined based on the first intermediate cooperative sub-features corresponding to each of the plurality of first cooperative subtasks. A task execution method used in large-scale models.

2. Multiple of the aforementioned first cooperative subtasks are associated with the same aforementioned first cooperative subweight. The method according to claim 1.

3. The aforementioned second collaborative task includes a plurality of second collaborative subtasks, Here, performing a collaborative computing task using a target computing unit according to the characteristics to be processed is: Using the target calculation unit, read the second intermediate cooperative sub-features corresponding to the second cooperative subtask from the target storage unit, wherein the second intermediate cooperative sub-features are determined based on the intermediate cooperative features. The process involves using the target calculation unit to perform the second cooperative subtask based on the second intermediate cooperative sub-features and the second cooperative sub-weights to obtain a target cooperative sub-features, wherein the target cooperative feature is determined according to the target cooperative sub-features corresponding to each of the plurality of second cooperative subtasks. The method according to claim 1.

4. Multiple of the aforementioned second cooperative subtasks are associated with the same aforementioned second cooperative subweight. The method according to claim 3.

5. The feature to be processed is determined based on the initial feature, or The features to be processed are obtained by the target calculation unit performing the previous cooperative calculation task and the previous basic calculation task. The method according to claim 1.

6. The feature to be processed includes the text feature to be processed, the initial feature is determined according to the initial text, and the result of the target calculation unit performing the basic calculation task and the cooperative calculation task is the output text corresponding to the initial text. The method according to claim 5.

7. A target memory unit that stores cooperative computing tasks, A cooperative computation task is performed according to the feature to be processed as a target, and a target cooperative feature is obtained, wherein the cooperative computation task includes a first cooperative task for obtaining an intermediate cooperative feature by processing the feature to be processed as a target and a first cooperative subweight, and a second cooperative task for obtaining the target cooperative feature by processing the intermediate cooperative feature and a second cooperative subweight, the first cooperative subweight and the second cooperative subweight being determined by processing the cooperative weights according to a matrix multiplication mechanism of a general matrix. The target basic features and the target co-features are merged to obtain the next feature to be processed as a target. Here, the target basic features are obtained by performing a basic computation task in the target computation unit, and the basic computation task is used to process the basic weights and the feature to be processed as a target. Includes the target calculation unit, The first collaborative task includes a plurality of first collaborative subtasks, The target calculation unit performs a collaborative calculation task according to the features to be processed as a target. The process involves reading a first cooperative subweight and a sub-feature to be processed from the target memory unit, wherein the sub-feature to be processed is obtained by dividing the feature to be processed. The system is configured to perform the following: execute the first cooperative subtask using the target calculation unit based on the first cooperative subweight and the sub-features to be processed as targets to obtain a first intermediate cooperative sub-feature, wherein the intermediate cooperative feature is determined based on the first intermediate cooperative sub-features corresponding to each of the plurality of first cooperative subtasks. A task execution device used in large-scale models.

8. Multiple of the aforementioned first cooperative subtasks are associated with the same aforementioned first cooperative subweight. The apparatus according to claim 7.

9. The aforementioned second collaborative task includes a plurality of second collaborative subtasks, Here, the target calculation unit performs a collaborative calculation task according to the features to be processed as a target. The process involves reading a second intermediate cooperative sub-feature corresponding to the second cooperative subtask from the target memory unit, wherein the second intermediate cooperative sub-feature is determined based on the intermediate cooperative feature. The method is configured to perform the second cooperative subtask to obtain a target cooperative subfeature, where the target cooperative subfeature is determined according to the target cooperative subfeature corresponding to each of the multiple second cooperative subtasks. The apparatus according to claim 7.

10. Multiple of the aforementioned second cooperative subtasks are associated with the same aforementioned second cooperative subweight. The apparatus according to claim 9.

11. The feature to be processed is determined based on the initial feature, or The features to be processed are obtained by the target calculation unit performing the previous cooperative calculation task and the previous basic calculation task. The apparatus according to claim 7.

12. The feature to be processed includes the text feature to be processed, the initial feature is determined according to the initial text, and the result of the target calculation unit performing the basic calculation task and the cooperative calculation task is the output text corresponding to the initial text. The apparatus according to claim 11.

13. The apparatus includes the apparatus described in any one of claims 7 to 12. Task execution devices used in large-scale models.

14. At least one processor, Includes a memory that is communicably connected to at least one of the processors, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the method according to any one of claims 1 to 6. electronic equipment.

15. A non-temporary, computer-readable storage medium storing computer instructions that cause a computer to perform the method described in any one of claims 1 to 6.

16. A computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images