Task execution method and apparatus, model training method and apparatus, electronic equipment, storage medium, and computer program

Abstract

The present invention provides a method for executing tasks, a method for training models, apparatus, electronic equipment, storage media, and programs. [Solution] The task execution method reads the hyperparameters of the target network layer in the target model from the memory unit, executes a first computation subtask in the computation task based on the parameters using the operator unit, obtains a first feature output from the layer, and if the hyperparameters of the layer are multiplexed, executes a second computation subtask in the computation task based on the first feature obtained using the operator unit, and obtains a second feature output from the layer. The first and second computation subtasks are executed sequentially in the computation task, and the model output result of the target model is identified using the operator unit based on the second feature read from the memory unit.

Description

Technical Field

[0004] , , ,

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly to the fields of deep learning technology and large-scale model technology. More specifically, the present disclosure provides a task execution method and apparatus, a model training method and apparatus, an electronic device, a storage medium, and a computer program.

Background Art

[0002] With the rapid development of artificial intelligence technology, it is possible to process various types of data such as text, images, and audio based on large-scale models, and to meet the actual needs of various scenarios according to the output results of the large-scale models.

Summary of the Invention

Means for Solving the Problems

[0003] The present disclosure provides a task execution method and apparatus, a model training method and apparatus, an electronic device, a storage medium, and a computer program.

[0004] According to one aspect of the present disclosure, reading hyperparameters of a target network layer in a target model from a storage unit, executing a first calculation subtask in a calculation task based on the hyperparameters of the target network layer using an operator unit, and obtaining a first feature output from the target network layer; when multiplexing the hyperparameters of the target network layer, executing a second calculation subtask in the calculation task based on the first feature read from the storage unit using the operator unit, and obtaining a second feature output from the target network layer, where the first calculation subtask and the second calculation subtask are subtasks sequentially executed in the calculation task; and specifying a model output result of the target model using the operator unit based on the second feature read from the storage unit. A task execution method is provided.

[0005] According to another aspect of this disclosure, a model training method is provided which includes: reading the hyperparameters of the initial network layer in the initial model from a memory unit; using an operator unit to perform a first computation subtask in a computation task based on the hyperparameters of the initial network layer and obtaining a first initial feature output from the initial network layer; if the hyperparameters of the initial network layer are multiplexed, using an operator unit to perform a second computation subtask in a computation task based on the first initial feature read from the memory unit and obtaining a second initial feature output from the initial network layer, wherein the first and second computation subtasks are subtasks that are executed sequentially in the computation task; using an operator unit to identify the model output result of the initial model based on the second initial feature read from the memory unit; using an operator unit to identify target loss information based on the model output result of the initial model; and using an operator unit to train the initial model based on the target loss information and obtain a trained target model.

[0006] In other aspects of the present disclosure, a task execution device is provided, comprising a storage unit and an operator unit, the operator unit reading hyperparameters of a target network layer in a target model from the storage unit, executing a first computation subtask in a computation task based on the hyperparameters of the target network layer, obtaining a first feature output from the target network layer, and, if the hyperparameters of the target network layer are multiplexed, executing a second computation subtask in a computation task based on the first feature read from the storage unit, obtaining a second feature output from the target network layer, the first and second computation subtasks being subtasks executed sequentially in a computation task, and the operator unit being configured to identify the model output result of the target model based on the second feature read from the storage unit.

[0007] In other aspects of the present disclosure, a model training device is provided, comprising a memory unit and an operator unit, the operator unit reads the hyperparameters of the initial network layer in the initial model from the memory unit, performs a first computation subtask in a computation task based on the hyperparameters of the initial network layer, obtains a first initial feature output from the initial network layer, and, if the hyperparameters of the initial network layer are multiplexed, performs a second computation subtask in the computation task based on the first initial feature read from the memory unit, obtains a second initial feature output from the initial network layer, the first and second computation subtasks are subtasks that are executed sequentially in the computation task, the operator unit identifies the model output result of the initial model based on the second initial feature read from the memory unit, identifies target loss information based on the model output result of the initial model, trains the initial model based on the target loss information, and obtains a trained target model.

[0008] According to another aspect of the present disclosure, an electronic device is provided which includes at least one processor and a memory communicably connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a task execution method or a model training method.

[0009] Other aspects of this disclosure provide a non-temporary, computer-readable storage medium that stores computer instructions for causing a computer to perform a task execution method or a model training method.

[0010] According to other aspects of this disclosure, a computer program is provided that, when executed by a processor, implements a method for performing a task or training a model.

[0011] The information described in this section should be understood not to represent key points or important features of the embodiments of this disclosure, nor to limit the scope of this disclosure. Other features of this disclosure will be readily apparent from the following description. [Brief explanation of the drawing]

[0012] The drawings are for the purpose of better understanding the present invention and do not limit it.

[0013] [Figure 1] Figure 1 schematically shows an exemplary system architecture to which the task execution method and apparatus according to the embodiments of this disclosure can be applied. [Figure 2] Figure 2 schematically shows a flowchart of the task execution method according to an embodiment of the present disclosure. [Figure 3] Figure 3 schematically shows a schematic diagram of the target model according to an embodiment of this disclosure. [Figure 4] Figure 4 schematically shows a principle diagram for identifying a target model according to the embodiments of this disclosure. [Figure 5] Figure 5 schematically shows a flowchart of the model training method according to the embodiment of this disclosure. [Figure 6] Figure 6 schematically shows the principle of the model training method according to the embodiment of this disclosure. [Figure 7] Figure 7 schematically shows a block diagram of a task execution device according to an embodiment of the present disclosure. [Figure 8] Figure 8 schematically shows a block diagram of a model training device according to an embodiment of the present disclosure. [Figure 9] Figure 9 shows a schematic block diagram of an exemplary electronic device 900 for carrying out the task execution method and model training method of the embodiments of this disclosure. [Modes for carrying out the invention]

[0014] The following description will illustrate exemplary embodiments of the disclosure with reference to the drawings, but various details of the embodiments of the disclosure will be included for the sake of clarity, and these should be considered to be illustrative only. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and configurations will be omitted in the following description.

[0015] In the proposed technology disclosed herein, the acquisition, storage, and use of relevant user personal information are all in accordance with the provisions of relevant laws and regulations, necessary security measures are taken, and the technology does not violate public order and morals.

[0016] In the field of deep learning, large-scale models are widely applied in various application scenarios such as intelligent search, smart customer service, intelligent file editing, and intelligent device control. However, large-scale models have high inference costs during the scene application process, strict training conditions, and high deployment difficulty. Large-scale models can include large language models (LLMs), large image models, and large audio models.

[0017] Embodiments of this disclosure provide a task execution method, a training method, an apparatus, an electronic device, and a storage medium. The task execution method includes reading hyperparameters of a target network layer in a target model from a storage unit; executing a first computation subtask in a computation task based on the hyperparameters of the target network layer using an operator unit and obtaining a first feature output from the target network layer; if the hyperparameters of the target network layer are multiplexed, executing a second computation subtask in the computation task based on the first feature read from the storage unit using an operator unit and obtaining a second feature output from the target network layer; and identifying the model output result of the target model using an operator unit based on the second feature read from the storage unit, wherein the first and second computation subtasks are subtasks executed sequentially in the computation task.

[0018] According to the embodiments of this disclosure, when an operator unit executes a first computation subtask in a computation task based on the hyperparameters of a target network layer read from a storage unit, obtains a first feature output from the target network layer, and multiplexes the hyperparameters of the target network layer, the operator unit can execute the computation task by multiplexing the hyperparameters of the target network layer stored in the storage unit by executing a second computation subtask that is sequentially executed in the computation task based on the first feature using the operator unit, and the operator unit can execute the computation task by cyclically multiplexing the hyperparameters of the target network layer read from the storage unit and executing multiple subtasks in the computation task. This enables the simulation of deep data processing processes for multiple network layers by multiplexing the hyperparameters of the target network layer and sequentially executing multiple subtasks. Furthermore, it enables deep data processing processes for large-scale models with a large number of parameters under the condition that the number of hyperparameters stored in the memory unit is small. This brings the accuracy of the model output results of the target model closer to the accuracy of the output results of large-scale models with a large number of model parameters. This reduces the amount of memory space occupied by the memory unit, improves the convenience of deploying the target model, improves the inference cost and deployment cost of the target model, and improves the generalization capability of data processing by the target model.

[0019] Figure 1 schematically shows an exemplary system architecture to which the task execution method and apparatus according to the embodiments of this disclosure can be applied.

[0020] Note that FIG. 1 is merely an example of a system architecture to which embodiments of the present disclosure can be applied for those skilled in the art to understand the technical content of the present disclosure, and it does not mean that the embodiments of the present disclosure cannot be applied to other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the task execution method and apparatus can be applied may include terminal devices, but the terminal devices can implement the task execution method and apparatus according to the embodiments of the present disclosure without interacting with a server.

[0021] As shown in FIG. 1, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 provides a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types such as, for example, wired and / or wireless communication links.

[0022] Users can use the terminal devices 101, 102, and 103 to interact with the server 105 via the network 104 to send and receive messages and the like. For example, the terminal device 101 can send a request for text processing to the server 105 via the network 104.

[0023] Various communication client applications such as, for example, a knowledge browsing system application, a web browser application, a search system application, an instant messaging tool, a mailbox client, and / or social platform software (only for illustration) may be installed on the terminal devices 101, 102, 103.

[0024] The terminal devices 101, 102, 103 may be various electronic devices having a display and supporting web page browsing, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers.

[0025] Server 105 may be a server that provides various services, for example, a background management server (example only) that provides support for content viewed by the user using terminal devices 101, 102, and 103. The background management server can perform processing such as analysis on received data such as user requests and feed back the processing results (for example, web pages, information, or data acquired or generated in response to user requests) to the terminal devices.

[0026] Server 105 may be a cloud server, also called a cloud computing server or cloud host, and is a host product in a cloud computing service system that solves the problems of high management difficulty and low service scalability that exist in conventional physical hosts and VPS services (abbreviated as "Virtual Private Server" or "VPS"). The server may be a server in a distributed system or a server that combines blockchain.

[0027] The task execution method according to the embodiments of this disclosure may generally be executed by server 105. Accordingly, the task execution device according to the embodiments of this disclosure may generally be provided on server 105. The task execution method according to the embodiments of this disclosure may be executed by a server or server cluster that can communicate with terminal devices 101, 102, 103 and / or server 105, instead of server 105. Accordingly, the task execution device according to the embodiments of this disclosure may be provided on a server or server cluster that can communicate with terminal devices 101, 102, 103 and / or server 105, instead of server 105. For example, the method according to this disclosure can be executed using the computing unit of a hardware device on server 105 or a server node. According to the task execution method according to this disclosure, a multi-head self-attention mechanism can be converted to a group querier attention mechanism with low computing and memory resources.

[0028] Alternatively, the task execution method according to the embodiment of this disclosure may be executed by terminal devices 101, 102, or 103. Accordingly, the task execution device according to the embodiment of this disclosure may be provided in terminal devices 101, 102, or 103.

[0029] The number of terminal devices, networks, and servers shown in Figure 1 should be understood as illustrative only. Any number of terminal devices, networks, and servers may be used as required by the implementation.

[0030] Figure 2 schematically shows a flowchart of the task execution method according to an embodiment of this disclosure.

[0031] As shown in Figure 2, the task execution method includes operations S210 to S240.

[0032] In operation S210, the hyperparameters of the target network layer in the target model are read from the memory unit.

[0033] In operation S220, the operator unit is used to execute the first computation subtask in the computation task based on the hyperparameters of the target network layer, and the first feature output from the target network layer is obtained.

[0034] In operation S230, when multiplexing the hyperparameters of the target network layer, the operator unit executes a second computation subtask in the computation task based on the first feature read from the memory unit, and obtains the second feature output from the target network layer.

[0035] In operation S240, the operator unit is used to identify the model output result of the target model based on the second feature read from the memory unit.

[0036] According to embodiments of this disclosure, an operator unit may be a component for performing computational operations in a computing module. The computing module may include at least one of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and an Artificial Intelligence Computing Unit. The Artificial Intelligence Computing Unit may include at least one of a Neural Processing Unit (NPU), a Tensor Processing Unit (TPU), and a Kunlun Xin. An operator unit may be, for example, a CUDA computing core or a stream processor unit in a Graphics Processing Unit (GPU). The embodiments of this disclosure do not limit the specific type of operator unit, as long as it can perform data computation tasks.

[0037] According to embodiments of the present disclosure, the storage unit may be a device or component for storing data, for example, the storage unit may be any type of data caching component, such as an on-chip cache or an off-chip cache, and embodiments of the present disclosure do not limit the specific type of storage unit.

[0038] According to embodiments of this disclosure, the target model may be an algorithmic model built on a deep learning algorithm, for example, the target model may be built on any type of deep learning algorithm such as an attention network algorithm or a convolutional neural network algorithm. The target model may include one or more target network layers, and the hyperparameters of the target network layers may be understood as the model parameters of the target network layers. For example, the hyperparameters of the target network layers may be weight parameters w, bias parameters b, etc. The hyperparameters of each network layer in the target model are stored in a memory unit, any type of data to be processed, such as text, images, audio, or graph data, is input to the target model, and the operator unit processes the data to be processed using the target model by performing computational tasks on the data to be processed based on the hyperparameters of one or more target network layers in the target model received from the memory unit, thereby obtaining the model output result of the target model. The model output result may include data processing results for any type of data to be processed, such as text, images, audio, or graph data. For example, the model output result may be output text, rendered images, edited audio data, etc.

[0039] According to embodiments of this disclosure, a computation task may include a plurality of subtasks executed in a predetermined order, and the first computation subtask and the second computation subtask may be two subtasks executed sequentially in the computation task. Of the plurality of subtasks executed sequentially, the preceding subtask may be the first computation subtask, and the subsequent subtask executed adjacent to the preceding subtask may be the second computation subtask.

[0040] According to embodiments of this disclosure, the first and second features may be hidden features output from the target network layer. The first feature is obtained by executing a first computation subtask based on the hyperparameters of the target network layer using an operator unit, and the second computation subtask is executed sequentially using an operator unit under the condition that the hyperparameters of the target network layer read from the memory unit are multiplexed. This allows for the execution of multiple subtasks in a computation task by cyclically multiplexing the hyperparameters of the target network layer in the memory unit. This enables deep data processing of the data to be processed input to the target model by cyclically reading the hyperparameters and multiplexing the same hyperparameters using a computation unit, under the condition that only the hyperparameters of the target network layer are stored in the memory unit. This avoids increasing the memory space occupied by the memory unit by executing multiple different subtasks using the respective hyperparameters of multiple different network layers, thereby reducing the memory space occupied by the memory unit. This enables the simulation of a deep data processing process for a large-scale model with a large number of model parameters under the condition that the scale of hyperparameters stored in the memory unit is small. The accuracy of the output results of the target model identified based on the second feature can be close to the accuracy of the output results obtained when a large-scale model with multiple network layer structures processes the target data. This avoids loss of algorithmic performance in the target model, reduces the computational overhead and memory space occupied during the process in which the target model performs data processing, reduces the computational performance and memory space requirements of the computing device on which the model is deployed, and further improves the convenience of deployment and generalization ability of the target model on the computing device, thereby reducing the inference cost and deployment cost of the target model.

[0041] According to embodiments of the present disclosure, the first and second features may include text features, which are identified based on initial text, and the model output results include output text corresponding to the initial text.

[0042] According to embodiments of this disclosure, the initial text may be data to be processed that is input to the target model. The initial text may include any type of text used for input, such as a question text or a demand text. The output text may be feedback text output by the target model to satisfy the demand intent represented by the initial text, for example, the output text may be an answer text corresponding to a question text.

[0043] The data to be processed used in the computation task may be stored in a memory unit associated with the operator unit, and the data to be processed may be the same as or different from the memory space of the hyperparameters of the target model, and the embodiments of this disclosure are not limited thereto.

[0044] According to the method of the embodiment of this disclosure, by cyclically multiplexing the hyperparameters of the target network layer in the memory unit using an operator unit and sequentially executing multiple subtasks to process the initial text, the target model can achieve deep semantic understanding of the initial text under conditions that reduce the scale of the number of hyperparameters. Furthermore, by realizing the text semantic analysis capability and task execution control capability of the large number of model parameters of the large language model for the initial text, the accuracy of the output text of the target model can be brought closer to the accuracy of the text output by the large language model, the memory space occupied by the computing device is reduced, the deployment cost for the computing device to deploy the large language model is also reduced, and the generalization and adaptability of the target model to different application scenes is improved.

[0045] According to embodiments of this disclosure, the first or second feature may further include hidden features representing other modal data, for example, the first or second feature may be an image feature representing image modal data, an audio feature representing audio modal data, and so on. The computation task may be any modal data processing task, such as an image processing task or an audio processing task.

[0046] Alternatively, the first or second feature may further include a multimodal fusion feature representing the fusion of multimodal data, for example, the first or second feature may be an image-character fusion feature that fuses image semantics and text semantics. The embodiments of this disclosure do not limit the modal type of data represented by the first or second feature.

[0047] According to embodiments of the present disclosure, the memory unit includes an on-chip memory unit in which the hyperparameters, first feature and second feature of the target network layer are stored.

[0048] According to embodiments of this disclosure, the on-chip storage unit may be on the same processor as the operator unit. The hyperparameters, first feature, and second feature of the target network layer are stored in the on-chip storage unit, and by quickly reading the hyperparameters, first feature, or second feature of the target network layer, the efficiency of subtask execution by the operator unit can be improved, thereby improving the computational efficiency of the target model. At the same time, by storing the hyperparameters of the target network model only in the on-chip storage unit, the storage space occupied by off-chip storage units such as video memory components and memory components can be reduced, thereby improving the overall computational efficiency of the computing device deploying the target model.

[0049] In one example, the operator unit of the graphics processor GPU may be a CUDA compute core, and the on-chip memory unit may be an L1 cache component or an L2 cache component of the graphics processor GPU. The CUDA compute core can improve the computational efficiency of the graphics processor GPU and reduce the occupation of computational resources by frequently and cyclically calling hyperparameters of the same target network layer from the L1 cache component or L2 cache component to sequentially execute multiple subtasks of a computation task.

[0050] In one example, the memory unit may be an off-chip memory unit, for example, the memory unit may be the off-chip video memory unit of a graphics processor. If the target model includes multiple target network layers, the hyperparameters of the target network layers can be stored in the off-chip video memory unit, so that the operator unit of the graphics processor can read the hyperparameters of the target network layers from the off-chip video memory unit multiple times in the process of executing a subtask to execute the subtask. The first or second feature may be stored in off-chip memory so that the operator unit can sequentially execute multiple subtasks of a computation task.

[0051] In one example, the memory units may be on-chip memory units and off-chip memory units, and the hyperparameters of the target network layer may be stored in the on-chip memory unit because the hyperparameters of the target network layer need to be cyclically called by the operator unit to execute subtasks, thereby enabling the operator unit to quickly and frequently multiplex the hyperparameters of the target network layer to execute multiple subtasks of the computation task, and the network layer features output from the target network layer obtained by executing the subtasks may be stored in the off-chip memory unit so as to accommodate large-dimensional network layer features using the large memory space of the off-chip memory unit. It should be understood that these network layer features may include first and second features.

[0052] According to embodiments of the present disclosure, the target network layer includes at least one attention layer and a feedforward layer, and the first or second feature includes at least one attention feature and a feedforward feature.

[0053] According to embodiments of this disclosure, the target model may be constructed based on an attention network algorithm, and the attention layer may include an attention layer constructed based on an attention mechanism. For example, the attention layer may include a multi-head attention layer constructed based on a multi-head self-attention mechanism (MHA). The feedforward layer may be a network layer constructed based on a feed-forward neural network (FFNN). Attention features should be understood to be features output by the attention layer.

[0054] In one example, the target network layer may be an attention layer. By multiplexing the hyperparameters of the attention layer using operator units in this way, multi-layered deep attention fusion can be performed on the data to be processed. Furthermore, the final attention feature obtained by the computation task sequentially executing multiple subtasks can be sufficiently trained on semantic attribute features such as image semantic features and text semantic features of the data to be processed, thereby improving the output accuracy of the model output and improving the degree of matching between the model output and the demand intent of the target object.

[0055] Figure 3 schematically shows a schematic diagram of the target model according to an embodiment of this disclosure.

[0056] As shown in Figure 3, the target model 310 may include a first target network layer 311 and a second target network layer 312. The first target network layer 311 and the second target network layer 312 may be a multi-head attention layer and a feedforward layer, respectively. The initial text features of the demand text are stored in a memory unit, and the initial text features of the demand text are processed using an operator unit based on the hyperparameters of the first target network layer 311, thereby executing the first calculation subtask in the first calculation task and obtaining the first text features output from the first target network layer 311. If the predetermined number of sequentially executed subtasks included in the first calculation task is i=2, the hyperparameters of the first target network layer 311 in the memory unit are multiplexed, and the first text features and the hyperparameters of the first target network layer 311 are read from the memory unit using an operator unit, the second calculation subtask is executed, and the second text features output from the first target network layer 311 are obtained.

[0057] For comparison, the trained base large model 320 may include a first base network layer 321, a second base network layer 322, a third base network layer 323, and a fourth base network layer 324. When the operator unit multiplexes the hyperparameters of the first target network layer 311 to sequentially execute two subtasks in the first computation task, it simulates the feature data deep processing process on the initial text features of the demand text by the first base network layer 321 and the second base network layer 322 in the base large model 320, avoiding the computational performance degradation caused by directly deleting the base network layers of the base large model 320 to identify the target model, reducing the memory space occupied by the hyperparameters stored in the memory unit, and reducing the occupation of computational resources on the computing device.

[0058] As shown in Figure 3, the second text features output from the first target network layer 311 are stored in a memory unit, and the second text features are processed using an operator unit based on the hyperparameters of the second target network layer 312, thereby executing the first computation subtask in the second computation task and obtaining the first text features output from the second target network layer 312. When the predetermined number of sequentially executed subtasks included in the second computation task is i=2, the hyperparameters of the second target network layer 312 are multiplexed in the memory unit, the first text features output from the second target network layer 312 and the hyperparameters of the second target network layer 312 are read from the memory unit using an operator unit, the second computation subtask in the second computation task is executed, the second text features output from the second target network layer 312 are obtained, and the target feedback image can be obtained by processing the second text features output from the second target network layer 312 based on the activation function using an operator unit. The target feedback image may be an image that satisfies the demand intention represented by the demand text.

[0059] When the operator unit multiplexes the hyperparameters of the second target network layer 312 in the memory unit to sequentially execute two subtasks in the second computation task, it simulates the third base network layer 323 and the fourth base network layer 324 in the base large-scale model 320. This avoids the drawback of reduced computational performance caused by directly deleting the base network layers of the base large-scale model 320 to identify the target model, enables the generation of a target feedback image with image quality close to that of the base feedback image, and further reduces the memory space occupied by the target model's hyperparameters in the memory unit, improving the deployment efficiency and inference efficiency of the target model in the computing device.

[0060] According to embodiments of this disclosure, a computation task includes a predetermined number of subtasks that are executed sequentially. The operator unit can perform a predetermined number of subtasks using the hyperparameters of the target network layer a predetermined number of times, thereby multiplexing the hyperparameters of the target network layer a predetermined number of times to realize a data deep processing process for a large-scale model and improve the accuracy of the model's output results.

[0061] In one example, the predetermined number is determined based on the placement operation of the target object, and the placement operation allows the target object to set a predetermined number of times in which the hyperparameters of at least one target network layer in the target model are multiplexed. This enables the operator unit to dynamically set the deep processing requirements for the data to be processed according to the demands of the target object, thereby obtaining model output results that satisfy the actual demand intentions of the target object, allowing the target model to adapt to the various output accuracy and output delay requirements of the target object, while also flexibly setting the number of times the memory unit can multiplex the hyperparameters of the same target network layer, improving the utilization efficiency of the memory space of the memory unit, avoiding the occupation of communication resources by transmitting hyperparameters of different network layers to the computing unit multiple times, and reducing the occupation of memory space by writing hyperparameters of different network layers to the memory unit multiple times, thereby improving the flexibility and generalization ability of the target model deployment and inference process.

[0062] According to embodiments of this disclosure, a predetermined number is determined based on demand information, and the demand information includes at least one of delayed demand information and accuracy demand information.

[0063] According to embodiments of this disclosure, the delay demand information represents the output delay demand for the target model to output model output results. The output delay demand can represent the delay time for the target object to process the data to be processed and output model output results for the target model, and the delay time may be directly proportional to a predetermined number. If the delay time corresponding to the output delay demand is long, setting a large predetermined number allows deep data processing to be performed on the data to be processed based on the large predetermined number, thereby improving the accuracy of the model output results of the target model. If the delay time corresponding to the output delay demand is short, setting a small predetermined number controls the number of executions of subtasks in the computation task, allowing the target model to perform a relatively rapid data processing process on the data to be processed, improving the rate at which the target model obtains model output results, reducing delays in the inference process of the target model, and meeting the actual demand for low-latency scenes.

[0064] According to embodiments of this disclosure, the accuracy demand information represents the output accuracy demand of the model output results. The output accuracy demand can represent the quality of the model output results of the target model. The output accuracy demand can represent the degree of matching between the model output results of the target model and the actual demand target of the target object.

[0065] For example, if the data processed by the target model is text, the output accuracy requirement can represent text accuracy attributes such as the accuracy, relevance, fluency, and consistency of the output text of the target model. Accuracy can indicate whether the output text contains correct information and whether it meets factual, logical, and semantic requirements. Relevance is used to evaluate whether the output text is closely related to the input prompt or context. Fluency and consistency are used to indicate whether the output text is natural and logically consistent.

[0066] Furthermore, for example, if the data to be processed by the target model is an image, the output accuracy requirement can represent image accuracy attributes such as class accuracy, detection accuracy, or generation quality of the output image of the target model. Class accuracy indicates the percentage of times the target model correctly recognizes the class to which the image belongs. Detection accuracy indicates the accuracy of the recognition results, such as the accurate recognition of the object position, type, size, etc., in the image by the target model. Generation quality indicates the visual quality of the output image of the target model, the degree of matching with the target's requirements and intentions, etc.

[0067] The model output results produced by the target model and the data processed by the target model may be any modal data, and the embodiments of this disclosure are not limited thereto.

[0068] According to embodiments of this disclosure, delay demand information and accuracy demand information may be identified based on a target placement operation. For example, by performing a placement operation on an interaction interface based on a target and setting the delay demand information and accuracy demand information, the inference process of the target model can be flexibly controlled, the inference accuracy and inference delay of the target model can be flexibly controlled, and the actual demands for various artificial intelligence application scenarios can be realized.

[0069] According to embodiments of the present disclosure, the target model is identified by the following operations: using an operator unit to identify a similarity index for at least one designated base network layer in a pre-trained base large model; identifying N base network layers from the pre-trained base large model that satisfy the similarity criteria based on the similarity index; and identifying the target model based on M base network layers that satisfy the similarity criteria.

[0070] According to embodiments of this disclosure, the trained base large model may be a generative large model that performs a specific function. The trained base large model may include multiple base network layers. Other base network layers may be base network layers other than the specified base network layer in the trained base large model. The similarity index represents the hyperparameter similarity between the specified base network layer and the other base network layer in the trained base large model. The similarity index between the specified base network layer and the other base network layer can be determined by calculating the similarity between the hyperparameters of the specified base network layer and the hyperparameters of the other base network layer.

[0071] According to the embodiments of this disclosure, N > M ≥ 1, and N and M are integers. For example, a target model can be constructed by identifying one base network layer as the target network layer from 10 base network layers that satisfy the similarity condition.

[0072] For example, identifying a target model based on M base network layers that satisfy the similarity criteria may include obtaining the target model by removing NM base network layers from N base network layers that satisfy the similarity criteria in the base large-scale model, thereby retaining M base network layers that satisfy the similarity criteria.

[0073] According to the embodiments of this disclosure, the similarity condition can indicate that the hyperparameter similarity between N base network layers is greater than or equal to a predetermined similarity threshold, and that N base network layers satisfying the similarity condition have similar or similar task execution capabilities, for example, that they have similar learning capabilities for feature information such as text features and image features. Therefore, by identifying M base network layers that satisfy the similarity condition as target network layers from among the N base network layers that satisfy the similarity condition and constructing a target model, it is possible to realize specific functions such as text response and image generation of the trained large-scale base model under the condition of reducing the number of hyperparameters in the target model. Simultaneously, by setting a predetermined number of subtasks in the computation task, the hyperparameters of the target network layer are multiplexed to execute a predetermined number of subtasks. This allows for the execution of multiple subtasks by multiplexing the hyperparameters of the same target network layer using an operator unit, thereby bringing the accuracy between the model output of the target model and the base output of the trained base large-scale model relatively close. This reduces the deployment and inference costs of the target model, improves the inference efficiency and accuracy when the target model is deployed to computing devices, and meets the actual intelligent model output demands of diverse scenarios.

[0074] Figure 4 schematically shows a principle diagram for identifying a target model according to the embodiments of this disclosure.

[0075] As shown in Figure 4, the trained base large model 410 includes a first base network layer 411, a second base network layer 412, a third base network layer 413, and a fourth base network layer 414. A similarity index for the second base network layer 412 is obtained by calculating the similarity between the hyperparameters of the second base network layer 412 and the hyperparameters of the other base network layers of the trained base large model 410. The similarity index for the second base network layer 412 may indicate that the first base network layer 411 and the second base network layer 412 satisfy the similarity condition. A similarity index for the third base network layer 413 is obtained by calculating the similarity between the hyperparameters of the third base network layer 413 and the hyperparameters of the other base network layers of the trained base large model 410. The similarity index for the third base network layer 413 may indicate that the third base network layer 413 and the fourth base network layer 414 satisfy the similarity condition. The target model 420 can be obtained by removing the second base network layer 412 and the third base network layer 413 from the trained base large-scale model 410. The deep data processing process of the first base network layer 411 and the second base network layer 412 of the trained base large-scale model 410 can be simulated by using an operator unit to multiplex the hyperparameters of the first base network layer 411 and executing two subtasks sequentially in the first computation task. Then, the deep data processing process of the third base network layer 413 and the fourth base network layer 414 of the trained base large-scale model 410 can be simulated by using an operator unit to multiplex the hyperparameters of the fourth base network layer 414 and executing two subtasks sequentially in the second computation task.In this way, the generation accuracy between the model output results and the base output results obtained by processing the target data by both the target model 420 and the trained base large-scale model 410 can be made close, ensuring that the target model 420 has a generation effect similar to the trained base large-scale model 410 with a large number of parameters, reducing the amount of computational resources occupied, and lowering the inference cost and deployment difficulty of the target model 420. At the same time, by setting a predetermined number of subtasks to be executed sequentially in each computation task, the target model 420 can perform deeper data processing on the target data than the trained base large-scale model 410, flexibly adapting to the accuracy and latency requirements of the target object for the model output results, and improving the deployment flexibility and generalization ability of the target model 420.

[0076] Figure 5 schematically shows a flowchart of the model training method according to the embodiment of this disclosure.

[0077] As shown in Figure 5, the training method for this model includes operations S510 to S560.

[0078] In operation S510, the hyperparameters of the initial network layer in the initial model are read from the memory unit.

[0079] In operation S520, the operator unit is used to execute the first computation subtask in the computation task based on the hyperparameters of the initial network layer, and the first initial features output from the initial network layer are obtained.

[0080] In operation S530, when multiplexing the hyperparameters of the initial network layer, the operator unit is used to execute the second computation subtask in the computation task based on the first initial features read from the memory unit, and the second initial features output from the initial network layer are obtained.

[0081] In operation S540, the operator unit is used to identify the model output result of the initial model based on the second initial feature read from the memory unit.

[0082] In operation S550, the target loss information is identified using the operator unit based on the model output results of the initial model.

[0083] In operation S560, the initial model is trained using the operator unit based on the target loss information, and the trained target model is obtained.

[0084] According to embodiments of this disclosure, the first and second computation subtasks are subtasks executed sequentially in a computation task, and the operator unit can sequentially execute multiple subtasks under the condition of multiplexing the hyperparameters of the initial network layer, thereby reducing the amount of hyperparameters occupied in the memory space of the memory unit.

[0085] According to embodiments of the present disclosure, the storage unit may be a device or component for storing data, for example, the storage unit may be a component for caching any type of data, such as an on-chip cache or an off-chip cache, and embodiments of the present disclosure do not limit the specific type of storage unit.

[0086] According to embodiments of this disclosure, the initial model may be an algorithmic model built on a deep learning algorithm, for example, an attention network algorithm, a convolutional neural network algorithm, or any other type of deep learning algorithm. The initial model may include one or more initial network layers, and the hyperparameters of the initial network layers may be understood as model parameters of the initial network layers. For example, the hyperparameters of the initial network layers may be a weight parameter w, a bias parameter b, etc. Any type of data to be processed, such as text, images, audio, or graph data, can be input to the initial model, and the operator unit can enable the initial model to process the data and obtain a model output result by performing computational tasks on the data to be processed based on the hyperparameters of one or more initial network layers in the initial model. The model output result may include data processing results for any type of data to be processed, such as text, images, audio, or graph data. For example, the model output result may be output text, a rendered image, edited audio data, etc.

[0087] The technical terms relating to the model training method provided in the embodiments of this disclosure include, but are not limited to, the first initial feature, second initial feature, operator unit, and memory unit. Similarly, the technical terms relating to the task execution method provided in the embodiments of this disclosure include, but are not limited to, those having the same attributes as the first feature, second feature, operator unit, and memory unit. In the embodiments of this disclosure, further explanation is omitted here.

[0088] According to embodiments of this disclosure, the first and second initial features may be hidden features output from the initial network layer. The first initial features can be obtained by executing a first computation subtask based on the hyperparameters of the initial network layer using an operator unit, and the second computation subtask, which is executed sequentially in the computation task, can be executed sequentially using an operator unit under the condition that the hyperparameters of the initial network layer read from the memory unit are multiplexed. This allows for the execution of multiple subtasks in the computation task by cyclically multiplexing the hyperparameters of the initial network layer. As a result, under the condition that only the hyperparameters of the initial network layer are stored in the memory unit, deep data processing can be performed on the sample data to be processed input to the initial model by cyclically reading the said hyperparameters, thereby reducing the amount of memory space occupied by the memory unit. The memory unit simulates the data deep processing process of a large-scale model with a large number of model parameters under the condition that the size of the stored hyperparameters is relatively small. This brings the accuracy of the output results of the initial model, identified based on the second initial features, closer to the accuracy with which the large-scale model processes the sample data being processed. This avoids algorithmic performance loss in the initial model and reduces the computational overhead and memory space occupied by the initial model when it performs the data processing process, while reducing the size of the initial model's hyperparameters. By training the initial model using the operator unit based on target loss information identified based on the model output results of the initial model, the memory space occupied by the memory unit during model training can be reduced. This makes it easier for the operator unit to quickly recall the initial model's hyperparameters to perform subtasks, further reducing the training cost for the computing device to train the initial model and reducing the occupation of computational resources during model training.

[0089] A target model identified by the model training method according to the embodiments of this disclosure can be applied to the task execution method according to the embodiments of this disclosure. For example, by storing the hyperparameters of the target model identified based on the model training method provided according to the embodiments of this disclosure in a storage unit, and by using an operator unit to execute a computation task based on the hyperparameters of the target network layer of the target model according to the task execution method provided according to the embodiments of this disclosure, it is possible to process arbitrary modal data to be processed and obtain model output results.

[0090] According to embodiments of the present disclosure, the initial model is identified by an operation that uses an operator unit to identify a similarity index for at least one designated base network layer in the base large model, identifies K base network layers from the base large model that satisfy the similarity criteria based on the similarity index, and identifies the initial model based on the P base network layers that satisfy the similarity criteria.

[0091] According to embodiments of this disclosure, the base large-scale model may include a deep learning model having any type of large-scale hyperparameters, for example, the base large-scale model may include a trained large-scale language model. Alternatively, the base large-scale model may be a large-scale model for processing any modal data such as audio data, image data, or graph data. The base large-scale model may include multiple base network layers, and the similarity metric represents the hyperparameter similarity between a specified base network layer and other base network layers in the base large-scale model. The other network layers should be understood to include other base network layers in the base large-scale model other than the specified base network layer. The similarity metric between the specified base network layer and the other base network layer can be identified by calculating the similarity between the hyperparameters of the specified base network layer and the hyperparameters of the other base network layer.

[0092] According to the embodiments of this disclosure, K > P ≥ 1, and K and P are integers. For example, an initial model can be constructed by identifying one base network layer as the initial network layer from 10 base network layers that satisfy the similarity condition.

[0093] For example, identifying an initial model based on P base network layers that satisfy the similarity condition may include obtaining the initial model by removing KP base network layers from the K base network layers that satisfy the similarity condition in the base large-scale model, thereby retaining P base network layers that satisfy the similarity condition.

[0094] According to the embodiments of this disclosure, the similarity condition can indicate that the hyperparameter similarity among K base network layers is greater than or equal to a predetermined similarity threshold, and the K base network layers that satisfy the similarity condition can indicate that they have similar or similar task execution capabilities, for example, that they have similar learning capabilities for feature information such as text features and image features. Therefore, by identifying P base network layers that satisfy the similarity condition from among the K base network layers that satisfy the similarity condition as initial network layers and constructing an initial model, it is possible to realize specific functions such as text response and image generation of the base large-scale model under the condition of reducing the scale of the number of hyperparameters in the initial model. At the same time, by setting a predetermined number of subtasks in the computation task, it is possible to execute a predetermined number of subtasks by multiplexing the hyperparameters of the initial network layer, thereby achieving relatively close accuracy between the model output result of the initial model and the base output result of the trained base large-scale model by multiplexing the hyperparameters of the same initial network layer using an operator unit and executing multiple subtasks. Thus, the initial model is constructed by identifying a small number of P base network layers from among the K base network layers that satisfy the similarity condition. By reducing the memory space required to store the initial model's hyperparameters in the memory unit, and by multiplexing the initial network layer's hyperparameters based on the operator unit to sequentially execute multiple subtasks, the deep data processing performance of the base large-scale model can be achieved. Furthermore, the target model, trained based on target loss information, can achieve data processing performance and model output result generation accuracy close to that of a large-scale model, even with a small number of hyperparameters, improving training efficiency and meeting the actual output demands of intelligent models in diverse scenarios.

[0095] According to embodiments of the present disclosure, identifying a similarity index for at least one specified base network layer in a base scalar model using an operator unit includes: removing at least one specified base network layer from the base scalar model using an operator unit to obtain a processed base scalar model; performing a base computation task based on the hyperparameters of the updated base scalar model using an operator unit based on predetermined training data to obtain a base output result of the processed base scalar model; processing the base output result of the processed base scalar model and labels corresponding to the training data using an operator unit to obtain the base output precision of the processed base scalar model; and identifying a similarity index for a specified base network layer using an operator unit based on the base output precision of the processed base scalar model.

[0096] According to embodiments of this disclosure, the predetermined training data may be sample data to be processed by any modal, for example, the training data may include sample text, sample images, sample audio, etc. An operator unit can be used to process the training data based on the hyperparameters of a processed base large model and obtain the base output result output from the base large model. The base output result may be a feedback result corresponding to the demand intent represented by the training data.

[0097] For example, if the training data is the sample text to be processed, the base output of the processed base large model may be the response text corresponding to the sample text to be processed.

[0098] For example, if the training data is the sample text to be processed, the base output of the processed base large model may be a generated image corresponding to the demand intent of the sample text to be processed.

[0099] For example, processing the base output results and labels corresponding to the training data of a processed base large-scale model using an operator unit may include processing the base output results and labels of the processed base large-scale model based on the loss function and obtaining base loss information. The base output accuracy is determined based on the difference information represented by the base loss information.

[0100] According to embodiments of the present disclosure, identifying a similarity metric for a specified base network layer using an operator unit based on the base output accuracy of a processed base large model may include querying relevant similarity metric information based on the base output accuracy.

[0101] For example, the labels may be the base output results obtained when an unprocessed base scalar model processes the training data. The similarity metric may be understood as the precision similarity of the generated results between the base output results produced by the processed base scalar model of the deleted specified base network layer and the base output results of the original base scalar model. A high precision similarity indicates a large hyperparameter similarity represented by the similarity metric, and if the deleted specified base network layer is missing, it can be understood that the impact on the precision of the base output results of the base scalar model is small. Conversely, a low precision similarity indicates a small hyperparameter similarity represented by the similarity metric, and if the deleted specified base network layer is missing, it can be understood that the impact on the precision of the base output results of the base scalar model is large. The base output accuracy is directly proportional to the hyperparameter similarity represented by the similarity index. Thus, by identifying the similarity index of a specified base network layer, K base network layers that satisfy the similarity condition can be identified, and from these K base network layers, P base network layers that satisfy the similarity condition in fewer numbers can be identified to construct the initial model. As a result, the initial model can produce an output result that is close in accuracy to the base output result of the base large-scale model, under the condition that the operator unit multiplexes the hyperparameters of the initial network layer. This reduces the complexity of constructing the initial model, reduces the difficulty of constructing the knowledge transfer from the base large-scale model to the initial model, and improves the training efficiency of the initial model.

[0102] According to embodiments of this disclosure, identifying target loss information using an operator unit based on the model output results of an initial model may include processing the model output results of an initial model and the base output results of a base large-scale model using an operator unit based on a knowledge distillation mechanism to obtain output loss information; processing at least one initial feature and the base task calculation results using an operator unit to obtain intermediate loss information; and fusing the output loss information and intermediate loss information using an operator unit to obtain target loss information.

[0103] According to embodiments of this disclosure, the base task calculation results are identified by performing a base calculation task using an operator unit based on the hyperparameters of the base network layer in the base large-scale model, and the base output results are identified based on the base task calculation results. The base task calculation results may be hidden features output by the base network layer, and the base output results may be generated results output by the base large-scale model.

[0104] For example, the base output and the initial model output may be obtained by processing the same batch of training data for both the base large-scale model and the initial model, respectively.

[0105] According to embodiments of this disclosure, processing the model output results of the initial model and the base output results of the base large-scale model using an operator unit based on a knowledge distillation mechanism may include processing the base output results and the model output results of the initial model based on a loss function using an operator unit, with the base output results as pseudo-labels, and the obtained output loss information can represent the difference in generation accuracy between the base output results and the model output results of the initial model.

[0106] According to embodiments of this disclosure, the initial feature includes at least one of a first initial feature and a second initial feature. The initial feature can represent a hidden feature of the initial model obtained by an operator unit completing a subtask based on the hyperparameters of the initial network layer. Processing at least one initial feature and the base task calculation result using the operator unit may include using the base calculation result output from the base network layer as a pseudo-label and processing the base calculation result and the corresponding initial feature based on a loss function using the operator unit, thereby representing the feature difference between the hidden feature of the initial network layer obtained by the initial network layer executing each subtask and the hidden feature output from the base network layer corresponding to that subtask in the base large-scale model. This makes it possible to show the difference between the initial feature obtained by the operator unit executing a subtask based on the hyperparameters of the initial network layer and the base calculation result obtained by the operator unit executing a base calculation task based on the hyperparameters of the base network layer, based on the intermediate loss information. In this way, by fusing intermediate loss information and output loss information to obtain target loss information, and adjusting the initial model's hyperparameters during the training process, the operator unit can simulate the data processing and semantic analysis capabilities of each base network layer by sequentially executing each subtask using the hyperparameters of the trained target network layer. Using the target model, it is possible to generate data processing capabilities and model output result accuracy close to the algorithmic performance of a large-scale model with a larger parameter scale, based on the small hyperparameter scale stored in the memory unit, thereby improving the inference efficiency of the target model during the inference phase and reducing the deployment cost of the target model.

[0107] Figure 6 schematically shows the principle of the model training method according to the embodiment of this disclosure.

[0108] As shown in Figure 6, the initial model 610 includes a first initial network layer 611 and a second initial network layer 612, and the base large-scale model 620 includes a first base network layer 621, a second base network layer 622, a third base network layer 623, and a fourth base network layer 624. The base large-scale model 620 is used as the training model, and the initial model 610 is used as the student model. Here, the operator unit processes the training data based on the hyperparameters of the first base network layer 621 and obtains the first base task calculation result output from the first base network layer 621. The operator unit processes the first base task calculation result based on the hyperparameters of the second base network layer 622 and obtains the second base task calculation result output from the second base network layer 622. The operator unit processes the second base task calculation result based on the hyperparameters of the third base network layer 623 and obtains the third base task calculation result output from the third base network layer 623. The operator unit processes the results of the third base task calculation based on the hyperparameters of the fourth base network layer 624, and obtains the base output results output from the fourth base network layer 624.

[0109] Using the operator unit, the first computation subtask in the first computation task is executed based on the hyperparameters and training data of the first initial network layer 611, and the first initial features output from the first initial network layer 611 are obtained. The hyperparameters of the first initial network layer 611 are multiplexed, and using the operator unit, the second computation subtask in the first computation task is executed based on the hyperparameters and the first initial features output from the first initial network layer 611, and the second initial features output from the first initial network layer 611 are obtained. Using the operator unit, the first computation subtask in the second computation task is executed based on the hyperparameters and the second initial features output from the first initial network layer 611, and the first initial features output from the second initial network layer 612 are obtained. The hyperparameters of the second initial network layer 612 are multiplexed, and the operator unit is used to execute the second computation subtask in the second computation task based on the hyperparameters of the second initial network layer 612 and the first initial features output from the second initial network layer 612, thereby obtaining the model output result output from the first initial network layer 611.

[0110] Using the operator unit, the first initial features output from the first initial network layer 611 and the first base task calculation results output from the first base network layer 621 are processed based on the loss function to obtain first intermediate loss information. Using the operator unit, the second initial features output from the first initial network layer 611 and the second base task calculation results output from the second base network layer 622 are processed based on the loss function to obtain second intermediate loss information. Using the operator unit, the first initial features output from the second initial network layer 612 and the third base task calculation results output from the third base network layer 623 are processed based on the loss function to obtain third intermediate loss information. Using the operator unit, the model output results output by the second initial network layer 612 and the base output results output by the fourth base network layer 624 are processed based on the loss function to obtain output loss information.

[0111] The target loss information is obtained by weighting and averaging the first intermediate loss information, second intermediate loss information, third intermediate loss information, and output loss information using an operator unit. The trained target model is obtained under the condition that the hyperparameters of the initial model 610 are adjusted based on the target loss information until the target loss information converges or a predetermined number of adjustments are reached.

[0112] According to the embodiments of this disclosure, by fusing output loss information and intermediate loss information based on a knowledge distillation mechanism, the target loss can represent the difference in algorithmic performance between each network layer of the initial model and the base network layer. This enables the transfer of knowledge from the base large-scale model to the target model with fewer parameters by adjusting the parameters of the initial model based on the target loss information. This helps the target model to learn quickly, avoids algorithmic performance loss due to the removal of hyperparameters of some network layers in the base large-scale model, and improves the training effect.

[0113] In one example, the loss function may be the Mean Squared Error (MSE) function.

[0114] For example, the predetermined number of subtasks that each computational task in a model training method sequentially performs may be determined based on placement operations so that the target model obtained through training can be adapted to the model performance corresponding to the application scene of the target object.

[0115] In one example, the predetermined number of subtasks that each computational task in the model training method sequentially executes may be determined based on a random policy. For example, in each round of training operations, the predetermined number of subtasks that each computational task sequentially executes is a random number determined based on the random policy. In this way, the initial network layer can be randomly adapted to the degree of hyperparameter multiplexing for the initial network layer in each round of training operations, and the trained target model can produce highly accurate model output results in application scenarios with different output delay demands, thereby improving the generalization ability of the target model.

[0116] Figure 7 schematically shows a block diagram of a task execution device according to an embodiment of the present disclosure.

[0117] As shown in Figure 7, the task execution device 700 includes a storage unit 710 and an operator unit 720.

[0118] The operator unit 720 reads the hyperparameters of the target network layer in the target model from the memory unit, executes a first computation subtask in the computation task based on the hyperparameters of the target network layer, obtains a first feature output from the target network layer, and, if the hyperparameters of the target network layer are multiplexed, executes a second computation subtask in the computation task based on the first feature read from the memory unit, obtains a second feature output from the target network layer. The first and second computation subtasks are subtasks executed sequentially in the computation task and are configured to identify the model output result of the target model based on the second feature read from the memory unit.

[0119] According to embodiments of the present disclosure, the memory unit includes an on-chip memory unit in which the hyperparameters, first feature and second feature of the target network layer are stored.

[0120] According to embodiments of the present disclosure, a computation task includes a predetermined number of subtasks that are executed sequentially, the predetermined number of which is determined based on demand information, the demand information including at least one of delay demand information representing the output delay demand for the target model to output model output results, and accuracy demand information representing the output accuracy demand for the model output results.

[0121] According to embodiments of the present disclosure, the operator unit is further configured to identify a target model by performing the following operations: using the operator unit to identify a similarity index of at least one designated base network layer in a trained base large model, where the similarity index represents the hyperparameter similarity between the designated base network layer and other base network layers in the trained base large model; identifying N base network layers from the trained base large model that satisfy a similarity condition based on the similarity index; and identifying a target model based on M base network layers that satisfy the similarity condition, where N > M ≥ 1 and N and M are integers.

[0122] According to embodiments of the present disclosure, the target network layer includes at least one attention layer and a feedforward layer, and the first or second feature includes at least one attention feature and a feedforward feature.

[0123] According to embodiments of this disclosure, the first and second features include text features, the text features are identified based on initial text, and the model output results include output text corresponding to the initial text.

[0124] Figure 8 schematically shows a block diagram of a model training device according to an embodiment of the present disclosure.

[0125] As shown in Figure 8, the model training device 800 includes a memory unit 810 and an operator unit 820.

[0126] The operator unit 820 reads the hyperparameters of the initial network layer in the initial model from the memory unit, executes a first computation subtask in the computation task based on the hyperparameters of the initial network layer, and obtains the first initial features output from the initial network layer. If the hyperparameters of the initial network layer are multiplexed, it executes a second computation subtask in the computation task based on the first initial features read from the memory unit, and obtains the second initial features output from the initial network layer. Here, the first and second computation subtasks are subtasks executed sequentially in the computation task, and are configured to identify the model output result of the initial model based on the second initial features read from the memory unit, identify the target loss information based on the model output result of the initial model, train the initial model based on the target loss information, and obtain the trained target model.

[0127] According to embodiments of the present disclosure, the operator unit is further configured to identify an initial model by performing the following operations: using the operator unit to identify a similarity index of at least one designated base network layer in a base large model, where the similarity index represents the hyperparameter similarity between the designated base network layer and other base network layers in the base large model; identifying K base network layers from the base large model that satisfy a similarity condition based on the similarity index; and identifying an initial model based on the P base network layers that satisfy the similarity condition, where K > P ≥ 1 and K and P are integers.

[0128] According to embodiments of the present disclosure, the operator unit is further configured to identify a similarity index of at least one specified base network layer in the base scalar model by performing the following operations: removing at least one specified base network layer in the base scalar model to obtain a processed base scalar model; performing a base computation task based on the updated hyperparameters of the base scalar model based on predetermined training data to obtain a base output result of the processed base scalar model; processing the base output result of the processed base scalar model and labels corresponding to the training data to obtain the base output precision of the processed base scalar model; and identifying a similarity index of the specified base network layer based on the base output precision of the processed base scalar model.

[0129] According to embodiments of the present disclosure, the operator unit is further configured to identify target loss information based on the model output results of the initial model, by processing the model output results of the initial model and the base output results of the base large-scale model based on a knowledge distillation mechanism to obtain output loss information, and by obtaining intermediate loss information based on at least one initial feature and the base task calculation result, wherein the base task calculation result is identified by performing a base calculation task using the operator unit based on the hyperparameters of the base network layer in the base large-scale model, the base output result is identified based on the base task calculation result, and the initial feature includes at least one of a first initial feature and a second initial feature, and by fusing the output loss information and intermediate loss information to obtain target loss information.

[0130] According to embodiments of the present disclosure, the present disclosure further provides electronic devices, readable storage media, and computer programs.

[0131] According to embodiments of the present disclosure, an electronic device includes at least one processor and a memory communicably connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can perform a task execution method or a model training method.

[0132] According to embodiments of the present disclosure, this is a non-temporary computer-readable storage medium that stores computer instructions for causing a computer to perform a task execution method or a model training method.

[0133] According to embodiments of this disclosure, the computer program, when executed by a processor, implements a method for performing a task or training a model.

[0134] Figure 9 shows a schematic block diagram of an exemplary electronic device 900 for carrying out the task execution method and model training method of an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may further represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are illustrative and do not limit the implementation of the present disclosure as described herein and / or requested.

[0135] As shown in Figure 9, the device 900 includes a computing unit 901, which may perform various appropriate operations and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. The RAM 903 may also store various programs and data necessary for the operation of the device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0136] Multiple components in the device 900 are connected to the I / O interface 905 and include, for example, an input unit 906 such as a keyboard or mouse; an output unit 907 such as various types of displays or speakers; a storage unit 908 such as a magnetic disk or optical disk; and a communication unit 909 such as a network card, modem, or wireless communication transceiver. The communication unit 909 enables the device 900 to exchange information and data with other devices via computer networks such as the Internet and / or various electrical networks.

[0137] The computing unit 901 may be various general-purpose and / or dedicated processing modules having processing and computational capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units for running various machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 901 performs each of the methods and processes described in the preceding paragraph, such as the task execution method and the model training method. For example, in some embodiments, the task execution method and the model training method may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a memory unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed into the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the task execution method and the model training method described in the preceding paragraph may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform task execution methods and model training methods by any other suitable method (e.g., via firmware).

[0138] Various embodiments of the systems and technologies described herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented in one or more computer programs which can be executed and / or interpreted on a programmable system which includes at least one programmable processor, which may be a dedicated or general-purpose programmable processor which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0139] Program code for carrying out the methods of this disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions and operations defined in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on a device, partially on a device, partially on a device as a standalone software package, partially on a remote device, or entirely on a remote device or server.

[0140] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores programs used in or in combination with an instruction execution system, device, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wired electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0141] To provide interaction with a user, a computer may be made to implement the systems and techniques described herein, the computer comprising a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor), a keyboard and a pointing device (e.g., a mouse or trackball), the user may provide input to the computer via the keyboard and the pointing device. Other types of devices may further provide interaction with the user, for example, feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback), and input from the user may be received in any form (including voice input, speech input, or haptic input).

[0142] The systems and technologies described herein can be implemented in computing systems including background components (e.g., a data server), computing systems including middleware components (e.g., an application server), computing systems including front-end components (e.g., a user computer having a graphical user interface or a web browser, through which the user can interact with embodiments of the systems and technologies described herein), or in computing systems including any combination of such background components, middleware components, or front-end components. Components of the system can be connected to one another by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include, but are not limited to, local area networks (LANs), wide area networks (WANs), and the Internet.

[0143] A computer system may include clients and servers. Clients and servers are generally geographically separated and typically communicate via a communication network. The client-server relationship is generated by a computer program running on the relevant computer that has a client-server relationship. The server may be a cloud server, a server in a distributed system, or a server combined with blockchain technology.

[0144] It should be understood that various forms of flows shown above may be used, and operations may be re-sorted, added, or deleted. For example, each operation described herein may be performed in parallel, sequentially, or in a different order, as long as the desired results of the proposed techniques disclosed herein can be achieved.

[0145] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. Reading the hyperparameters of the target network layer in the target model from the memory unit, Using an operator unit, the first computation subtask in the computation task is executed based on the hyperparameters of the target network layer, and the first feature output from the target network layer is obtained. When multiplexing the hyperparameters of the target network layer, the operator unit is used to execute a second computation subtask in the computation task based on the first feature read from the storage unit, and the second feature output from the target network layer is obtained, wherein the first computation subtask and the second computation subtask are subtasks that are executed sequentially in the computation task. This includes identifying the model output result of the target model using the operator unit based on the second feature read from the memory unit. Task execution method.

2. The memory unit includes an on-chip memory unit, and the hyperparameters of the target network layer, the first feature, and the second feature are stored in the on-chip memory unit. The method according to claim 1.

3. The calculation task includes a predetermined number of subtasks that are executed sequentially, the predetermined number of which is determined based on demand information, and the demand information is The target model outputs the model output result, and the delay demand information represents the output delay demand, The model output results include, at least one of the following: accuracy demand information representing the output accuracy demand of the model output results, The method according to claim 1.

4. The target model is identified based on the following operations: Using the operator unit, a similarity index is identified for at least one specified base network layer in the trained base large-scale model, the similarity index representing the hyperparameter similarity between the specified base network layer and other base network layers in the trained base large-scale model. Based on the similarity index, N base network layers that satisfy the similarity condition are identified from the trained base large-scale model. The target model is identified based on M base network layers that satisfy the similarity condition, where N > M ≥ 1 and N and M are integers. The method according to claim 1.

5. The target network layer includes at least one of an attention layer and a feedforward layer, and the first feature or the second feature includes at least one of an attention feature and a feedforward feature. The method according to claim 1.

6. The first and second features include text features, the text features are identified based on initial text, and the model output results include output text corresponding to the initial text. The method according to claim 1.

7. Reading the hyperparameters of the initial network layer in the initial model from the memory unit, Using the operator unit, the first computation subtask in the computation task is executed based on the hyperparameters of the initial network layer, and the first initial features output from the initial network layer are obtained. When multiplexing the hyperparameters of the initial network layer, the operator unit is used to execute a second computation subtask in the computation task based on the first initial features read from the storage unit, and the second initial features output from the initial network layer are obtained, wherein the first computation subtask and the second computation subtask are subtasks executed sequentially in the computation task. Based on the second initial features read from the memory unit, the operator unit is used to identify the model output result of the initial model, Based on the model output results of the initial model, the target loss information is identified using the operator unit. This includes training the initial model using the operator unit based on the target loss information and obtaining the trained target model. Model training methods.

8. The aforementioned initial model was identified based on the following operations: The operator unit is used to identify a similarity index for at least one designated base network layer in the base large-scale model, and the similarity index represents the hyperparameter similarity between the designated base network layer and other base network layers in the base large-scale model. Identifying K base network layers that satisfy the similarity conditions from the base large-scale model based on a similarity index, The initial model is identified based on P base network layers that satisfy the similarity condition, where K > P ≥ 1, and K and P are integers. The method according to claim 7.

9. Identifying a similarity index for at least one designated base network layer in the base large-scale model using the operator unit is: Using the operator unit, remove at least one of the specified base network layers in the base large-scale model and obtain the processed base large-scale model. Based on predetermined training data, the operator unit is used to perform a base calculation task based on the updated hyperparameters of the base large-scale model, and the base output results of the processed base large-scale model are obtained. Using the operator unit, the base output results of the processed base large-scale model and the labels corresponding to the training data are processed, and the base output accuracy of the processed base large-scale model is obtained. This includes identifying a similarity index for the specified base network layer using the operator unit based on the base output accuracy of the processed base large-scale model, The method according to claim 8.

10. Based on the model output results of the initial model, identifying the target loss information using the operator unit is: Based on the knowledge distillation mechanism, the operator unit processes the model output results of the initial model and the base output results of the base large-scale model to obtain output loss information. The operator unit processes at least one initial feature and the base task calculation result, and obtains intermediate loss information, wherein the base task calculation result is identified by performing a base calculation task based on the hyperparameters of the base network layer in the base large-scale model using the operator unit, the base output result is identified based on the base task calculation result, and the initial feature includes at least one of the first initial feature and the second initial feature. This includes fusing the output loss information and the intermediate loss information using the operator unit to obtain the target loss information, The method according to claim 7.

11. Memory unit and Operator unit, including, The aforementioned operator unit is Read the hyperparameters of the target network layer in the target model from the memory unit. Based on the hyperparameters of the target network layer, a first computation subtask is executed in the computation task, and a first feature output from the target network layer is obtained. When multiplexing the hyperparameters of the target network layer, a second computation subtask in the computation task is executed based on the first feature read from the memory unit, and the second feature output from the target network layer is obtained, and the first computation subtask and the second computation subtask are subtasks that are executed sequentially in the computation task. Based on the second feature read from the aforementioned storage unit, the system is configured to identify the model output result of the target model. Task execution device.

12. Memory unit and Operator unit, including, The aforementioned operator unit is Read the hyperparameters of the initial network layer in the initial model from the memory unit. Based on the hyperparameters of the initial network layer, the first computation subtask in the computation task is executed, and the first initial features output from the initial network layer are obtained. When the hyperparameters of the initial network layer are multiplexed, the second computation subtask in the computation task is executed based on the first initial features read from the memory unit, the second initial features output from the initial network layer are obtained, and the first and second computation subtasks are subtasks that are executed sequentially in the computation task. Based on the second initial features read from the memory unit, the model output result of the initial model is identified. Based on the model output results of the initial model, the target loss information is identified. Based on the target loss information, the system is configured to train the initial model and obtain the trained target model. Model training equipment.

13. At least one processor, Includes a memory that is communicably connected to at least one of the processors, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 10. electronic equipment.

14. A non-temporary computer-readable storage medium in which computer instructions are stored, The computer instruction causes the computer to perform the method described in any one of claims 1 to 10. A non-temporary, computer-readable storage medium on which computer instructions are stored.

15. A computer program, when executed by a processor, that implements the method according to any one of claims 1 to 10.

Images