Model training method and device, pre-training model, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a model training method and device, a pre-training model, electronic equipment and a storage medium, and the method comprises the steps: carrying out the simulation mapping of a to-be-trained model to a data flow architecture processor, and obtaining mapping information; determining performance information of the to-be-trained model according to the mapping information, wherein the performance information is used for indicating time consumption and accuracy of executing a data processing task by the to-be-trained model; according to the performance information, coefficients of multiple alternative weight matrixes included in an attention module in the to-be-trained model are adjusted until the performance information meets a termination condition, and the structures of the multiple alternative weight matrixes corresponding to the same weight matrix are different; and determining the structure of the weight matrix included in the attention module in the target model according to the coefficients of the plurality of alternative weight matrixes. According to the scheme, the running efficiency of the target model on the data stream architecture processor can be ensured.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, pre-trained model, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence (AI) technology, large models are playing an increasingly important role in the field of AI and are widely used in customer service, content creation, education and training, search engines, translation, data analysis, entertainment and games, healthcare, scientific research and many other fields.

[0003] Currently, large models mainly run on graphics processing units (GPUs) and achieve satisfactory results.

[0004] However, due to the different architectures of GPUs and dataflow processors, large models trained on GPUs are less efficient on dataflow processors. Summary of the Invention

[0005] In view of this, embodiments of this application provide a model training method, apparatus, pre-trained model, electronic device, and storage medium to at least solve or alleviate the above-mentioned problems.

[0006] According to a first aspect of the embodiments of this application, a training method is provided, comprising: simulating mapping a model to be trained to a dataflow architecture processor to obtain mapping information; determining performance information of the model to be trained based on the mapping information, the performance information being used to indicate the time consumption and accuracy of the model to be trained in performing data processing tasks; adjusting the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained based on the performance information until the performance information meets a termination condition, wherein the multiple candidate weight matrices corresponding to the same weight matrix have different structures; and determining the structure of the weight matrix included in the attention module of the target model based on the coefficients of the multiple candidate weight matrices.

[0007] According to a second aspect of the embodiments of this application, a pre-trained model is provided, the pre-trained model including at least one attention module, the pre-trained model being trained by the method described in the first aspect above.

[0008] According to a third aspect of the embodiments of this application, a model training apparatus is provided, comprising: a mapping unit, configured to simulate mapping a model to be trained to a dataflow architecture processor to obtain mapping information; an analysis unit, configured to determine performance information of the model to be trained based on the mapping information, wherein the performance information is used to indicate the time consumption and accuracy of the model to be trained in performing data processing tasks; an adjustment unit, configured to adjust the coefficients of a plurality of candidate weight matrices included in the attention module of the model to be trained based on the performance information until the performance information meets a termination condition, wherein the structures of the plurality of candidate weight matrices corresponding to the same weight matrix are different; and a determination unit, configured to determine the structure of the weight matrix included in the attention module of the target model based on the coefficients of the plurality of candidate weight matrices.

[0009] According to a fourth aspect of the present application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the method described in the first aspect.

[0010] According to a fifth aspect of the embodiments of this application, a computer storage medium is provided, on which a computer program is stored, the program being executed by a processor according to the method described in the first aspect above.

[0011] According to a sixth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform the method described in the first aspect above.

[0012] According to the model training scheme provided in the embodiments of this application, the model to be trained is simulated and mapped to a dataflow architecture processor to obtain mapping information. Based on the mapping information, the performance information of the model to be trained can be determined. Based on the performance information, the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained can be adjusted. Based on the adjusted coefficients and mapping information, new performance information can be determined. Then, based on the new performance information, the coefficients of the candidate weight matrices can be adjusted again. The above coefficient adjustment process is repeated until the performance information meets the termination condition. Then, based on the coefficients of the candidate weight matrices, the structure of the weight matrix included in the attention module of the target model is determined, so that the attention module included in the target module is suitable for running on the dataflow architecture processor, thereby ensuring the efficiency of the target model running on the dataflow architecture processor. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a schematic diagram of an exemplary system applied in one embodiment of this application;

[0015] Figure 2 This is a flowchart of a model training method according to an embodiment of this application;

[0016] Figure 3 This is a schematic diagram illustrating the mapping relationship between operation and execution units according to another embodiment of this application;

[0017] Figure 4 This is a schematic diagram illustrating the operation execution sequence of one embodiment of this application;

[0018] Figure 5 This is a schematic diagram illustrating the start and end times of an operation according to an embodiment of this application;

[0019] Figure 6 This is a schematic diagram of a model training apparatus according to an embodiment of this application;

[0020] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the essence of the present application, well-known methods, processes, and flows are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0022] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows.

[0023] Large models: Large models refer to machine learning models with large-scale parameters and complex computational structures, including large language models (Natural Language Processing, NLP), large vision models (Computer Vision, CV), and multimodal large models.

[0024] Dataflow Architecture: Dataflow architecture is a computing architecture distinct from shared-memory architectures such as CPUs and GPUs. It employs a multi-core architecture for both spatial and temporal computation, dividing data and tasks into microtasks mapped to multiple processing units and orchestrating data movement between these units. Dataflow architecture is a hardware and software architecture for implementing parallel computing in computer systems, processing and transferring data between system components in the form of continuous data transfers or data packets. Originating in the world of functional programming as a programming paradigm, dataflow represents operations as data streams. Operations transform one data stream into other data streams, commonly used in parallel computing and forming the foundation of many large-scale distributed computing frameworks.

[0025] Dataflow architecture processors: Dataflow architecture processors are computer processors designed with a dataflow architecture. Unlike the von Neumann architecture, which drives computation through instruction streams, dataflow architecture processors drive computation through data streams.

[0026] Network on Chip (NoC): Network on Chip (NoC) is a novel communication method for System on Chip (SoC). NoC connects multiple nodes on a chip, enabling reliable communication between them. The topologies that can be formed by the nodes in a NoC include 2D / 3D mesh networks, torus networks, and ring networks.

[0027] Processing Element (PE): A processing element is a basic computational block in a dataflow architecture. Processing elements are connected via an on-chip network.

[0028] Tensor: A tensor is data that is passed between processing units. Tensors are multi-dimensional data, such as matrices and vectors.

[0029] Attention Block: The attention block is a module in a large model that processes data based on an attention mechanism. Attention is a widely used technique in machine learning and deep learning, particularly in tasks such as Natural Language Processing (NLP), computer vision, and speech recognition. The core idea of ​​the attention mechanism is to allow the model to selectively focus on certain key parts of the input information, rather than averaging and uniformly processing all inputs, thereby improving the model's efficiency and performance.

[0030] Pre-trained models: Pre-trained models are a type of model used in machine learning. Their key feature is that they are pre-trained on massive amounts of data, allowing the model to learn rich semantic representations and knowledge before being applied to specific downstream tasks. This training method can significantly improve model performance and accelerate model convergence.

[0031] Application Environment of this Application

[0032] This application proposes a model training scheme. The entire model training scheme is relatively universal and can be used on various types of hardware devices with large-scale model training needs, such as data centers, servers, personal computers, Internet of Things (IoT) devices, and embedded devices. The model training scheme is independent of the hardware deployed on the computing device executing the scheme. The model training scheme provided in this application can be applied to train large-scale pre-trained models to obtain pre-trained models that can achieve good performance on dataflow architecture processors.

[0033] Exemplary System

[0034] Figure 1 An exemplary system for a model training method applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system may include a server 102, a communication network 104, and at least one user equipment 106. Figure 1 The example shown is multiple user devices 106. It should be noted that the solution in this embodiment can be applied to both the cloud server 102 and the user devices 106.

[0035] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can perform any suitable function. For example, in some embodiments, the cloud server 102 can be used for model training. As an optional example, in some embodiments, the cloud server 102 can train a large model running on a dataflow architecture processor. After mapping the model to be trained to the dataflow architecture processor, the structure of the weight matrix included in the attention module of the large model is determined based on the model performance to train a large model suitable for running on the dataflow architecture processor.

[0036] Communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, communication network 104 can include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. User equipment 106 can be connected to communication network 104 via one or more communication links (e.g., communication link 112), which can be linked to cloud server 102 via one or more communication links (e.g., communication link 114). Communication links can be any communication link suitable for transmitting data between cloud server 102 and user equipment 106, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0037] User device 106 may include any one or more user devices suitable for interaction. In some embodiments, when model training is performed by cloud server 102, user device 106 may send model training instructions to cloud server 102, causing cloud server 102 to simulate and map the corresponding model to be trained to a dataflow architecture processor according to the model training instructions, and determine the structure of the weight matrix included in the attention module of the model to be trained based on the running performance of the model to be trained on the dataflow architecture processor, thereby training a large model suitable for running on the dataflow architecture processor. In other embodiments, user device 106 may perform model training locally. User device 106 simulates and maps the corresponding model to be trained to a dataflow architecture processor based on the user's model training operations, and determines the structure of the weight matrix included in the attention module of the model to be trained based on the running performance of the model to be trained on the dataflow architecture processor, thereby training a large model suitable for running on the dataflow architecture processor.

[0038] User equipment 106 may include any suitable type of device, such as mobile devices, tablet computers, laptop computers, desktop computers, wearable computers, vehicle systems, smart speakers and / or any other suitable type of user equipment.

[0039] This application embodiment mainly focuses on the process of model training on the cloud server 102 or user equipment 106. The model training process will be described in detail later.

[0040] Model training methods

[0041] Based on the above system, this application provides a model training method, which can be executed by the cloud server 102 or the user equipment 106 in the above system embodiment. The following describes the model training method in detail through multiple embodiments.

[0042] Figure 2 This is a flowchart of a model training method according to an embodiment of this application. Figure 2 As shown, the model training method includes the following steps:

[0043] Step 201: Simulate mapping the model to be trained to the dataflow architecture processor to obtain mapping information.

[0044] The model to be trained is a large model that needs to be deployed to run on a dataflow architecture processor, and it includes an attention module. For example, the model to be trained can be a generative pre-trained (GPT) model. Since GPUs and dataflow architecture processors have different architectures, in order to make the trained model suitable for running on dataflow architecture processors and achieve good performance, it is necessary to verify the performance of the model to be trained on dataflow architecture processors during the model training process.

[0045] During model training, the model to be trained needs to undergo multiple iterative optimizations. If the model to be trained is deployed to a dataflow architecture processor to verify its performance, it needs to be deployed to the dataflow architecture processor after each iteration, which consumes a lot of time and leads to low model training efficiency. By simulating and mapping the model to be trained onto a dataflow architecture processor, the performance of the model to be trained after each iteration can be verified through simulation, thus ensuring the efficiency of model training.

[0046] After simulating the mapping of the model to be trained onto the dataflow architecture processor, mapping information can be obtained. The mapping information is the information needed to determine the performance of the model to be trained by simulation. The mapping information can at least indicate the mapping relationship between the multiple operations included in the attention module of the model to be trained and the multiple execution units included in the dataflow architecture processor. The mapping of a certain operation to a certain execution unit means that when the model to be trained performs a data processing task, the operation is executed by that execution unit.

[0047] Step 202: Determine the performance information of the model to be trained based on the mapping information.

[0048] The structure of the weight matrix within the attention module of the model to be trained affects its performance. The training process includes optimizing both the weight matrix structure and its numerical values. After one round of optimization of the weight matrix structure, the performance of the model after optimization can be determined based on mapping information. The performance of the model to be trained can be characterized by performance information, which indicates the time and accuracy of the model in performing data processing tasks.

[0049] Step 203: Adjust the coefficients of the multiple candidate weight matrices included in the attention module of the model to be trained based on the performance information until the performance information meets the termination condition.

[0050] The attention module includes multiple weight matrices, such as the query weight matrix W. Q Key weight matrix W K Value weight matrix W V Output weight matrix W O and linear weight matrix W Linear The attention module processes the input data based on the attention mechanism through its weight matrices.

[0051] During the training of the model to be trained, the attention module in the model to be trained includes multiple candidate weight matrices. Each weight matrix in the attention module of the target model (the model obtained by training the model to be trained) corresponds to one or more candidate weight matrices. Different candidate weight matrices corresponding to the same weight matrix have different structures. The structure of a matrix refers to the number of rows and columns; the structure of a candidate weight matrix refers to the number of rows and columns; and the structure of a weight matrix refers to the number of rows and columns. Different structures of two matrices mean that the number of rows and / or columns of these two matrices are different.

[0052] Each candidate weight matrix in the attention module of the model to be trained has a corresponding coefficient. If a certain weight matrix corresponds to only one candidate weight matrix, then the coefficient of the candidate weight matrix is ​​1. If a certain weight matrix corresponds to multiple candidate weight matrices, then the sum of the coefficients of these multiple candidate weight matrices is equal to 1.

[0053] It should be noted that if the attention module in the model to be trained has only one candidate weight matrix corresponding to a certain weight matrix, then the structure of the weight matrix in the attention module of the target model corresponding to that candidate weight matrix is ​​the same as the structure of that candidate weight matrix.

[0054] During the training process of the model to be trained, the structure of the weight matrix is ​​optimized by adjusting the coefficients of the candidate weight matrices. After the optimization of the weight matrix structure is completed, the structure of the weight matrix is ​​determined based on the coefficients of at least one candidate weight matrix corresponding to the same weight matrix.

[0055] In optimizing the weight matrix structure, the training model can be trained using a training set. During training, based on the coefficients and mapping information of the candidate weight matrices, the time required for the model to perform data processing tasks is determined. The accuracy of the model in performing these tasks is determined based on its processing results on the samples in the training set. The coefficients of the candidate weight matrices can then be adjusted based on the performance information indicating time and accuracy. New performance information can be determined based on the adjusted coefficients. This adjustment process is repeated iteratively to optimize the coefficients of the candidate weight matrices until the performance information of the model meets the termination condition. The termination condition can be that the time and accuracy of the model in performing data processing tasks converge, or the time is less than the target value and the accuracy is higher than the target value, or the number of training samples reaches a set target value.

[0056] Step 204: Determine the structure of the weight matrix included in the attention module of the target model based on the coefficients of multiple candidate weight matrices.

[0057] The coefficients of the candidate weight matrices in the model to be trained are adjusted until the performance information obtained by the model after coefficient adjustment meets the termination condition. Then, based on the coefficients of the candidate weight matrices, the structure of the weight matrices included in the attention module of the target model is determined. If the attention module in the model to be trained has only one candidate weight matrix corresponding to a certain weight matrix, the coefficient of that candidate weight matrix remains 1 throughout the optimization process. Therefore, the structure of the weight matrix in the target model corresponding to that candidate weight matrix is ​​determined to be the structure of that candidate weight matrix. If the attention module in the model to be trained has multiple candidate weight matrices corresponding to a certain weight matrix, in one example, the structure of the weight matrix in the target model corresponding to these multiple candidate weight matrices can be determined as the structure of the candidate weight matrix with the largest corresponding coefficient among these multiple candidate weight matrices.

[0058] It should be noted that training the model to be trained includes not only optimizing the structure of the weight matrix, but also optimizing the values ​​of the weight matrix. In addition to determining the structure of the weight matrix included in the attention module, it is also necessary to determine the values ​​of the weight matrix included in the attention module. Only after determining the structure and values ​​of the weight matrix included in the attention module can the training of the model to be trained be considered complete, and the target model be obtained.

[0059] In this embodiment, the model to be trained is simulated and mapped to a dataflow architecture processor to obtain mapping information. Based on the mapping information, the performance information of the model to be trained can be determined. Based on the performance information, the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained can be adjusted. Based on the adjusted coefficients and the mapping information, new performance information can be determined. Then, based on the new performance information, the coefficients of the candidate weight matrices can be adjusted again. The above coefficient adjustment process is repeated until the performance information meets the termination condition. Then, based on the coefficients of the candidate weight matrices, the structure of the weight matrix included in the attention module of the target model is determined, so that the attention module included in the target module is suitable for running on the dataflow architecture processor, thereby ensuring the efficiency of the target model running on the dataflow architecture processor.

[0060] In one possible implementation, the attention module in the model to be trained includes multiple first candidate weight matrices, multiple second candidate weight matrices, multiple third candidate weight matrices, and multiple fourth candidate weight matrices, wherein the first candidate weight matrix corresponds to the query weight matrix W. Q Bond weight matrix W K The first candidate weight matrix is ​​the second candidate weight matrix, which corresponds to the value weight matrix W. V The third candidate weight matrix corresponds to the output weight matrix W. O The fourth candidate weight matrix corresponds to the linear weight matrix W. Linear The alternative weight matrix.

[0061] When processing input data based on the attention mechanism, the attention module compares the input data with the query weight matrix W. Q Bond weight matrix W K Multiply the input data with the query weight matrix W. Q Multiply to obtain the query matrix Q, input data and key weight matrix W K The key matrix K is obtained by multiplying the query matrix Q. Then, the transposed key matrix K needs to be multiplied. For this, the query matrix Q and the key matrix K must have the same structure. The structure of the query matrix Q is determined by the query weight matrix W. Q The structure of the bond matrix K is determined by the bond weight matrix W. K The decision is made, so the weight matrix W is queried. Q Bond weight matrix W K They need to have the same structure, therefore, when determining the query weight matrix W Q After structuring, the key weight matrix W K The structure was then determined.

[0062] It should be noted that the attention module in the model to be trained includes multiple weight matrices W corresponding to the query. QThe first candidate weight matrix, and includes multiple corresponding key weight matrices W. K The first alternative weight matrix. For example, the attention module in the model to be trained includes a weight matrix W corresponding to the query. Q The first alternative weight matrix W Q1 First candidate weight matrix W Q2 First candidate weight matrix W Q3 and the first candidate weight matrix W Q4 It also includes the key weight matrix W K The first alternative weight matrix W K1 First candidate weight matrix W K2 First candidate weight matrix W K3 and the first candidate weight matrix W K4 W Q1 W Q2 W Q3 and W Q4 Different structures, W Q1 and W K1 The structures are the same, W Q2 and W K2 The structures are the same, W Q3 and W K3 The structures are the same, W Q4 and W K4 The structures are the same.

[0063] In the attention module of the model to be trained, the first, second, third, and fourth candidate weight matrices each have corresponding coefficients. In one example, this corresponds to the query weight matrix W. Q The sum of the coefficients of each of the first candidate weight matrices is equal to 1, corresponding to the key weight matrix W. K The sum of the coefficients of each of the first candidate weight matrices is equal to 1, corresponding to the value weight matrix W. V The sum of the coefficients of each of the second candidate weight matrices is equal to 1, corresponding to the output weight matrix W. O The sum of the coefficients of each of the third candidate weight matrices is equal to 1, corresponding to the linear weight matrix W. Linear The sum of the coefficients of each of the fourth candidate weight matrices is equal to 1.

[0064] Different alternative weight matrices corresponding to the same weight matrix have different structures, that is, they correspond to the query weight matrix W. Q The structures of the first candidate weight matrices are different, corresponding to the key weight matrix W. K The structures of the first candidate weight matrices are different, corresponding to the value weight matrix W. V The structures of the second candidate weight matrices are different, corresponding to the output weight matrix W. OThe structures of the third candidate weight matrices are different, corresponding to the linear weight matrix W. Linear The structures of the fourth candidate weight matrices are different.

[0065] During the optimization of the weight matrix structure, the coefficients of the first, second, third, and fourth candidate weight matrices are adjusted based on performance information. After the weight matrix structure optimization is complete, the coefficients of the first candidate weight matrix will determine the query weight matrix W. Q Bond weight matrix W K The structure of the second alternative weight matrix determines the value of the weight matrix W. V The structure of the third alternative weight matrix determines the output weight matrix W. O The structure of the fourth candidate weight matrix determines the linear weight matrix W. Linear The structure.

[0066] In this embodiment, the general attention module includes a query weight matrix W. Q Key weight matrix W K Value weight matrix W V Output weight matrix W O and linear weight matrix W Linear The attention module in the model to be trained includes the weight matrix W corresponding to the query. Q Bond weight matrix W K Multiple first candidate weight matrices, corresponding to the value weight matrix W V Multiple second candidate weight matrices, corresponding to the output weight matrix W O Multiple third alternative weight matrices, and corresponding linear weight matrix W Linear Multiple fourth candidate weight matrices are given, each of the first, second, third, and fourth candidate weight matrices having corresponding coefficients. The coefficients of these matrices are adjusted based on performance information. The query weight matrix W is then determined based on the coefficients of the first candidate weight matrices. Q Bond weight matrix W K The structure is based on the coefficient determination value of the second candidate weight matrix, weight matrix W. V The structure is such that the output weight matrix W is determined based on the coefficients of the third candidate weight matrix. O The structure is such that the linear weight matrix W is determined based on the coefficients of the fourth candidate weight matrix. Linear The structure of W includes the query weight matrix. Q Key weight matrix W K Value weight matrix W V Output weight matrix WO and linear weight matrix W Linear The attention module is suitable for running on dataflow architecture processors, thereby ensuring the efficiency of the target model on dataflow architecture processors.

[0067] In one possible implementation, after optimizing the weight matrix structure, the structure of the weight matrix included in the attention module of the target model can be determined based on the coefficients of the candidate weight matrices. In one example, the structure of a certain weight matrix in the target model can be determined to be the same as the structure of the candidate weight matrix with the largest corresponding coefficient among multiple candidate weight matrices. Specifically, the query weight matrix W included in the attention module of the target model is determined. Q Bond weight matrix W K The structure is the same as that of the first candidate weight matrix with the largest corresponding coefficient in each of the first candidate weight matrices; determine the value weight matrix W included in the attention module of the target model. V The structure is the same as that of the second candidate weight matrix with the largest corresponding coefficient in each of the second candidate weight matrices; determine the output weight matrix W included in the attention module of the target model. O The structure is the same as that of the third candidate weight matrix with the largest corresponding coefficient in each of the third candidate weight matrices; determine the linear weight matrix W included in the attention module of the target model. Linear The structure is the same as that of the fourth candidate weight matrix with the largest corresponding coefficient in each of the fourth candidate weight matrices.

[0068] In this embodiment, during the optimization of the weight matrix structure, the coefficients of the candidate weight matrices are adjusted based on performance information. The coefficients of candidate weight matrices that reduce the time consumption and improve the accuracy of the data processing task performed by the model to be trained are increased, while the coefficients of candidate weight matrices that increase the time consumption and decrease the accuracy are decreased. After the optimization of the weight matrix structure is completed, the structure of the candidate weight matrix with the largest corresponding coefficient is used as the structure of the corresponding weight matrix included in the attention module of the target model. This allows the target model to have lower time consumption and higher accuracy when running and performing data processing tasks on a dataflow architecture processor, thereby ensuring the efficiency and accuracy of the target model in performing data processing tasks.

[0069] In one possible implementation, when simulating the mapping of the model to be trained to the dataflow architecture processor, the data processing procedures included in the attention of the model to be trained can be broken down into multiple operations, and then the multiple operations can be mapped to multiple execution units included in the dataflow architecture processor to obtain mapping information that can indicate the mapping relationship between operations and execution units.

[0070] The data processing of the attention module in the model to be trained includes various types of operations such as computation, transmission, and memory access. The dataflow architecture processor includes various types of execution units, such as computation units for computational operations, transmission units for data transmission, and storage units for data storage. Different types of operations need to be executed by corresponding execution units. The time taken for an execution unit to perform a corresponding operation is affected by the performance of the execution unit and the complexity of the operation. The complexity of the operation is related to the structure of the weight matrix included in the attention module. It should be noted that the time taken for a storage unit to perform an operation refers to the time taken to write data to or read data from the storage unit.

[0071] In one example, the computing unit can be a CPU core, a CUDA core, or a Processing Element (PE). The transmission unit can be a Network On Chip (NoC), a Crossbar (Xbar) bus, a Compute Express Link (CXL) bus, or an Advanced eXtensible Interface (AXI) bus. The storage unit can be a Scratchpad Memory (SPM), a cache, Dynamic Random Access Memory (DRAM), Double Data Rate Synchronous Dynamic Random-Access Memory (DDR), Static Random Access Memory (SRAM), or a Solid State Disk (SSD).

[0072] After the data processing of the attention module in the model to be trained is broken down into multiple operations, the broken-down operations are mapped to the data flow architecture processor, including the execution unit, and mapping information is generated to record the mapping relationship between the operations and the execution unit.

[0073] In this embodiment, the data processing process of the attention module in the model to be trained is broken down into multiple operations. These operations are then mapped to multiple execution units included in the dataflow architecture processor. Mapping information indicating the mapping relationship between the operations and the execution units is obtained. The time taken by the execution units to perform the corresponding operations can be analyzed based on the mapping information. By combining the execution time of each operation, the time taken by the model to be trained to perform the data processing task can be determined. This eliminates the need to determine the time taken by actually performing the data processing task on the model to be trained, thus improving the efficiency of model training.

[0074] In one possible implementation, when determining the performance information of the model to be trained based on the mapping information, the start and end times of each operation can be determined based on the mapping information, the dependencies between multiple operations obtained from the decomposition, and the attribute information of the dataflow architecture processor. Based on the start and end times of each operation, the time taken by the attention module to perform the data processing task can be determined, and the time taken by the attention module to perform the data processing task can reflect the time taken by the model to be trained to perform the data processing task.

[0075] The execution time of a specific execution unit is determined by the performance of the dataflow architecture processor and the complexity of the operation. The dataflow architecture processor's attribute information indicates its performance, while mapping information indicates the complexity of the operation. Therefore, the execution time of an execution unit can be determined based on the mapping information and the dataflow architecture processor's attribute information. In the attention module of the model under training, some operations can be executed in parallel during data processing, while others require the completion of preceding operations before they can begin. Therefore, determining only the execution time of each individual operation is insufficient to determine the overall data processing time of the attention module. To address this, the start and end times of each operation can be determined based on its execution time and the dependencies between operations.

[0076] It should be noted that the start and end times of an operation are relative times, not absolute times. For example, the start time of the first operation can be set as the starting point, and the start and end times of other operations can be referenced to this point.

[0077] During the training process of the model to be trained, samples from the training dataset are input into the model for processing. The samples in the training dataset include labels. By comparing the output of the model to be trained based on the input samples with the labels included in the samples, the accuracy of the model to be trained in processing the samples in the training dataset can be determined, thereby obtaining accuracy information that can indicate this accuracy.

[0078] Based on the start and end times of the operation, the time consumed by the attention module in the model to be trained to process data can be determined. Since the optimization of the weight matrix structure is to improve the data processing efficiency of the attention module in the model to be trained, the data processing efficiency of other modules in the model to be trained can be disregarded. Therefore, the time consumed by the attention module in the model to be trained to process data is taken as the time consumed by the model to be trained to execute the data processing task. By combining the accuracy information, performance information indicating the time consumed and the accuracy of the model to be trained to execute the data processing task can be obtained.

[0079] In this embodiment, the start and end times of operations are determined based on mapping information, dependencies between operations, and attribute information of the data flow architecture processor. The time taken for the model to be trained to perform data processing tasks can be determined based on the start and end times of each operation. Accuracy information can be obtained based on the accuracy of the model to be trained in processing samples in the training dataset. In turn, performance information including the time taken and accuracy of the model to be trained to perform data processing tasks can be obtained, ensuring that the performance information can accurately reflect the time taken and accuracy of the model to be trained to perform data processing tasks.

[0080] In one possible implementation, multiple operations can be obtained by splitting the data processing of the attention module in the model to be trained. The split operations can include at least one of computation operations, data transfer operations, and memory access operations. The computation operations can include matrix operation operations and / or vector operation operations.

[0081] In one example, matrix operations may include General Matrix to Matrix Multiplication (GEMM) and General Matrix Addition (GMA). Vector operations may include Vector Permutate Unit (VPU), vector addition, vector dot product, vector inner product, and vector outer product.

[0082] After breaking down the data processing of the attention module in the model to be trained into multiple operations, the operations can be mapped to the execution units included in the dataflow architecture processor according to the type of operation, and the execution units can simulate the execution of the corresponding operations. Figure 3 A schematic diagram illustrating the mapping relationship between operations and execution units according to an embodiment of this application is shown, such as... Figure 3 As shown, the dataflow architecture processor includes a computing unit, a transmission unit, and a storage unit. Matrix operations and vector operations can be mapped to the computing unit, data transmission operations can be mapped to the transmission unit, and memory access operations can be mapped to the storage unit.

[0083] In this embodiment, the attention module performs computation, data transmission, and memory access operations when performing data processing. The data processing process of the attention module in the model to be trained is broken down into one or more of computation operations, data transmission operations, and memory access operations. Based on each of the decomposed operations, the time consumed by the attention module when performing data processing can be comprehensively analyzed, thereby ensuring the accuracy of the time consumed by the model to be trained to perform data processing tasks determined according to each operation.

[0084] In one possible implementation, after the data processing of the attention module in the model to be trained is split into multiple operations, at least some of these operations have dependencies on each other. The dependencies between operations determine the order in which the operations are executed. Based on the order in which the split operations are executed, the start time of the first executed operation is set to 0, and the start time of the i-th executed operation is set to the end time of the (i-1)-th executed operation. i is an integer greater than 1, and i is less than or equal to the number of split operations.

[0085] Depending on the dependencies between operations, some operations that do not have dependencies can be executed in parallel, while operations that have dependencies need to be executed sequentially. Figure 4 A schematic diagram illustrating the operation execution sequence of one embodiment of this application is shown, as follows: Figure 4 As shown, the data processing of the attention module in the model to be trained is broken down into operations A, B, C, D, and E. Operation A can output tensor T2 based on the input tensor T0; operation B can output tensor T3 based on the input tensor T1; operation C can output tensor T4 based on the input tensor T3; operation D can transfer tensor T4 to the execution unit executing operation E; and operation E can output tensor T5 based on the input tensors T2 and T4. Operations A and B are memory access operations, such as reading from SPM, cache, or DRAM; operation C is a vector operation; operation D is a data transfer operation; and operation E is a matrix operation. Operations A and E are executed by PE0, while operations B, C, and D are executed by PE1.

[0086] Operations A and B have no constraints, so they can be executed in parallel. Operation C is dependent on operation B; the input tensor of operation C is the output tensor of operation B, so the start time of operation C is the end time of operation B. Operation C is dependent on operation D; operation D transmits the output tensor of operation C to the execution unit executing operation E, so the start time of operation D is the end time of operation C. Operation E is dependent on both operations A and D; the input tensor of operation E is the output tensor of operation A and the tensor transmitted by operation D, so the start time of operation E is the later of the end times of operation A and operation D. If operation A sends its tensor to the execution unit executing operation E before operation D, then the start time of operation E is the end time of operation D; if operation D sends its tensor to the execution unit executing operation E before operation A, then the start time of operation E is the end time of operation A.

[0087] exist Figure 4 In the operation execution sequence shown, operation A and operation B are the first operations to be executed, operation C is the second operation to be executed, operation D is the third operation to be executed, and operation E is the fourth operation to be executed.

[0088] Different types of operations can have their execution time determined according to the corresponding strategies. After determining the start time of the operation, the end time of the operation can be determined based on its execution time.

[0089] If an operation is a computational operation, the end time of the operation is determined based on the structure of the candidate weight matrix included in the attention module of the model to be trained, the matrix partitioning rules, the attributes of the execution unit that performs the operation, and the start time of the operation.

[0090] If an operation is a transmission operation, the end time of the operation is determined based on the amount of data in the tensor to be transmitted, the transmission link and link bandwidth determined based on the mapping information, and the start time of the operation.

[0091] If an operation is a memory access operation, the end time of the operation is determined based on the amount of data in the access tensor, the memory access bandwidth and memory access policy determined based on the mapping information, and the start time of the operation.

[0092] In this embodiment of the application, different types of operations are affected by different factors in terms of their operation time. After determining the start time of the operation, the operation time of the operation is determined according to the various factors affecting the operation time. Then, the end time of the operation is determined according to the start time and operation time, so as to ensure that the end time of different types of operations can be accurately determined, thereby ensuring the accuracy of performance information.

[0093] In one possible implementation, since the start time of the first executed operation is 0 and the start time of the i-th executed operation is the end time of the (i-1)-th executed operation, the data processing of the attention module in the model to be trained is completed when the last executed operation is finished. Therefore, the end time of the last executed operation can be determined as the time taken for the model to be trained to perform the data processing task, based on the order in which the operations are executed.

[0094] by Figure 4 Taking the execution order of the operations shown as an example, Figure 5 The start and end times of each operation are shown. For example... Figure 5 As shown, the start time of operations A and B is 0, the end time of operation A is t1, and the end time of operation B is t2, where t1 is greater than t2. The start time of operation C is t2, and the end time of operation C is t3, where t3 equals t1. The start time of operation D is t3, and the end time of operation D is t4, where t4 is greater than t3. The start time of operation E is t4, and the end time of operation E is t5, where t5 is greater than t4. Since operation E is the last operation executed, the time taken for the training model to perform the data processing task is equal to the end time of operation E, i.e., the time taken for the training model to perform the data processing task is t5.

[0095] In this embodiment, the start time of the first executed operation is 0. Since the start time of the i-th executed operation is the end time of the (i-1)-th executed operation, the end time of the last executed operation is the time consumed by the attention module in the model to be trained to perform data processing. Because the weight matrix optimization process only focuses on the time consumed by the attention module to perform data processing, this time can be determined as the time consumed by the model to be trained to perform data processing. Determining the end time of the last executed operation as the time consumed by the model to be trained eliminates the need for other calculations, saving computation time and ensuring the efficiency of model training.

[0096] In one possible implementation, the training process of the model to be trained includes not only optimizing the structure of the weight matrix but also optimizing the values ​​of the weight matrix. When optimizing the weight matrix values, the elements of the candidate weight matrices can be adjusted using the training dataset. Specifically, the training samples in the training dataset are input into the model to be trained, and the values ​​of the elements in each candidate weight matrix are adjusted based on the output of the model and the labels of the training samples. It should be noted that the values ​​of the elements in the candidate weight matrices can be adjusted in any suitable way, such as by adjusting the values ​​of the middle elements of the candidate weights using backpropagation algorithms or loss functions. This application does not limit the method used to adjust the values ​​of the elements in the candidate weight matrices.

[0097] The optimization of the weight matrix structure and the optimization of the weight matrix values ​​can be carried out alternately. That is, the adjustment of the coefficients of the candidate weight matrix and the adjustment of the elements included in the candidate weight matrix are carried out alternately. When adjusting the coefficients of the candidate weight matrix, the elements included in the candidate weight matrix remain unchanged, and when adjusting the elements included in the candidate weight matrix, the coefficients of the candidate weight matrix remain unchanged.

[0098] In one example, when adjusting the coefficients of the candidate weight matrix, the values ​​of the elements in the candidate weight matrix are locked, and the coefficients are adjusted over 5-10 epochs. Here, an epoch refers to the entire training dataset being used completely once during training; one epoch means that each training sample in the training dataset is used to update the model's parameters.

[0099] The optimization of the weight matrix structure and weight matrix values ​​can be performed multiple times until the time and accuracy of the model to be trained in performing data processing tasks converge, or the time of the model to be trained in performing data processing tasks is less than the target value and the accuracy is higher than the target value, or the number of training samples used to optimize the weight matrix structure and the number of training samples used to optimize the weight matrix data both reach the set target number. In one example, the element values ​​in each candidate weight matrix are locked, and the coefficients of each candidate weight matrix are optimized for 10 epochs. Then, the coefficients of each candidate weight matrix are locked, and the element values ​​in each candidate weight matrix are optimized for 10 epochs. Then, the element values ​​in each candidate weight matrix are locked, and the coefficients of each candidate weight matrix are optimized for 10 epochs. Then, the coefficients of each candidate weight matrix are locked, and the element values ​​in each candidate weight matrix are optimized for 10 epochs. After two rounds of optimization of the weight matrix structure and weight matrix values, the target model is determined based on the coefficients of each candidate weight matrix in the model to be trained at this point.

[0100] After the performance information of the model to be trained meets the termination condition, the candidate weight matrix with the largest corresponding coefficient among multiple candidate weight matrices corresponding to the same weight matrix in the model to be trained is determined as the weight matrix included in the attention module of the target model.

[0101] In one example, after the performance information of the model to be trained meets the termination condition, the corresponding query weight matrix W in the model to be trained is... Q The candidate weight matrix with the largest coefficient among multiple candidate weight matrices is determined as the query weight matrix W of the attention module in the target model.Q The corresponding key weight matrix W in the model to be trained K The candidate weight matrix with the largest coefficient among multiple candidate weight matrices is determined as the key weight matrix W of the attention module in the target model. K The corresponding weight matrix W in the model to be trained V The candidate weight matrix with the largest coefficient among multiple candidate weight matrices is determined as the value weight matrix W of the attention module in the target model. V The corresponding output weight matrix W in the model to be trained O The candidate weight matrix with the largest coefficient among multiple candidate weight matrices is determined as the output weight matrix W of the attention module in the target model. O The corresponding linear weight matrix W in the model to be trained Linear The candidate weight matrix with the largest coefficient among multiple candidate weight matrices is determined as the linear weight matrix W of the attention module in the target model. Linear .

[0102] In this embodiment, when optimizing the weight matrix structure, the values ​​of the elements in the weight matrix are locked, and when optimizing the weight matrix values, the coefficients of the weight matrix are locked. The coefficients and values ​​of the candidate weight matrices included in the attention module of the model to be trained are optimized alternately. Thus, the model to be trained is optimized in terms of both the structure and the values ​​of the weight matrix, ensuring that the trained target model is suitable for running on a dataflow architecture processor and has high timeliness and accuracy when performing data processing tasks.

[0103] pre-trained model

[0104] One embodiment of this application provides a pre-trained model, which includes at least one attention module. This pre-trained model is obtained by training using the model training method described in any of the above embodiments. The pre-trained model can be a generative pre-trained (GPT) model, etc., and is suitable for running on a dataflow architecture processor.

[0105] It should be noted that the pre-trained model in this application embodiment is trained based on the model training method in the aforementioned method embodiment. Therefore, the pre-trained model in this application embodiment is a specific application of the model training method in the aforementioned embodiment. The pre-trained model can be a GPT model, etc. For the specific training process of the pre-trained model, please refer to the description in the aforementioned model training method embodiment, which will not be repeated here.

[0106] Model training device

[0107] Corresponding to the above-described model training method implementation examples, Figure 6A schematic diagram of a model training apparatus according to an embodiment of this application is shown. Figure 6 As shown, the model training device 600 includes:

[0108] The mapping unit 601 is used to simulate mapping the model to be trained to the dataflow architecture processor to obtain mapping information;

[0109] Analysis unit 602 is used to determine the performance information of the model to be trained based on the mapping information. The performance information is used to indicate the time and accuracy of the model to be trained in performing data processing tasks.

[0110] The adjustment unit 603 is used to adjust the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained according to the performance information until the performance information meets the termination condition. The multiple candidate weight matrices corresponding to the same weight matrix have different structures.

[0111] The determination unit 604 is used to determine the structure of the weight matrix included in the attention module of the target model based on the coefficients of multiple candidate weight matrices.

[0112] In this embodiment, the mapping unit 601 simulates mapping the model to be trained onto a dataflow architecture processor to obtain mapping information. The analysis unit 602 determines the performance information of the model to be trained based on the mapping information. The adjustment unit 603 adjusts the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained based on the performance information. The analysis unit 602 determines new performance information based on the adjusted coefficients and the mapping information. Then, the adjustment unit 603 adjusts the coefficients of the candidate weight matrices based on the new performance information. The above coefficient adjustment process is repeated until the performance information meets the termination condition. Then, the determination unit 604 determines the structure of the weight matrix included in the attention module of the target model based on the coefficients of the candidate weight matrix, so that the attention module included in the target module is suitable for running on the dataflow architecture processor, thereby ensuring the efficiency of the target model running on the dataflow architecture processor.

[0113] It should be noted that the model training device in this embodiment is used to implement the model training method in the aforementioned method embodiment and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0114] electronic devices

[0115] Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Specific embodiments of this application do not limit the specific implementation of the electronic device. Figure 7As shown, the electronic device may include: a processor 702, a communications interface 704, a memory 706, and a communications bus 708. Wherein:

[0116] The processor 702, communication interface 704, and memory 706 communicate with each other via communication bus 708.

[0117] Communication interface 704 is used to communicate with other electronic devices or servers.

[0118] The processor 702 is used to execute program 710, which can specifically execute the relevant steps in any of the aforementioned model training method embodiments.

[0119] Specifically, program 710 may include program code that includes computer operation instructions.

[0120] The processor 702 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0121] RISC-V is an open-source instruction set architecture based on the Reduced Instruction Set Computing (RISC) principle. It can be applied to various aspects of microcontrollers and FPGA chips, specifically in areas such as IoT security, industrial control, mobile phones, and personal computers. Because its design considers small size, speed, and low power consumption, it is particularly suitable for modern computing devices such as warehouse-scale cloud computers, high-end mobile phones, and tiny embedded systems. With the rise of AIoT (Artificial Intelligence of Things), the RISC-V instruction set architecture is receiving increasing attention and support and is expected to become the next generation of widely used CPU architecture.

[0122] The computer operation instructions in this application embodiment can be computer operation instructions based on the RISC-V instruction set architecture. Correspondingly, the processor 702 can be designed based on the RISC-V instruction set. Specifically, the processor chip in the electronic device provided in this application embodiment can be a chip designed using the RISC-V instruction set. This chip can execute executable code based on the configured instructions, thereby implementing the model training method in the above embodiment.

[0123] Memory 706 is used to store program 710. Memory 706 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0124] Specifically, program 710 can be used to cause processor 702 to execute the model training method in any of the foregoing embodiments.

[0125] The specific implementation of each step in program 710 can be found in the corresponding steps and units described in any of the aforementioned model training method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0126] The electronic device in this application embodiment simulates mapping the model to be trained onto a dataflow architecture processor to obtain mapping information. Based on the mapping information, the performance information of the model to be trained can be determined. Based on the performance information, the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained can be adjusted. Based on the adjusted coefficients and mapping information, new performance information can be determined. Then, based on the new performance information, the coefficients of the candidate weight matrices can be adjusted again. The above coefficient adjustment process is repeated until the performance information meets the termination condition. Then, based on the coefficients of the candidate weight matrices, the structure of the weight matrix included in the attention module of the target model is determined, so that the attention module included in the target module is suitable for running on the dataflow architecture processor, thereby ensuring the efficiency of the target model running on the dataflow architecture processor.

[0127] Computer storage media

[0128] This application also provides a computer-readable storage medium storing instructions for causing a machine to perform the model training method as described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0129] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute a part of this application.

[0130] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0131] Computer program products

[0132] This application also provides a computer program product, including computer instructions that instruct a computing device to perform any corresponding operation in the above-described plurality of method embodiments.

[0133] It should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0134] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0135] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0136] It should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0137] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0138] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A model training method, comprising: The model to be trained is simulated and mapped to the dataflow architecture processor to obtain mapping information; The performance information of the model to be trained is determined based on the mapping information, and the performance information is used to indicate the time and accuracy of the model to be trained in performing data processing tasks; The coefficients of multiple candidate weight matrices included in the attention module of the model to be trained are adjusted according to the performance information until the performance information meets the termination condition, wherein the structures of multiple candidate weight matrices corresponding to the same weight matrix are different. Based on the coefficients of the multiple candidate weight matrices, the structure of the weight matrix included in the attention module of the target model is determined.

2. The method according to claim 1, wherein, The attention module in the model to be trained includes multiple first candidate weight matrices, multiple second candidate weight matrices, multiple third candidate weight matrices, and multiple fourth candidate weight matrices. The first candidate weight matrix corresponds to the query weight matrix, the second candidate weight matrix corresponds to the value weight matrix, the third candidate weight matrix corresponds to the output weight matrix, and the fourth candidate weight matrix corresponds to the linear weight matrix. Each of the first, second, third, and fourth candidate weight matrices has corresponding coefficients. The step of adjusting the coefficients of the multiple candidate weight matrices included in the attention module of the model to be trained based on the performance information includes: Based on the performance information, the coefficients of the first candidate weight matrix, the second candidate weight matrix, the third candidate weight matrix, and the fourth candidate weight matrix are adjusted respectively.

3. The method according to claim 2, wherein, The step of determining the structure of the weight matrix included in the attention module of the target model based on the coefficients of the plurality of candidate weight matrices includes: The structures of the query weight matrix and key weight matrix included in the attention module of the target model are determined to be the same as the structure of the first candidate weight matrix with the largest corresponding coefficient among the plurality of first candidate weight matrices; The structure of the value weight matrix included in the attention module of the target model is determined to be the same as the structure of the second candidate weight matrix with the largest corresponding coefficient among the plurality of second candidate weight matrices; The structure of the output weight matrix included in the attention module of the target model is determined to be the same as the structure of the third candidate weight matrix with the largest corresponding coefficient among the plurality of third candidate weight matrices; The structure of the linear weight matrix included in the attention module of the target model is determined to be the same as the structure of the fourth candidate weight matrix with the largest corresponding coefficient among the plurality of fourth candidate weight matrices.

4. The method according to claim 1, wherein, The step of simulating the mapping of the model to be trained onto a dataflow architecture processor to obtain mapping information includes: The data processing procedure of the attention module in the model to be trained is broken down into multiple operations; The multiple operations are mapped to multiple execution units included in the dataflow architecture processor to obtain mapping information indicating the mapping relationship between the operations and the execution units.

5. The method according to claim 4, wherein, Determining the performance information of the model to be trained based on the mapping information includes: Based on the mapping information, the dependencies between the multiple operations, and the attribute information of the data flow architecture processor, the start and end times of the execution of the multiple operations are determined. Based on the start and end times of the multiple operations, the time taken for the model to be trained to perform the data processing task is determined. Based on the accuracy of the model to be trained in processing samples in the training dataset, accuracy information is obtained; Obtain the performance information, which includes the time taken by the model to be trained to perform the data processing task and the accuracy information.

6. The method according to claim 5, wherein, The plurality of operations includes at least one of computation operations, data transfer operations, and memory access operations, wherein the computation operations include matrix operations and / or vector operations.

7. The method according to claim 6, wherein, The step of determining the start and end times of the execution of the multiple operations based on the mapping information, the dependencies between the multiple operations, and the attribute information of the data flow architecture processor includes: Based on the order in which the multiple operations are executed, the start time of the first executed operation is determined to be 0, and the start time of the i-th executed operation is determined to be the end time of the (i-1)-th executed operation, where i is an integer greater than 1; If the operation is a computational operation, the end time of the operation is determined based on the structure of the candidate weight matrix included in the attention module of the model to be trained, the matrix partitioning rules, the attributes of the execution unit that performs the operation, and the start time of the operation. If the operation is a data transmission operation, the end time of the operation is determined based on the amount of data in the tensor to be transmitted, the transmission link and link bandwidth determined based on the mapping information, and the start time of the operation. If the operation is a memory access operation, the end time of the operation is determined based on the amount of data in the tensor to be accessed, the memory access bandwidth and memory access strategy determined based on the mapping information, and the start time of the operation.

8. The method according to claim 7, wherein, Determining the time taken for the model to be trained to perform the data processing task based on the start and end times of the multiple operations includes: Based on the order in which the multiple operations are executed, the end time of the last executed operation is determined as the time consumed by the model to be trained to perform the data processing task.

9. The method according to any one of claims 1-8, further comprising: The elements of the plurality of candidate weight matrices are adjusted using the training dataset, wherein the adjustment of the coefficients of the plurality of candidate weight matrices and the adjustment of the elements of the plurality of candidate weight matrices are performed alternately, while the elements of the candidate weight matrices remain unchanged when the coefficients of the plurality of candidate weight matrices are adjusted, and the coefficients of the candidate weight matrices remain unchanged when the elements of the plurality of candidate weight matrices are adjusted. After the performance information meets the termination condition, the candidate weight matrix with the largest corresponding coefficient among multiple candidate weight matrices corresponding to the same weight matrix is ​​determined as the weight matrix included in the attention module of the target model.

10. A pre-trained model, the pre-trained model comprising at least one attention module, the pre-trained model being trained by the method of any one of claims 1-9.

11. A model training device, comprising: The mapping unit is used to simulate mapping the model to be trained to the dataflow architecture processor and obtain mapping information. An analysis unit is used to determine the performance information of the model to be trained based on the mapping information, wherein the performance information is used to indicate the time consumption and accuracy of the model to be trained in performing data processing tasks; An adjustment unit is used to adjust the coefficients of multiple candidate weight matrices included in the attention module of the model to be trained according to the performance information until the performance information meets the termination condition, wherein the multiple candidate weight matrices corresponding to the same weight matrix have different structures. The determining unit is used to determine the structure of the weight matrix included in the attention module of the target model based on the coefficients of the plurality of candidate weight matrices.

12. An electronic device, comprising: The processor, memory, communication interface, and communication bus communicate with each other through the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to any one of the methods in claims 1-9.

13. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1-9.

14. A computer program product comprising computer instructions that instruct a computing device to perform the method of any one of claims 1-9.

Images