Operator fusion method, system, equipment and medium

Abstract

The invention provides an operator fusion method, system and device and a medium. The method comprises the steps of obtaining acceleration chip configuration information of a current hardware platform and operator type information contained in a computational graph; inputting the acquired acceleration chip configuration information, operator type information and the computational graph to a trained operator fusion model to output a target operator; wherein the operator fusion model adjusts low-rank weight adjustment parameters in the operator fusion model by loading a target first configuration file matched with the acceleration chip configuration information and a target second configuration file matched with the operator type information so as to adapt to a hardware platform and an operator type. By dynamically loading the configuration file related to the current hardware and operator type, the operator is automatically adjusted when facing algorithm and application demand changes and different acceleration chip architectures, and the performance consistency and reliability in different task scenes are ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an operator fusion method, system, device and medium. Background Art

[0002] In many complex application scenarios of modern high-performance computing, computational graphs are relied upon to manage and optimize the computing process. As a directed non-changing graph of computing tasks, computational graphs not only define the origin relationship between operators, but also directly affect the execution efficiency of computing tasks. At the same time, modern high-performance computing systems are gradually developing towards heterogeneous computing architectures, consisting of central processing units and a variety of dedicated acceleration chips. Various types of acceleration chips have significant differences in computing power and storage architecture, enabling heterogeneous computing systems to perform well in different types of tasks. Therefore, in heterogeneous computing systems, reasonable operator fusion in computational graphs is particularly important.

[0003] However, different acceleration chips have their own characteristics in hardware architecture, storage level, data transmission speed, and parallel computing capability. The same computational graph may require different operator fusion strategies on different acceleration chips. Therefore, it is necessary to tailor appropriate operator fusion strategies for different acceleration chips to maximize the computing capabilities of different acceleration chips.

[0004] Therefore, an operator fusion method is urgently needed to solve the above technical problems. Summary of the invention

[0005] Based on this, it is necessary to provide an operator fusion method to improve adaptability in response to the above technical problems.

[0006] In a first aspect, the present application provides an operator fusion method, the method comprising:

[0007] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0008] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0009] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0010] In some embodiments, the operator fusion model includes:

[0011] An input pre-processing module, used for encoding the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector;

[0012] A chip architecture adaptation module, used to generate a first configuration file during training and load a target first configuration file matching the acceleration chip configuration information during operation;

[0013] An operator type adaptation module, used to generate a second configuration file during training and load a target second configuration file matching the operator type information during operation;

[0014] A post-processing adaptation module, used for generating a fused coding vector according to a chip coding vector and an operator coding vector;

[0015] The operator generation module is used to generate a target operator according to the fused coding vector.

[0016] In some embodiments, encoding the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector includes:

[0017] The encoder is used to encode the input acceleration chip configuration information according to a first preset formula to generate a chip encoding vector, wherein the first preset formula is:

[0018] O arch =TE((I config ⊕E(I config ))⊕(I operator ⊕E(I operator ))),

[0019] Among them, I config Indicates the acceleration chip configuration information, E(I config ) represents the encoding vector corresponding to the acceleration chip, I operator Indicates operator type information, E(I operator ) represents the encoding vector corresponding to the operator type, TE represents the conversion operation, ⊕ represents the concatenation operation, O arch represents the chip encoding vector;

[0020] The encoder is used to encode the input operator type information according to a second preset formula to generate an operator encoding vector, where the second preset formula is:

[0021] O operator =TE(I operator ⊕E(I operator )),

[0022] Among them, O operator Represents the operator encoding vector.

[0023] In some embodiments, generating a fused coding vector according to the chip coding vector and the operator coding vector includes:

[0024] Concatenate the chip encoding vector and the operator encoding vector to generate a first concatenated vector;

[0025] Inputting the first concatenated vector into a multilayer perceptron to generate a transform coding vector;

[0026] Inputting the transformed coding vector into a preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector;

[0027] Performing weighted processing on the chip code vector according to a first weight, and performing weighted processing on the operator code vector according to a second weight;

[0028] The weighted chip code vector and the operator code vector are concatenated to generate a second concatenated vector;

[0029] The second concatenated vector is input into the multilayer perceptron to generate a fused encoding vector.

[0030] In some embodiments, the step of inputting the transformed coding vector into a preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector includes:

[0031] The selection network generates a first weight corresponding to the chip code vector based on a third preset formula, and the third preset formula is:

[0032]

[0033] Where A represents the network parameters, x represents the first concatenated vector of the input, and s1 represents the first weight;

[0034] The selection network generates a second weight corresponding to the operator encoding vector based on a fourth preset formula, and the fourth preset formula is:

[0035]

[0036] Among them, B represents the network parameter, and s2 represents the second weight.

[0037] In some embodiments, the method further includes a training method for the operator fusion model:

[0038] Pre-training the operator fusion model according to a large-scale original operator data set to ensure that the operator fusion model can output a fusion operator according to an input calculation graph;

[0039] Selecting a first operator data set that meets preset conditions according to preset parameters, and performing a second training on the pre-trained operator fusion model according to the first operator data set to optimize the operator fusion model;

[0040] After the optimized operator fusion model is verified, a second operator data set that meets the requirements of the adaptation scenario is selected according to different adaptation scenarios;

[0041] The optimized operator fusion model is trained for the third time according to the second operator data set to ensure that the operator fusion model is adaptively adjusted according to the adaptation scenario.

[0042] In some embodiments, before performing a third training on the optimized operator fusion model according to the second operator data set, the method includes:

[0043] Freezing all parameters of the operator fusion model and adding low-rank weight adjustment parameters in the chip architecture adaptation module and the operator adaptation module in the operator fusion model;

[0044] The performing a third training on the optimized operator fusion model according to the second operator data set further includes:

[0045] Saving a first low-rank weight adjustment parameter matching the adaptation scenario in the chip architecture adaptation module as a first configuration file;

[0046] The second low-rank weight adjustment parameter matching the adaptation scenario in the operator type adaptation module is saved as a second configuration file.

[0047] In a second aspect, the present application provides an operator fusion system, the system comprising:

[0048] The data preparation module is used to obtain the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0049] An operator fusion module, used to input the acquired acceleration chip configuration information, the operator type information and the calculation graph into the trained operator fusion model to output the target operator;

[0050] The operator fusion model adjusts the low-rank weight adjustment parameters within the operator fusion model to adapt to the hardware platform and operator type through a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0051] In a third aspect, the present application provides a computer program product, which, when executed by a processor, implements the steps of the following method:

[0052] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0053] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0054] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0055] In a fourth aspect, the present application provides an electronic device, the electronic device comprising:

[0056] one or more processors;

[0057] and a memory associated with one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the following operations:

[0058] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0059] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0060] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0061] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program enables a computer to perform the following operations:

[0062] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0063] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0064] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0065] The beneficial effects achieved by this application are:

[0066] The present application provides an operator fusion method, including obtaining the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph; inputting the obtained acceleration chip configuration information, the operator type information and the calculation graph into the trained operator fusion model to output the target operator; wherein the operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information. By dynamically loading configuration files related to the current hardware and operator type, it is possible to automatically adjust the operator implementation in the face of changes in algorithm and application requirements and different acceleration chip architectures, ensuring performance consistency and reliability in different task scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0068] Figure 1 is a schematic diagram of an operator fusion method provided in an embodiment of the present application;

[0069] Figure 2 This is an operator fusion model architecture diagram provided in an embodiment of the present application;

[0070] Figure 3 It is a schematic diagram of a coding processing method provided in an embodiment of the present application;

[0071] Figure 4 It is a schematic diagram of a fusion processing method provided in an embodiment of the present application;

[0072] Figure 5 This is an operator fusion system architecture diagram provided in an embodiment of the present application;

[0073] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0075] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like in the entire specification and claims should be interpreted as inclusive rather than exclusive or exhaustive; that is, the meaning of "including but not limited to".

[0076] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0077] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0078] As mentioned in the background technology, as modern high-performance computing systems gradually develop towards heterogeneous computing architectures, in order to improve the computing power of heterogeneous computing systems, it is necessary to reasonably fuse the operators in the computing graph. Operator fusion is an optimization technology that combines multiple adjacent operators into a larger operator to reduce the migration and exchange of data between different operators, thereby reducing the demand for memory bandwidth, increasing computing density, and further significantly improving computing speed. Therefore, a reasonable operator fusion strategy plays an important role in improving the efficiency of high-performance computing.

[0079] However, due to the different structures and performances of different acceleration chips, the same computational graph may require different operator fusion strategies on different acceleration chips. For example, some chips may be suitable for fusing more operators together to reduce memory access, while other chips may need to keep operators separate to fully utilize their parallel processing capabilities. Therefore, operator fusion strategies tailored for each acceleration chip can better unleash the computing potential of these chips and ultimately achieve a more efficient computing process. The operator fusion strategy in a heterogeneous computing system not only has a direct impact on the execution efficiency of the computational graph, but also effectively improves the computing speed and maximizes the hardware advantages of dedicated acceleration chips.

[0080] Existing operator fusion methods mainly include the following: graph pattern fusion, dynamic fusion, static fusion and hierarchical fusion. Among them, graph pattern fusion uses predefined computational graph patterns for optimization, reduces computational and data transmission overhead by identifying and fusing common patterns, is simple to implement and has significant effects, and is suitable for processing fixed and common graph structures; however, it is limited to the fixedness of the pattern library and cannot cope with all computational graph structures, has poor adaptability, and may not be able to effectively optimize complex or uncommon patterns. Among them, dynamic fusion performs operator fusion according to the actual computational graph situation at runtime, has high flexibility and adaptability, and can handle complex and dynamically changing computing tasks; it improves computing performance through real-time optimization, but also has high runtime overhead and high implementation complexity, which may affect the overall performance. Among them, static fusion optimizes by globally analyzing the computational graph at the compilation stage, reduces the computational and data transmission overhead at runtime, and can significantly improve performance; however, static fusion has poor adaptability to dynamically changing computational graphs, and the complex compilation process may cause the compilation time to be extended, which makes the effect limited when processing dynamic tasks. Among them, hierarchical fusion gradually optimizes the computational graph according to the hierarchical structure, fusing from low levels to high levels, systematically improving the computing efficiency, allowing optimization at different levels, and enhancing the overall performance; however, the implementation of hierarchical fusion is complex and requires effective hierarchical management, and the selection of different levels has a direct impact on the final optimization effect. Improper selection may reduce the effect of performance improvement.

[0081] Therefore, the present application discloses an adaptive operator fusion method, which automatically adjusts the operator implementation to adapt to the characteristics of various acceleration chips, including GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array) and TPU (Tensor Processing Unit), etc., to ensure the best performance on different platforms.

[0082] Embodiment 1

[0083] The present application embodiment provides an operator fusion method, specifically, as follows: Figure 1 As shown, the method disclosed in the embodiment of the present application is used to implement operator fusion of operators contained in the calculation graph, including the following contents:

[0084] S1. Obtain the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph.

[0085] In order to improve the adaptability of the same computational graph to different platforms, the application needs to collect the corresponding acceleration chip configuration information on the platform. The above-mentioned acceleration chip configuration information is configuration information related to the acceleration chip and affects the computing performance of the acceleration chip, including but not limited to hardware architecture information, storage hierarchy, data transmission characteristics, etc. The above-mentioned acceleration chip configuration information is provided by the hardware platform and can be obtained through query requests, etc. This application does not limit the specific method for obtaining the acceleration chip configuration information.

[0086] Among them, acceleration chips refer to different types of computing accelerators, such as GPU, TPU and FPGA, etc. The computational graph is a directed acyclic graph that represents the computing process, in which each represents an operation or operator (such as addition, multiplication, convolution, etc.), and each edge represents the flow of data between operations / operators. The computational graph is widely used in deep learning, automatic differentiation, and data computing. The above operator types have defined type attributes when the computational graph is constructed, and the type of the operator can be obtained through this attribute. Specifically, the acquisition of the operator type can be achieved through programming, which is a conventional technical means in this field and will not be expanded in this application.

[0087] S2. Input the acquired acceleration chip configuration information, operator type information, and calculation graph into the trained operator fusion model to output the target operator.

[0088] According to the output target operator, the computation graph is updated and the updated computation graph is executed on the hardware platform. The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading the target first configuration file matching the acceleration chip configuration information and the target second configuration file matching the operator type information.

[0089] This application uses a pre-trained operator fusion model to automatically adjust the operator implementation according to the acceleration chip configuration and operator type, ensuring that the calculation graph can achieve the best performance when executed on different platforms; by automatically adapting to hardware platform characteristics, optimizing performance, reducing development and maintenance costs, and responding to hardware evolution and demand changes, it ensures that the calculation graph achieves consistent performance and stability in a variety of computing environments.

[0090] Further, by optimizing the specific hardware architecture, the execution efficiency of the operator is effectively improved, the complexity and limitations of the traditional manual optimization method are overcome, and the operator can be executed more efficiently. In addition, by reducing the manual optimization and adjustment of each platform, the complexity and cost of development and maintenance are reduced. At the same time, it can quickly adapt to the update of acceleration chip technology, avoiding the challenge that traditional methods are difficult to keep up with hardware evolution. Further, the method disclosed in this application can also automatically adjust the operator implementation according to changes in algorithms and application requirements, ensuring that the system always maintains efficient adaptability in different scenarios, and ensuring the flexibility and scalability of the application. It ensures performance consistency on a variety of hardware platforms, improves the reliability and stability of cross-platform applications, and enables the system to maintain optimal performance in various computing environments.

[0091] In a specific embodiment, Figure 2 As shown in the model architecture diagram, the operator fusion model includes an input pre-processing module, which is used to encode the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector; a chip architecture adaptation module, which is used to generate a first configuration file during training, and load a target first configuration file that matches the acceleration chip configuration information during operation; an operator type adaptation module, which is used to generate a second configuration file during training, and load a target second configuration file that matches the operator type information during operation; a post-processing adaptation module, which is used to generate a fused encoding vector based on the chip encoding vector and the operator encoding vector; and an operator generation module, which is used to decode the fused encoding vector to generate a target operator. The operator fusion model ensures that the fused target operator finally output by the model can achieve optimal performance on different chip architectures and operator types through the mutual cooperation of the above five modules.

[0092] The above input pre-processing module is as follows Figure 3 As shown in the schematic diagram of the encoding processing method, the acceleration chip configuration information and operator type information in the input model are encoded.

[0093] Specifically, the encoder is used to encode the input acceleration chip configuration information according to a first preset formula to generate a chip encoding vector. The first preset formula is:

[0094] O arch =TE((I config ⊕E(I config ))⊕(I operator ⊕E(I ooperator ))), where I config Indicates the acceleration chip configuration information, E(I config ) represents the encoding vector corresponding to the acceleration chip, I operator Indicates operator type information, E(Ioperator ) represents the encoding vector corresponding to the operator type, TE represents the conversion operation, ⊕ represents the concatenation operation, O arch Represents the chip encoding vector.

[0095] The encoder is used to encode the input operator type information according to the second preset formula to generate an operator encoding vector. The second preset formula is:

[0096] O operator =TE(I operator ⊕E(I operator )), where O operator Represents the operator encoding vector.

[0097] It can be understood that in the encoding process disclosed in the embodiment of the present application, by adding marking information, that is, by uniformly adding the encoding vector E(I config ); E(I operator ) Ensure the independence between the chip encoding vector and the operator encoding vector to avoid deep coupling.

[0098] The chip architecture adaptation module is specifically responsible for adapting to a specific chip architecture to ensure that the system can run efficiently on different hardware platforms. The chip architecture adaptation module adopts a Transformer-based architecture and combines the LoRA (Low-Rank Adaptation) method to improve the adaptability and scalability of the operator fusion model. To further expand, in the training phase, the chip architecture adaptation module uses LoRA for refined training based on different selected adaptation scenarios (i.e., including different chip architectures), and saves the first configuration file of each architecture. The first configuration file contains the low-rank weight adjustment parameters required to adapt to a specific chip architecture, so that the entire model does not need to be retrained during training, and only a small part of the parameters need to be adjusted to complete the adaptation. Among them, Transformer is a revolutionary architecture design in the field of natural language processing (NLP). It abandons the traditional method based on recurrent neural networks (RNNs) and uses self-attention mechanisms instead, so that the model can process sequence data more efficiently. Among them, low-rank adaptation technology is a technology for fine-tuning based on a pre-trained model, which aims to adapt to new tasks or data sets by adjusting part of the model's parameters while trying to retain most of the knowledge of the pre-trained model. In the Transformer model, low-rank adaptation techniques can be used to reduce the computational resources and time cost required during fine-tuning while maintaining good performance.

[0099] During actual operation, the chip architecture adaptation module can dynamically load the corresponding target first configuration file according to the currently running chip architecture to achieve automatic adaptation of the chip architecture. Through this dynamic loading mechanism, the system can seamlessly switch between different hardware environments without the need to redesign or deploy the entire system every time the architecture changes. It greatly reduces the cost and time required to adapt to different hardware platforms and ensures that the best performance is always achieved on a variety of chip architectures. It further improves the flexibility of heterogeneous computing systems and enhances their ability to cope with changing hardware environments, ensuring the stability and efficiency of heterogeneous computing systems under various architectures.

[0100] The main functions of the above-mentioned operator type adaptation module are similar to those of the chip architecture adaptation module, but it focuses on handling adaptation tasks related to different operator types. The operator type adaptation module also adopts a Transformer-based architecture and combines low-rank adaptation technology to learn and optimize strategies for different operators. During the training process, the operator type adaptation module performs detailed learning for various operator types and saves the optimization strategy as a second configuration file; the second configuration file records the low-rank weight adjustment parameters corresponding to the specific operator type, so that these optimization strategies can be quickly loaded and applied in practical applications. At runtime, the operator type adaptation module can dynamically load the target second configuration file that matches the current operator type, thereby realizing automatic optimization of a specific operator type. The scalability of the operator type adaptation module is greatly enhanced, so that the target operator generated by the model can flexibly cope with a variety of computing tasks, ensuring that the best operator performance can be provided in each task. By combining the powerful modeling capabilities of the Transformer architecture and the efficient adaptability of the low-rank adaptation technology, the operator type adaptation module can achieve the most efficient performance with minimal computing overhead when facing different operator requirements. This dynamic adaptability ensures the system's flexibility and superiority in diverse computing environments, allowing it to always remain in optimal condition when handling a variety of complex tasks.

[0101] By using low-rank adaptation for fine-tuning and saving the configuration file locally, it effectively avoids repeated loading of redundant model parameters during inference, thus significantly alleviating the pressure on on-chip storage. This optimization method not only improves storage utilization, but also ensures the efficiency and stability of the system during runtime.

[0102] like Figure 4The schematic diagram of the fusion processing method shown in the figure, the above-mentioned post-processing adaptation module effectively fuses the chip coding vector and the operator coding vector that have been encoded in the input pre-processing module. Specifically, the chip coding vector and the operator coding vector are spliced ​​to generate a first splicing vector; the first splicing vector is input into the multilayer perceptron to generate a transformation coding vector; the transformation coding vector is input into the preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector; the chip coding vector is weighted according to the first weight, and the operator coding vector is weighted according to the second weight; the weighted chip coding vector and the operator coding vector are spliced ​​to generate a second splicing vector; the second splicing vector is input into the multilayer perceptron to generate a fusion coding vector. Among them, the multilayer perceptron (MultilayerPerceptron, referred to as MLP) is a part of the feed-forward neural network (Feed-ForwardNetwork), and its function is to perform nonlinear transformation on the input data to capture more complex feature representations.

[0103] The above-mentioned input of the transformation coding vector to the preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector includes: the selection network generates the first weight corresponding to the chip coding vector based on the third preset formula; and generates the second weight corresponding to the operator coding vector based on the fourth preset formula.

[0104] The third preset formula is:

[0105]

[0106] The fourth preset formula is:

[0107]

[0108] Among them, A represents the network parameters, B represents the network parameters, x represents the first splicing vector of the input, s1 represents the first weight, and s2 represents the second weight. Specifically, it can be understood that A and B can be understood as matrices and are learnable parameters. By performing learning optimization in the back propagation process, the selected network structure can better determine the weights of the chip encoding vector and the operator encoding vector during fusion, that is, determine whether the current fusion strategy should be more inclined to adapt the chip or the operator, and finally perform weighted summation of the chip encoding vector and the operator encoding vector to achieve information fusion. The post-processing adaptation module provides a unified and optimized input basis for the subsequent operator generation module, that is, the fusion encoding vector O fusion , ensuring that the target operator finally output by the operator fusion model can run efficiently in a variety of architectures and operator type environments.

[0109] Similarly, the above-mentioned operator generation module uses Transformer-Decoder as the skeleton, and based on the fused coding vector generated by the above-mentioned post-processing adaptation module, comprehensively considers various adaptation information and fusion rules, and finally generates the optimal operator implementation (i.e., target operator) suitable for different chip architectures and operator types. It can be understood that a vocabulary is maintained in the operator fusion model, in which the word segmentation method selects BPE, and the final output vector is a string of token ids. According to the token id, we need to query the corresponding token from the vocabulary, and splice the decoded token to get the final operator. The operator here is usually a section of cuda code. Since the features output by the post-processing module may not be suitable for direct decoding, the operator generation module here uses the characteristics of the transformer module to process the fused coding vector and finally decode it into an operator.

[0110] In a specific embodiment, the present application also discloses a training method for the above operator fusion model:

[0111] The operator fusion model is pre-trained based on a large-scale original operator data set to ensure that the operator fusion model can output fusion operators based on the input computational graph; specifically, the above pre-training process gives the model the general ability to generate operators from computational graphs. In the pre-training stage, the model learns basic operator generation rules and patterns through a large amount of diverse data. These rules and patterns provide a solid foundation for subsequent optimization, enabling the model to have a wide range of adaptability in different computing tasks. Among them, the operator data set includes at least the acceleration chip type, operator type, and computational graph.

[0112] A first operator data set that meets the preset conditions is selected according to the preset parameters, and a second training is performed on the pre-trained operator fusion model according to the first operator data set to optimize the operator fusion model. The preset parameters include but are not limited to computing memory access ratio, throughput, memory bandwidth utilization, branch efficiency, kernel function execution time, etc. The preset conditions are thresholds set by the staff in this field for different parameters according to the operation requirements. Each preset parameter exceeding the corresponding threshold indicates that the preset parameter meets the operation requirements, that is, meets the preset conditions.

[0113] After the optimized operator fusion model is verified, a second operator data set that meets the requirements of the adaptation scenario is selected according to different adaptation scenarios; the optimized operator fusion model is trained for the third time according to the second operator data set to ensure that the operator fusion model is adaptively adjusted according to the adaptation scenario. Among them, the adaptation scenario means a scenario that adapts to different acceleration chips and / or different operator types, and the second operator data set is an operator data set that has been running in the adaptation scenario, such as a data set for TPU chips, a data set suitable for GPUs, a reduce operator data set, and an element-by-element operation operator data set. The optimized operator fusion model is trained for the third time according to the second operator data set, and also includes: saving the first low-rank weight adjustment parameter that matches the adaptation scenario in the chip architecture adaptation module as a first configuration file; saving the second low-rank weight adjustment parameter that matches the adaptation scenario in the operator type adaptation module as a second configuration file. In this training phase, the parameters of all modules in the operator fusion model need to be frozen so that they are not updated during the training process. Low-rank adaptation technology is used to fine-tune specific adaptation modules (i.e., chip architecture adaptation module and operator type adaptation), and the parameters of LoRA are saved for loading in the subsequent reasoning process. For example, if you need to train a LoRA parameter suitable for a TPU chip, on the basis of freezing all model parameters, add LoRA parameters to the QKV matrix of the transformer structure of the chip architecture adaptation module, and only update the LoRA parameters during training. The same is true for training the LoRA of the operator type adaptation module. After that, we will save a series of different LoRA parameter configuration files, and dynamically load different LoRA configuration files (i.e., target first configuration file and target second configuration file) during the reasoning process. For example, if you need to generate a reduce type operator suitable for a TPU chip, during the reasoning process, the chip architecture adaptation module loads the target first configuration file corresponding to the TPU, and the operator type adaptation module loads the target second configuration file that matches the reduce operator type.

[0114] In order to further reduce storage pressure, the model size can also be compressed, specifically including the following: in response to detecting that the operator fusion model training is completed and verified, determine the tensors to be quantized for each layer of the network in the operator fusion model; calibrate the operator fusion model based on a preset verification data set to determine quantization parameters, such as scaling factors and zero points; quantize the weights and / or parameters of the model based on the quantization parameters obtained in the calibration phase; in response to detecting that the quantization is completed, test the above operator fusion model based on the verification data set; detect that the model performance has not significantly decreased, and derive the quantized operator fusion model. It is possible to quantize the parameters of the model from high precision (such as floating point 32 bits) to low precision (such as integer 8 bits or 4 bits), thereby reducing storage requirements and computational complexity, and also reduce the size of the model, thereby reducing the pressure on the model in storage. In addition, the model can also be compressed using methods such as weight pruning, knowledge distillation, and network architecture search.

[0115] Through the above content, the model is trained so that the operator fusion model can be flexibly responded to in a variety of computing environments, ensuring that the generated operators can play the best effect in any situation. This layer-by-layer optimization and dynamic adaptation method provides strong support for modern computing tasks, especially in the face of constantly updated acceleration chips and diversified operator requirements, showing great advantages.

[0116] 6) Relieve on-chip storage pressure: By using LoRA for fine-tuning and saving the configuration file locally, this method effectively avoids repeated loading of redundant model parameters during the inference process, thereby significantly alleviating the pressure on on-chip storage. This optimization method not only improves storage utilization, but also ensures the efficiency and stability of the system during operation.

[0117] Embodiment 2

[0118] Corresponding to the above embodiment, Figure 5 As shown, the embodiment of the present application also provides an operator fusion system, including:

[0119] The data preparation module 510 is used to obtain the acceleration chip configuration information of the current hardware platform and the operator type information included in the calculation graph;

[0120] An operator fusion module 520 is used to input the acquired acceleration chip configuration information, the operator type information and the calculation graph into the trained operator fusion model to output a target operator;

[0121] The operator fusion model adjusts the low-rank weight adjustment parameters within the operator fusion model to adapt to the hardware platform and operator type through a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0122] Embodiment 3

[0123] The present application also provides a computer program product, which implements the following steps when executed by a processor:

[0124] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0125] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0126] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0127] Among them, the operator fusion model includes:

[0128] An input pre-processing module, used for encoding the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector;

[0129] A chip architecture adaptation module, used to generate a first configuration file during training and load a target first configuration file matching the acceleration chip configuration information during operation;

[0130] An operator type adaptation module, used to generate a second configuration file during training and load a target second configuration file matching the operator type information during operation;

[0131] A post-processing adaptation module, used for generating a fused coding vector according to a chip coding vector and an operator coding vector;

[0132] The operator generation module is used to generate a target operator according to the fused coding vector.

[0133] In some implementation scenarios, when the computer program product is executed by a processor, the steps of the following method are also implemented:

[0134] The encoder is used to encode the input acceleration chip configuration information according to a first preset formula to generate a chip encoding vector, wherein the first preset formula is:

[0135] O arch =TE((I config ⊕E(I config ))⊕(I operator ⊕E(I operator ))),

[0136] Among them, I config Indicates the acceleration chip configuration information, E(I config ) represents the encoding vector corresponding to the acceleration chip, I operator Indicates operator type information, E(I operator ) represents the encoding vector corresponding to the operator type, TE represents the conversion operation, ⊕ represents the concatenation operation, O arch represents the chip encoding vector;

[0137] The encoder is used to encode the input operator type information according to a second preset formula to generate an operator encoding vector, where the second preset formula is:

[0138] O operator =TE(I operator ⊕E(I operator )),

[0139] Among them, O operator Represents the operator encoding vector.

[0140] In some implementation scenarios, when the computer program product is executed by a processor, the steps of the following method are also implemented:

[0141] Concatenate the chip encoding vector and the operator encoding vector to generate a first concatenated vector;

[0142] Inputting the first concatenated vector into a multilayer perceptron to generate a transform coding vector;

[0143] Inputting the transformed coding vector into a preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector;

[0144] Performing weighted processing on the chip code vector according to a first weight, and performing weighted processing on the operator code vector according to a second weight;

[0145] The weighted chip code vector and the operator code vector are concatenated to generate a second concatenated vector;

[0146] The second concatenated vector is input into the multilayer perceptron to generate a fused encoding vector.

[0147] In some implementation scenarios, when the computer program product is executed by a processor, the steps of the following method are also implemented:

[0148] The selection network generates a first weight corresponding to the chip code vector based on a third preset formula, and the third preset formula is:

[0149]

[0150] Where A represents the network parameters, x represents the first concatenated vector of the input, and s1 represents the first weight;

[0151] The selection network generates a second weight corresponding to the operator encoding vector based on a fourth preset formula, and the fourth preset formula is:

[0152]

[0153] Among them, B represents the network parameter, and s2 represents the second weight.

[0154] In some implementation scenarios, when the computer program product is executed by a processor, the steps of the following method are also implemented:

[0155] Pre-training the operator fusion model according to a large-scale original operator data set to ensure that the operator fusion model can output a fusion operator according to an input calculation graph;

[0156] Selecting a first operator data set that meets preset conditions according to preset parameters, and performing a second training on the pre-trained operator fusion model according to the first operator data set to optimize the operator fusion model;

[0157] After the optimized operator fusion model is verified, a second operator data set that meets the requirements of the adaptation scenario is selected according to different adaptation scenarios;

[0158] The optimized operator fusion model is trained for the third time according to the second operator data set to ensure that the operator fusion model is adaptively adjusted according to the adaptation scenario.

[0159] In some implementation scenarios, when the computer program product is executed by a processor, the steps of the following method are also implemented:

[0160] Freezing all parameters of the operator fusion model and adding low-rank weight adjustment parameters in the chip architecture adaptation module and the operator adaptation module in the operator fusion model;

[0161] The performing a third training on the optimized operator fusion model according to the second operator data set further includes:

[0162] Saving a first low-rank weight adjustment parameter matching the adaptation scenario in the chip architecture adaptation module as a first configuration file;

[0163] The second low-rank weight adjustment parameter matching the adaptation scenario in the operator type adaptation module is saved as a second configuration file.

[0164] Embodiment 4

[0165] Corresponding to all the above embodiments, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0166] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0167] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0168] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0169] in, Figure 6 The architecture of the electronic device is shown exemplarily, and may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, the video display adapter 611, the disk drive 612, the input / output interface 613, the network interface 614, and the memory 620 may be communicatively connected via a bus 630.

[0170] Among them, the processor 610 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in this application.

[0171] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store an operating system 621 for controlling the execution of the electronic device 600, and a basic input and output system (BIOS) 622 for controlling the low-level operations of the electronic device 600. In addition, a web browser 623, a data storage management system 624, and an icon font processing system 625, etc. can also be stored. The above-mentioned icon font processing system 625 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0172] The input / output interface 613 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0173] The network interface 614 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0174] The bus 630 comprises a pathway for transmitting information between the various components of the device (eg, the processor 610, the video display adapter 611, the disk drive 612, the input / output interface 613, the network interface 614, and the memory 620).

[0175] In addition, the electronic device 600 can also obtain information on specific collection conditions from the virtual resource object collection condition information database for use in condition judgment, etc.

[0176] It should be noted that, although the above device only shows a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, a memory 620, a bus 630, etc., in the specific implementation process, the device may also include other components necessary for normal execution. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0177] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for a computer device (which can be a personal computer, a cloud service end, or a network device, etc.) to execute the methods of various embodiments of the present application or certain parts of the embodiments.

[0178] Embodiment 5

[0179] Corresponding to all the above embodiments, the embodiments of the present application further provide a computer-readable storage medium storing a computer program, which enables a computer to perform the following operations:

[0180] Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph;

[0181] Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator;

[0182] The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0184] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An operator fusion method, characterized in that: The method comprises: Get the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph; Inputting the acquired acceleration chip configuration information, the operator type information, and the calculation graph into the trained operator fusion model to output the target operator; The operator fusion model adjusts the low-rank weight adjustment parameters in the operator fusion model to adapt to the hardware platform and operator type by loading a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

2. The method according to claim 1, characterized in that The operator fusion model includes: An input pre-processing module, used for encoding the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector; A chip architecture adaptation module, used to generate a first configuration file during training and load a target first configuration file matching the acceleration chip configuration information during operation; An operator type adaptation module, used to generate a second configuration file during training and load a target second configuration file matching the operator type information during operation; A post-processing adaptation module, used for generating a fused coding vector according to a chip coding vector and an operator coding vector; The operator generation module is used to generate a target operator according to the fused coding vector.

3. The method according to claim 2, characterized in that The step of encoding the input acceleration chip configuration information and operator type information to generate a chip encoding vector and an operator encoding vector includes: The encoder is used to encode the input acceleration chip configuration information according to a first preset formula to generate a chip encoding vector, wherein the first preset formula is: Among them, I config Indicates the acceleration chip configuration information, E(I config ) represents the encoding vector corresponding to the acceleration chip, I operator Indicates operator type information, E(I operator ) represents the encoding vector corresponding to the operator type, TE represents the conversion operation, represents the splicing operation, O arch represents the chip encoding vector; The encoder is used to encode the input operator type information according to a second preset formula to generate an operator encoding vector, where the second preset formula is: Among them, O operator Represents the operator encoding vector.

4. The method according to claim 3, characterized in that The generating of the fused coding vector according to the chip coding vector and the operator coding vector includes: Concatenate the chip encoding vector and the operator encoding vector to generate a first concatenated vector; Inputting the first concatenated vector into a multilayer perceptron to generate a transform coding vector; Inputting the transformed coding vector into a preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector; Performing weighted processing on the chip code vector according to a first weight, and performing weighted processing on the operator code vector according to a second weight; The weighted chip code vector and the operator code vector are concatenated to generate a second concatenated vector; The second concatenated vector is input into the multilayer perceptron to generate a fused encoding vector.

5. The method according to claim 4, characterized in that The step of inputting the transformed coding vector into a preset selection network to generate a first weight corresponding to the chip coding vector and a second weight corresponding to the operator coding vector includes: The selection network generates a first weight corresponding to the chip code vector based on a third preset formula, and the third preset formula is: Where A represents the network parameters, x represents the first concatenated vector of the input, and s1 represents the first weight; The selection network generates a second weight corresponding to the operator encoding vector based on a fourth preset formula, and the fourth preset formula is: Among them, B represents the network parameter, and s2 represents the second weight.

6. The method according to any one of claims 2 to 5, characterized in that: The method also includes a training method for the operator fusion model: Pre-training the operator fusion model according to a large-scale original operator data set to ensure that the operator fusion model can output a fusion operator according to an input calculation graph; Selecting a first operator data set that meets preset conditions according to preset parameters, and performing a second training on the pre-trained operator fusion model according to the first operator data set to optimize the operator fusion model; After the optimized operator fusion model is verified, a second operator data set that meets the requirements of the adaptation scenario is selected according to different adaptation scenarios; The optimized operator fusion model is trained for the third time according to the second operator data set to ensure that the operator fusion model is adaptively adjusted according to the adaptation scenario.

7. The method according to claim 6, characterized in that Before performing the third training on the optimized operator fusion model according to the second operator data set, the method includes: Freezing all parameters of the operator fusion model and adding low-rank weight adjustment parameters in the chip architecture adaptation module and the operator adaptation module in the operator fusion model; The performing a third training on the optimized operator fusion model according to the second operator data set further includes: Saving a first low-rank weight adjustment parameter matching the adaptation scenario in the chip architecture adaptation module as a first configuration file; The second low-rank weight adjustment parameter matching the adaptation scenario in the operator type adaptation module is saved as a second configuration file.

8. An operator fusion system, characterized in that: The system comprises: The data preparation module is used to obtain the acceleration chip configuration information of the current hardware platform and the operator type information contained in the calculation graph; An operator fusion module, used to input the acquired acceleration chip configuration information, the operator type information and the calculation graph into the trained operator fusion model to output the target operator; The operator fusion model adjusts the low-rank weight adjustment parameters within the operator fusion model to adapt to the hardware platform and operator type through a target first configuration file matching the acceleration chip configuration information and a target second configuration file matching the operator type information.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the method according to any one of claims 1-7 is executed.

10. A computer-readable storage medium, characterized in that: The computer program stores a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 7.

Images