Multi-element heterogeneous computing power adaptation method, device, equipment, medium and product

Abstract

The application relates to the technical field of computers and discloses a multi-element heterogeneous computing power adaptation method, device, equipment, medium and product, which comprises the following steps: adopting a framework unified interface to connect a calling interface of a target deep learning framework; registering operators related to a deep learning model by using the target deep learning framework, adopting a first operator unified interface to basicize the operators, and transmitting the basicized operator information to a second operator unified interface; adopting the second operator unified interface to connect an interface of an operator library, importing the basicized operator information into the operator library; and adapting and uniformly accessing multi-element heterogeneous artificial intelligence chips according to the operator information in the operator library. In this way, the framework and the computing power can be vertically connected, and then the rapid adaptation and uniform access of multiple different artificial intelligence chips can be realized, the compatibility of the target deep learning framework on different artificial intelligence chips is improved, the labor cost is saved, and the migration difficulty is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a multi-element heterogeneous computing power adaptation method, device, equipment, medium and product. BACKGROUND

[0002] With the explosive demand of high-throughput data processing applications, the artificial intelligence (AI) chip industry is developing rapidly. However, chip manufacturers usually need to invest a lot of manpower to adapt the current artificial intelligence chip to the mainstream artificial intelligence development framework, and there is a lack of compatibility of operators. This chimney type construction method causes waste of manpower in the industry and causes difficulty for users to migrate on different chips. Moreover, the development framework of the manufacturer also has the problem of not supporting new operators, and the algorithm is difficult to adapt, which needs the support of the manufacturer. SUMMARY

[0003] The purpose of the present application is to provide a multi-element heterogeneous computing power adaptation method, device, equipment, medium and product, which can realize the rapid adaptation and unified access of different artificial intelligence chips, and the target deep learning framework has strong compatibility on different chips, low migration difficulty and low cost.

[0004] In order to solve the above technical problems, the present application provides a multi-element heterogeneous computing power adaptation method, which comprises:

[0005] Adapting the calling interface of the target deep learning framework through a framework unified interface;

[0006] Registering the operators related to the deep learning model by using the target deep learning framework, and basing the operators by using a first operator unified interface, and transmitting the operator information after basing to a second operator unified interface;

[0007] Adapting the interface of the operator library by using the second operator unified interface, and importing the operator information after basing into the operator library;

[0008] According to the operator information in the operator library, the multi-element heterogeneous artificial intelligence chip is adapted and unified.

[0009] In the first aspect, in the multi-element heterogeneous computing power adaptation method provided by the present application, the operators related to the deep learning model are registered by using the target deep learning framework, and the operators are based by using a first operator unified interface, comprising:

[0010] Registering the operators related to the deep learning model by using the decoupling plug-in mode of the target deep learning framework itself;

[0011] The target deep learning framework is decoupled from the registered operator base by using a first operator unified interface, a virtual device is constructed and registered in the target deep learning framework;

[0012] Meanwhile, parameters corresponding to the target deep learning framework are multiplexed and converted to make the parameters uniform.

[0013] On the other hand, in the above-mentioned multi-element heterogeneous computing power adaptation method provided by the application, in the process of operator base, comprising:

[0014] According to the text information in the configuration file, a tree structure for representing a composite operator is constructed; the nodes of the tree structure contain base operators and operators;

[0015] The tree structure is parsed, the output value shape information of each node is automatically derived, and a temporary tensor storage area is created to obtain the base operator information.

[0016] On the other hand, in the above-mentioned multi-element heterogeneous computing power adaptation method provided by the application, the tree structure is parsed, the output value shape information of each node is automatically derived, and a temporary tensor storage area is created, comprising:

[0017] Each node in the tree structure is parsed;

[0018] The current node is operated, the output value shape information of the current node is automatically derived, and a temporary tensor storage area is created for the current node;

[0019] The base operators in all nodes are executed, and the recursion is performed layer by layer until the base operator information after base is obtained.

[0020] On the other hand, in the above-mentioned multi-element heterogeneous computing power adaptation method provided by the application, the operation of the current node comprises:

[0021] The current node is operated by a function for matrix operation;

[0022] And / or, the current node is operated by a corresponding activation function.

[0023] On the other hand, in the above-mentioned multi-element heterogeneous computing power adaptation method provided by the application, the interface of the operator library is connected by using the second operator unified interface, comprising:

[0024] An interface consistent with or similar to the interface of the deep neural network library is created as the second operator unified interface;

[0025] The second operator unified interface is connected with the interface of the operator library by using a vendor operator adapter.

[0026] In another aspect, in the multi-heterogeneous computing power adaptation method provided by the present application, after the imported basic operator information is imported into the operator library, the method further comprises:

[0027] The dynamic analysis is performed on the corresponding operator in a separate thread in the form of an insertion event, and the central processing unit time and device time consumed by the executed operator are counted respectively;

[0028] The analysis result and / or the statistical result are output to a display interface or a document.

[0029] In another aspect, in the multi-heterogeneous computing power adaptation method provided by the present application, after the multi-heterogeneous artificial intelligence chip is adapted and uniformly accessed, the method further comprises:

[0030] According to the target deep learning framework and the accessed multi-heterogeneous artificial intelligence chip, the deep learning model is post-pretrained, supervised fine-tuned and parameter fine-tuned.

[0031] In another aspect, in the multi-heterogeneous computing power adaptation method provided by the present application, in the process of supervised fine-tuning of the deep learning model, the method comprises:

[0032] The deep learning model is supervised fine-tuned in the form of a slicing strategy or using tensor parallel distributed training or using both tensor parallel and pipeline parallel distributed training;

[0033] In the process of parameter fine-tuning of the large model, the method comprises:

[0034] The deep learning model is parameter fine-tuned in the form of single-card training or using tensor parallel distributed training.

[0035] In another aspect, in the multi-heterogeneous computing power adaptation method provided by the present application, after the multi-heterogeneous artificial intelligence chip is adapted and uniformly accessed, the method further comprises:

[0036] The computing resource information of the multi-heterogeneous artificial intelligence chip is obtained;

[0037] The computing resource information of the multi-heterogeneous artificial intelligence chip is uniformly managed.

[0038] In another aspect, in the multi-heterogeneous computing power adaptation method provided by the present application, the computing resource information of the multi-heterogeneous artificial intelligence chip is uniformly managed, which comprises:

[0039] The computing resource information of the multi-heterogeneous artificial intelligence chip is memory cached and / or memory synchronized;

[0040] In the memory caching, according to the memory size of the application, the memory application is forwarded to different memory allocators, and according to the memory pointer of the incoming memory release function, the corresponding target memory allocator is determined to release the memory.

[0041] To solve the above technical problems, the application further provides a multi-element heterogeneous computing power adaptation device, which comprises:

[0042] The framework docking module is used for docking the calling interface of the target deep learning framework by using a framework unified interface.

[0043] The operator processing module is used for registering the operators related to the deep learning model by using the target deep learning framework, and performing the basicization of the operators by using a first operator unified interface, and transmitting the basicized operator information to a second operator unified interface.

[0044] The operator library docking module is used for docking the interface of the operator library by using the second operator unified interface, and importing the basicized operator information into the operator library.

[0045] The chip adaptation module is used for adapting and uniformly accessing the multi-element heterogeneous artificial intelligence chip according to the operator information in the operator library.

[0046] To solve the above technical problems, the application further provides a multi-element heterogeneous computing power adaptation device, which comprises:

[0047] The memory is used for storing the computer program.

[0048] The processor is used for executing the computer program to realize the steps of the multi-element heterogeneous computing power adaptation method.

[0049] To solve the above technical problems, the application further provides a non-volatile storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the multi-element heterogeneous computing power adaptation method.

[0050] To solve the above technical problems, the application further provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps of the multi-element heterogeneous computing power adaptation method.

[0051] From the above technical solution can be seen, the multi-element heterogeneous computing power adaptation method provided by the application, the method comprises: using a framework unified interface to connect the calling interface of the target deep learning framework;Using the target deep learning framework to register the operator related to the deep learning model, and using the first operator unified interface to base the operator, and transmitting the operator information after the base to the second operator unified interface;Using the second operator unified interface to connect the interface of the operator library, and importing the operator information after the base into the operator library;According to the operator information in the operator library, the multi-element heterogeneous artificial intelligence chip is adapted and uniformly accessed.

[0052] The beneficial effects of the application are that the multi-element heterogeneous computing power adaptation method provided by the application first performs framework connection, uses a framework unified interface to connect the calling interface of the target deep learning framework, uses the target deep learning framework to register the operator, then bases the operator, uses the operator unified interface to connect the interface of the operator library, and finally adapts and uniformly accesses the multi-element heterogeneous artificial intelligence chip based on the connected operator library, which can realize the vertical connection of the framework and the computing power, and further realize the rapid adaptation and uniform access of multiple different artificial intelligence chips. The whole method is easy to use, improves the compatibility of the target deep learning framework on different artificial intelligence chips, supports the target deep learning framework with the operator, does not need manufacturer support, saves manpower cost, reduces migration difficulty, and can better complete the migration work of the deep learning model training and reasoning code based on the target deep learning framework.

[0053] In addition, the application also provides a multi-element heterogeneous computing power adaptation device, a multi-element heterogeneous computing power adaptation equipment, a nonvolatile storage medium and a computer program product for the multi-element heterogeneous computing power adaptation method, which have the same or corresponding technical features as the multi-element heterogeneous computing power adaptation method mentioned above, and the effects are the same. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0055] Figure 1 The flow chart of the multi-element heterogeneous computing power adaptation method provided for the embodiments of the application is shown in the figure.

[0056] Figure 2 The framework schematic diagram corresponding to the multi-element heterogeneous computing power adaptation method provided for the embodiments of the application is shown in the figure.

[0057] Figure 3 The schematic diagram of the operator library encapsulation provided for the embodiments of the application is shown in the figure.

[0058] Figure 4 A structural schematic diagram of a multi-element heterogeneous computing power adaptation device provided for an embodiment of the present application is shown in FIG. 1.

[0059] Figure 5 A structural schematic diagram of a multi-element heterogeneous computing power adaptation device provided for an embodiment of the present application is shown in FIG. 1. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0061] The core of the present application is to provide a multi-element heterogeneous computing power adaptation method, device, equipment, medium and product to solve the technical problems of difficult algorithm adaptation between artificial intelligence chips and artificial intelligence development frameworks and high labor costs.

[0062] In order to enable personnel in the technical field to better understand the present application scheme, the present application will be further described in detail below with reference to the drawings and specific embodiments. Figure 1 A flowchart of a multi-element heterogeneous computing power adaptation method provided for an embodiment of the present application is shown in FIG. 2. Figure 1 As shown in the figure, the method comprises:

[0063] S101, adopting a framework unified interface to connect a calling interface of a target deep learning framework.

[0064] In actual application, a deep learning framework is a software toolkit for constructing and training a deep learning model. They provide a set of high-level application programming interfaces (Application Programming Interface, API) and tools, so that researchers and developers can more easily design, implement and optimize neural network models. Deep learning frameworks not only simplify the design and implementation process of models, but also provide rich pre-trained models and tools, so that developers can more efficiently conduct research and application development.

[0065] In actual application, mainstream deep learning frameworks can include TensorFlow (i.e., a symbolic mathematical system based on data flow programming, which is an open-source deep learning framework supporting distributed computing and capable of running on multiple hardware) and PyTorch (i.e., a Python machine learning library, which is an open-source deep learning framework with dynamic computation graph features and ease of use, supporting distributed training and multiple hardware acceleration).

[0066] In implementation, the target deep learning framework of the present application can select a domestic PaddlePaddle framework, that is, the calling interface of the PaddlePaddle framework is connected through the framework unified interface when step S101 is performed. PaddlePaddle is an open-source and industrial deep learning platform based on deep learning technology research and business application, integrating core framework, basic model library, end-to-end development kit, rich tool components, and Star River community. In this way, it can support domestic central processing units and domestic operating systems, help chip manufacturers and users to use the PaddlePaddle framework on multiple heterogeneous chips without feeling, help chip manufacturers and users to conveniently realize the migration of different open-source domestic large models, and specifically can complete the migration of the training and inference code of the domestic open-source large model based on the PaddlePaddle framework. Of course, the target deep learning framework of the present application can also use the above deep learning frameworks TensorFlow, PyTorch or other deep learning frameworks, which will not be described here.

[0067] S102, register the operators related to the deep learning model by using the target deep learning framework, and base the operators by using the first operator unified interface, and transmit the operator information after the basing to the second operator unified interface.

[0068] It should be noted that the operator is a basic operation or function for processing and optimizing a deep learning model (which can be a large model). With the increasing size and complexity of deep learning models, especially when training large models with billions of parameters, the design and implementation of operators become particularly important. Operator information can include model files, weight information, etc.

[0069] Considering that the operators related to the deep learning model have certain differences, the present application defines an operator unified interface, which aims to enable existing operators and models to be used in the framework and can call artificial intelligence chips for calculation.

[0070] When step S102 is performed, the operators related to the deep learning model can be registered by using the PaddlePaddle framework, and the operators are based by using the first operator unified interface (the operator unified interface of the local manufacturer), and the operator information after the basing is transmitted to the second operator unified interface (the operator unified interface of other manufacturers). The operator information after the basing can include registered operator information, composite operator information, and other new operator information, etc.

[0071] S103, connect the interface of the operator library by using the second operator unified interface, and import the operator information after the basing into the operator library.

[0072] It should be noted that the operator is the smallest unit of a deep learning model and algorithm, and adapting the operator means adapting the deep learning model, so the basic operator information needs to be imported into the operator library so that the deep learning model trained on hardware can run on different artificial intelligence chips. The operator library docking can realize the scheduling and distribution of different hardware operator libraries, reduce the complexity and manpower demand of chip operator library access.

[0073] S104, according to the operator information in the operator library, the multi-heterogeneous artificial intelligence chip is adapted and uniformly accessed.

[0074] It should be noted that the multi-heterogeneous artificial intelligence chip (Heterogeneous Multi-core Artificial Intelligence Chip) is an integrated circuit containing multiple different types of processing units. These processing units (or "cores") may have different architectures and functions, and are specifically optimized for performing specific types of tasks. This design method aims to fully utilize the advantages of various processing units to improve overall system performance and energy efficiency.

[0075] In implementation, the multi-heterogeneous artificial intelligence chip of the present application can be a domestic multi-heterogeneous artificial intelligence chip. The present application can widely and conveniently migrate the models or algorithms running on existing graphics processors to various domestic artificial intelligence chips, simplifying the application development adaptation for domestic artificial intelligence chips.

[0076] When performing step S104, multiple domestic multi-heterogeneous artificial intelligence chips can be adapted and uniformly accessed according to the operator information in the hardware operator library, and the subsequent Paddle framework can fully exert the high-efficiency distributed training and inference optimization capabilities of large models.

[0077] The above multi-heterogeneous computing adaptation method provided by the embodiment of the present application first performs framework docking, adopts a framework unified interface to dock the calling interface of the target deep learning framework, uses the target deep learning framework to register operators, then basicizes the operators, uses an operator unified interface to dock the interface of the operator library, and finally adapts and uniformly accesses the multi-heterogeneous artificial intelligence chip based on the docked operator library. This can realize the vertical connection of the framework and the computing power, and further realize the rapid adaptation and uniform access of multiple different artificial intelligence chips. The whole method is easy to use, improves the compatibility of the target deep learning framework on different artificial intelligence chips, supports the target deep learning framework with operators, does not require manufacturer support, saves manpower costs, reduces migration difficulty, and can better complete the migration work of deep learning model training and inference code based on the target deep learning framework.

[0078] It should be noted that, in view of the problems of the industry that the training and inference framework of the domestic open source large model is not unified, the adaptation and compatibility are poor, the domestic artificial intelligence acceleration chip support is insufficient, and the adaptation of foreign service framework is seriously dependent, the multi-element heterogeneous computing power adaptation method provided in the embodiment of the application can be understood as constructing a multi-element heterogeneous computing power execution framework, realizing upward docking of a target deep learning (such as PaddlePaddle) framework, downward docking of different domestic artificial intelligence acceleration chips, and supporting more than one mainstream open source large model, realizing a general performance optimization scheme, and solving the adaptation and migration problems of customers in various industries in artificial intelligence research and development deployment on multi-element heterogeneous chips.

[0079] Further, in the specific implementation, in the multi-element heterogeneous computing power adaptation method provided in the embodiment of the application, the step S102 registers the operators related to the deep learning model by using the target deep learning framework, and bases the operators by using the first operator unified interface, which specifically can include: registering the operators related to the deep learning model by using the decouplable plug-in mode of the target deep learning framework itself; decoupling the target deep learning framework and the registered operators by using the first operator unified interface, constructing a virtual device and registering the virtual device into the target deep learning framework; and simultaneously, reusing and converting the parameters corresponding to the target deep learning framework to make the parameters uniform.

[0080] In the implementation, the step S102 is a process of framework docking, realizes a unified registration mode of the target deep learning (such as PaddlePaddle) framework, and formulates a docking standard: registering to the front-end framework by combining the decouplable plug-in (customdevice) mode of the target deep learning framework itself, where the front-end framework refers to the software part interacting with the user, which can be a user interface (User Interface) or a command-line interface (Command-line Interface, CLI). The application does not intrude into the interface layer code for the user, and does not change the user's usage habit.

[0081] Figure 2 The framework schematic diagram corresponding to the multi-element heterogeneous computing power adaptation method provided in the embodiment of the application is shown in FIG. 1. Figure 2 As shown in FIG. 1, the target deep learning framework and the operators are decoupled by using the first operator unified interface, a virtual device is constructed and registered into each target deep learning framework, the operator conversion is implemented in the core program (Kernel), the operator registration function of the framework is reused, and the network tensor (Tensor) and other parameters of each framework are converted, so that the parameters of the framework are uniform.

[0082] Further, in the specific implementation, in the multi-element heterogeneous computing power adaptation method provided by the embodiment of the application, in the process of basisizing the operator, the step S102 can specifically include: constructing a tree structure for representing the composite operator according to the text information in the configuration file; the nodes of the tree structure contain the basic operator and the operator; parsing the tree structure, automatically deriving the output value shape information of each node and creating a temporary tensor storage area to obtain the operator information after basisization.

[0083] In the implementation, the application can develop operators for inference-side services and model requirements. The basisization of the operator can develop the operator to obtain a composite operator. The composite operator can be understood as a new operator or model constructed by multiple operators, for example, an add operator and a matmul operator. The two operators can constitute a new operator. The basisization of the operator is a composite operator implementation method based on a configuration file. The application can convert the text representation in the configuration file into a unified tree representation, implement the representation of the calculation processes of various composite operators as the same calculation process, specifically construct the composite operator as a tree structure, and the nodes of the tree structure contain the basic operator information and the operator. Then, the calculation function of the composite operator is completed by parsing and executing the tree structure. When performing the operation of each node, the output value shape information of the current node can be automatically derived and a temporary tensor can be created. As an extension means of operator development, the basisization of the operator module can reduce the development amount of the back-end code and quickly implement the calculation function of part of the composite operator based on the existing basic operator.

[0084] Further, in the specific implementation, in the multi-element heterogeneous computing power adaptation method provided by the embodiment of the application, the parsing of the tree structure, the automatic derivation of the output value shape information of each node, and the creation of the temporary tensor storage area can specifically include: traversing each node in the tree structure; performing the operation of the current node, automatically deriving the output value shape information of the current node, and creating a temporary tensor storage area for the current node; and executing the basic operators in all nodes, recursively, until the basisized operator information is obtained after all calculations are completed.

[0085] In the implementation, each node in the tree structure can be traversed from the beginning, a temporary tensor storage area can be automatically created for the node, and then the basic operator in the node can be executed, recursively, until the final result is obtained after all calculations are completed. Regardless of the calculation process of the composite operator, the parsing and execution can be completed by a set of codes, thereby reducing the system development difficulty.

[0086] Further, in the specific implementation, in the multi-element heterogeneous computing power adaptation method provided by the embodiment of the application, the operation of the current node can specifically include: performing the operation of the current node by using a function for matrix operation; and / or performing the operation of the current node by using a corresponding activation function.

[0087] In implementation, the current node can be operated on using the first formula through a function used for matrix operations; the first formula is:

[0088] Out1=β*input+α*(mat1@mat2); (1)

[0089] Where Out1 is the output matrix of the function used for matrix operations; input is the input element of the function used for matrix operations; mat1 and mat2 are two tensors for matrix multiplication; and α and β are weight coefficients. α and β can be set to a default value of 1.0.

[0090] Alternatively, the current node can be calculated using the second formula through the corresponding activation function; the second formula is:

[0091] Out2=0.5*x*(1+tanh(sqrt(2 / π)*(x+0.044715*x^3))); (2)

[0092] Where Out2 is the output of the activation function, x is the input value, tanh is the hyperbolic tangent function, sqrt is the square root function, and ^3 represents the cube.

[0093] In implementation, the mathematical expressions of the above formulas (1) and (2) are used to perform node operations, which can expand the scope of frame operator support, use basic operators to support complex operators, achieve rapid support for complex operators and new operators, and reduce dependence on chip operator libraries.

[0094] Furthermore, in specific implementation, in the above-mentioned multi-heterogeneous computing power adaptation method provided in the embodiments of the present invention, step S103 uses the second operator unified interface to connect with the interface of the operator library, which may specifically include: creating an interface that is consistent with or similar to the interface of the deep neural network library as the second operator unified interface; and using the manufacturer's operator adapter to connect the second operator unified interface with the interface of the operator library.

[0095] In implementation, such as Figure 2 As shown, an interface consistent with or similar to the CUDA Deep Neural Network library (CuDNN) is created as the unified interface for the second operator. This includes interface definition, return information, and input parameters. The aim is to minimize changes to user habits and facilitate use by both users and manufacturers. This unified interface can be the unified interface for operators from manufacturers with AI chips. When interfacing with operator libraries, the manufacturer's adapter can be used to design a unified interface for operator libraries from various manufacturers.

[0096] It should be noted that, since the parameter names and the parameter quantities of the operator library interfaces provided by various artificial intelligence chip manufacturers are often inconsistent, the application defines a manufacturer operator unified interface as a Target interface class, and unifies the data types and the operator library interfaces. The application can inherit the adapter class from the Target interface class, so as to facilitate data type conversion and eliminate a large amount of development adaptation work.

[0097] Figure 3 An encapsulation diagram of an operator library provided by an embodiment of the application is shown in FIG. 3. Figure 3 As shown in FIG. 3, the application encapsulates the second operator unified interface and the interface of the manufacturer operator adapter to obtain an operator library encapsulation, so that a third-party hardware operator library only needs to adapt to the interface after the encapsulation of the operator library, and through the standardization of the interface, code automatic generation of the adaptation process is realized. At this time, the manufacturer operator adapter can be directly connected to the interfaces of various artificial intelligence chips, such as the interface of the first artificial intelligence chip, the interface of the second artificial intelligence chip, and the interface of the third artificial intelligence chip, as shown in FIG. 4. Figure 3

[0098] Further, in specific implementation, in the above-mentioned multi-element heterogeneous computing power adaptation method provided by the embodiment of the application, after the step S103 of importing the operator information after the basis into the operator library is performed, the method can further include: performing dynamic analysis on the corresponding operator in a separate thread in the form of an insertion event, and respectively counting the central processing unit time consumption and the device time consumption of the executed operator; and outputting the analysis result and / or the statistical result to a display interface or a document.

[0099] In implementation, the application can perform performance dynamic analysis (Profiling) at the back end, and is applicable to the performance test and statistics of the operator level of all interfaced chips, so that the calculation performance bottleneck can be quickly counted and identified, and the performance optimization of the operator level is assisted. Specifically, a dynamic analysis function is added at the back end operator call, and through the form of an insertion event, the central processing unit time consumption and the device time consumption of the executed operator are respectively counted in a separate thread; wherein the operator call times, the maximum time consumption, the minimum time consumption, and the like can be counted, and the calculation performance of the operator, such as addition, subtraction, multiplication, division, matrix multiplication, matrix transposition, matrix transformation, and the like, can be tested; and the model training efficiency and the model inference efficiency of a complex operator (model algorithm). After execution, the data is output to the screen and the document.

[0100] ​Further, in specific implementation, after the step S104 of adapting and uniformly accessing the multi-heterogeneous artificial intelligence chips, the multi-heterogeneous computing power adaptation method provided by the embodiment of the present application can further include: performing post-pretraining (PostPretrain), supervised fine-tuning (Supervised fine-tuning, SFT) and parameter fine-tuning (Low-Rank Adaptation, LoRA) on the deep learning model according to the target deep learning framework and the accessed multi-heterogeneous artificial intelligence chips. In this way, on the premise of ensuring the model accuracy alignment, the user can conveniently and efficiently complete the construction of the post-pretraining, supervised fine-tuning and parameter fine-tuning of the deep learning model based on the PaddlePaddle framework and the multi-heterogeneous chips.

[0101] In implementation, in the process of performing supervised fine-tuning on the deep learning model, the supervised fine-tuning can be performed on the deep learning model in a way of using a sharding strategy or using tensor parallel distributed training or using both tensor parallel and pipeline parallel distributed training. In addition, in the process of performing parameter fine-tuning on the large model, the parameter fine-tuning can be performed on the deep learning model in a way of single-card training or using tensor parallel distributed training.

[0102] Further, in specific implementation, after the step S104 of adapting and uniformly accessing the multi-heterogeneous artificial intelligence chips, the multi-heterogeneous computing power adaptation method provided by the embodiment of the present application can further include: obtaining the computing resource information of the multi-heterogeneous artificial intelligence chips; and uniformly managing the computing resource information of the multi-heterogeneous artificial intelligence chips.

[0103] In implementation, the computing resource information can include chip quantity, chip state and other information. The present application can extract the general public part, i.e., the computing resource information of the multi-heterogeneous artificial intelligence chips, uniformly manage the back-end artificial intelligence chip resource cache, provide efficient memory application, release and synchronous management and optimization capabilities of computing resources, and thus realize the support for all chips, reduce repeated development and improve efficiency. In addition, the present application can evaluate the actual performance of the domestic chip business around the benchmark algorithm, the performance optimization scheme of the model and the practical cases.

[0104] Further, in specific implementation, the step of uniformly managing the computing resource information of the multi-heterogeneous artificial intelligence chips can specifically include: memory caching and / or memory synchronization of the computing resource information of the multi-heterogeneous artificial intelligence chips; wherein, in memory caching, the current memory application is forwarded to different memory allocators according to the size of the applied memory, and the corresponding target memory allocator is determined according to the memory pointer of the incoming memory release function to release the memory.

[0105] In implementation, the application can utilize a resource management module to uniformly manage the computing resource information of the multi-element heterogeneous artificial intelligence chip. The main functions of the resource management module include: resource caching, resource synchronization, resource Out Of Memory (OOM) prevention, and resource usage reporting. For resource caching, different memory caching schemes can be adopted according to different sizes of memory, further improving the performance of memory application and release. The resource management module can include a memory application and release forwarder and a memory allocator. The memory application and release forwarder can forward the current memory application to different memory allocators according to the size of the applied memory. Correspondingly, the memory application and release forwarder can determine which memory allocator to release the memory according to the memory pointer of the incoming memory release function. The memory allocator is responsible for specific memory application and release operations. Different memory allocators can adopt different strategies for applications of different sizes.

[0106] It should be noted that the resource management module supports three working modes, which can be selected by the user according to the characteristics and requirements of the model: the first mode is the default mode, which performs both memory caching and memory synchronization operations; the second mode is the memory caching mode, which only performs memory caching operations and does not perform memory synchronization operations; and the third mode is the memory synchronization mode, which only performs memory synchronization operations and does not perform memory caching operations.

[0107] The resource management module of the application has scalability, supports device memory, host memory, and other types, and can add other types as needed. For operators with more time-consuming, the core program of the operator can be fused and optimized to improve the overall execution performance.

[0108] In practical applications, the multi-element heterogeneous computing power adaptation method of the application can be combined with the joint solution of domestic servers, chip manufacturers, platform manufacturers, framework suppliers, and integrators to build and productize strategies, and widely marketized, and in the promotion process, according to the feedback of terminal customers, integrators, and partners on the development of large model business, framework technology evolution, etc. to upgrade the technology and optimize the capability.

[0109] In the above embodiments, the multi-element heterogeneous computing power adaptation method is described in detail, and the application also provides corresponding embodiments of a multi-element heterogeneous computing power adaptation device and a multi-element heterogeneous computing power adaptation equipment. It should be noted that the application describes the embodiments of the device from two angles: one is based on the functional module, and the other is based on the hardware.

[0110] Figure 4 The structure diagram of the multi-element heterogeneous computing power adaptation device provided by the embodiment of the application. The embodiment is based on the functional module, and the device includes:

[0111] The framework docking module 10 is used for docking a calling interface of a target deep learning framework by using a framework unified interface.

[0112] The operator processing module 11 is used for registering an operator related to a deep learning model by using the target deep learning framework, and basing the operator by using a first operator unified interface, and transmitting the based operator information to a second operator unified interface.

[0113] The operator library docking module 12 is used for docking an interface of an operator library by using the second operator unified interface, and importing the based operator information into the operator library.

[0114] The chip adaptation module 13 is used for adapting and uniformly accessing a multi-heterogeneous artificial intelligence chip according to the operator information in the operator library.

[0115] In the above multi-heterogeneous computing power adaptation device provided by the embodiment of the application, through the interaction of the above four modules, the framework is docked first, the calling interface of the target deep learning framework is docked by using the framework unified interface, the operator is registered by using the target deep learning framework, then the operator is based, the interface of the operator library is docked by using the operator unified interface, and finally the multi-heterogeneous artificial intelligence chip is adapted and uniformly accessed based on the docked operator library, so that the vertical connection of the framework and the computing power is realized, and then the rapid adaptation and uniform access of multiple different artificial intelligence chips are realized, the compatibility of the target deep learning framework on different artificial intelligence chips is improved, the operator supports the target deep learning framework, without the support of manufacturers, the manpower cost is saved, the migration difficulty is reduced, and the migration work of the deep learning model training and inference code based on the target deep learning framework can be better completed.

[0116] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described with reference to the description of the embodiments of the method part, which is not described here. And has the same beneficial effects as the above-mentioned multi-heterogeneous computing power adaptation method.

[0117] Further, in the specific implementation, in the above multi-heterogeneous computing power adaptation device provided by the embodiment of the application, the operator processing module 11 can specifically include:

[0118] The operator registration unit is used for registering an operator related to a deep learning model by using a decoupling plug-in mode of the target deep learning framework itself.

[0119] The operator basing unit is used for decoupling the target deep learning framework and the registered operator by using the first operator unified interface, constructing a virtual device and registering the virtual device into the target deep learning framework, and simultaneously reusing and converting parameters corresponding to the target deep learning framework to make the parameters uniform.

[0120] The operator base unit can be specifically configured to construct a tree structure for representing a composite operator according to text information in the configuration file; a node of the tree structure contains a base operator and an operator; the tree structure is parsed to automatically derive output value shape information of each node and create a temporary tensor storage area to obtain base operator information after base.

[0121] Further, in the specific implementation, in the multi-heterogeneous computing power adaptation device provided by the embodiment of the application, the operator library docking module 12 can be specifically configured to create an interface consistent with or similar to the interface of the deep neural network library as the second operator unified interface; and the second operator unified interface is docked with the interface of the operator library by using the vendor operator adapter.

[0122] Further, in the specific implementation, in the multi-heterogeneous computing power adaptation device provided by the embodiment of the application, the multi-heterogeneous computing power adaptation device can further include:

[0123] The dynamic analysis module is configured to perform dynamic analysis on the corresponding operator in a separate thread in the form of an insertion event, and count the central processing unit time consumption and device time consumption of the executed operator respectively; and output the analysis result and / or the statistical result to a display interface or a document.

[0124] Further, in the specific implementation, in the multi-heterogeneous computing power adaptation device provided by the embodiment of the application, the multi-heterogeneous computing power adaptation device can further include:

[0125] The model training module is configured to perform post-training, supervision fine-tuning and parameter fine-tuning on the deep learning model according to the target deep learning framework and the accessed multi-heterogeneous artificial intelligence chip.

[0126] Further, in the specific implementation, in the multi-heterogeneous computing power adaptation device provided by the embodiment of the application, the multi-heterogeneous computing power adaptation device can further include:

[0127] The resource management module is configured to acquire the computing resource information of the multi-heterogeneous artificial intelligence chip; and uniformly manage the computing resource information of the multi-heterogeneous artificial intelligence chip.

[0128] The resource management module can specifically include:

[0129] The memory processing unit is configured to perform memory caching and / or memory synchronization on the computing resource information of the multi-heterogeneous artificial intelligence chip; wherein, in the memory caching, the current memory application is forwarded to different memory allocators according to the memory size of the application, and the corresponding target memory allocator is determined according to the memory pointer of the incoming memory release function to release the memory.

[0130] Figure 5 The structure diagram of the multi-heterogeneous computing power adaptation device provided by the embodiment of the application. The embodiment is based on the hardware angle, likeFigure 5 As shown, the multi-heterogeneous computing power adaptation device comprises:

[0131] a memory 20 for storing a computer program;

[0132] a processor 21 for implementing the steps of the multi-heterogeneous computing power adaptation method as mentioned in the above embodiments when executing the computer program.

[0133] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA). The processor 21 can also include a main processor and a coprocessor, the main processor being a processor for processing data in a wake-up state, also known as a CPU; the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a Graphics Processing Unit (GPU) for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 21 can also include an Artificial Intelligence (AI) processor for processing machine learning-related computing operations.

[0134] The memory 20 can include one or more non-volatile storage media, which can be non-transitory. The memory 20 can also include a high-speed random access memory, and a non-volatile memory such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein the computer program is loaded and executed by the processor 21, and can implement the related steps of the multi-heterogeneous computing power adaptation method disclosed in any of the preceding embodiments. In addition, the resources stored by the memory 20 can also include an operating system 202 and data 203, etc., and the storage mode can be temporary storage or permanent storage. The operating system 202 can include Windows, Unix, Linux, etc. The data 203 can include but is not limited to the data involved in the multi-heterogeneous computing power adaptation method mentioned above, etc.

[0135] In some embodiments, the multi-heterogeneous computing power adaptation device can also include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26. Those skilled in the art can understand that,Figure 5 The structure shown in the figure does not constitute a limitation on the multi-element heterogeneous computing power adaptation device, and can include more or fewer components than the figure. The multi-element heterogeneous computing power adaptation device provided by the embodiment of the application includes a memory and a processor. When the processor executes the program stored in the memory, the following method can be implemented: a multi-element heterogeneous computing power adaptation method, and the effects are the same as above.

[0136] Finally, the application also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program. When the computer program is executed by a processor, the steps described in the above method embodiment are implemented.

[0137] It can be understood that if the method in the above embodiment is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and executes all or part of the steps of the method described in each embodiment of the application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes. The non-volatile storage medium provided by the application can implement the multi-element heterogeneous computing power adaptation method mentioned above, and the effects are the same as above.

[0138] Finally, the application also provides an embodiment corresponding to a computer program product. The computer program product includes computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps described in the above multi-element heterogeneous computing power adaptation method embodiment are implemented. The computer program product provided by the application can implement the multi-element heterogeneous computing power adaptation method mentioned above, and the effects are the same as above.

[0139] It should also be noted that in this specification, relationship terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0140] The multi-element heterogeneous computing power adaptation method, device, equipment, medium and product provided by the application are described in detail above. Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts are described in the method part. It should be pointed out that for ordinary technical personnel in the technical field, without departing from the principle of the application, the application can be improved and modified, and these improvements and modifications also fall within the protection scope of the application.

Claims

1. A multi-element heterogeneous computing power adaptation method, characterized in that, The method comprises: adopting a framework unified interface to connect a calling interface of a target deep learning framework; registering operators related to a deep learning model by using a decouplable plug-in mode of the target deep learning framework itself, and decoupling the target deep learning framework and the registered operators by using a first operator unified interface to construct a virtual device and register it into the target deep learning framework; meanwhile, reusing and converting parameters corresponding to the target deep learning framework to unify the parameters; in the process of basifying the operators, comprising: constructing a tree structure for representing a composite operator according to text information in a configuration file; the nodes of the tree structure contain basic operators and operators; traversing and analyzing each node in the tree structure; performing operation on the current node by using a first formula through a function for matrix operation; and / or performing operation on the current node by using a second formula through a corresponding activation function, automatically deriving output value shape information of the current node and creating a temporary tensor storage area for the current node; executing the basic operators in all nodes, recursively layer by layer, until the basified operator information is obtained after all calculations are completed; The first formula is: ; wherein Out1 is a result matrix of a function output for matrix operation; input is an input element of a function for matrix operation, mat1 and mat2 are two tensors for matrix multiplication, and a and β are weight coefficients; The second formula is: ; wherein Out2 is an output result of an activation function, x is an input value, tanh is a hyperbolic tangent function, sqrt is a square root function, represents a cube; transmitting the basified operator information to a second operator unified interface; adopting the second operator unified interface to connect an interface of an operator library, and importing the basified operator information into the operator library; adapting and uniformly accessing a multi-element heterogeneous artificial intelligence chip according to the operator information in the operator library.

2. The multi-element heterogeneous computing power adaptation method of claim 1, wherein, adopting the second operator unified interface to connect an interface of an operator library, comprising: creating an interface consistent with or similar to a deep neural network library interface as a second operator unified interface; using a vendor operator adapter to connect the second operator unified interface with the interface of the operator library.

3. The multi-element heterogeneous computing power adaptation method of claim 1, wherein, after importing the basified operator information into the operator library, further comprising: performing dynamic analysis on the corresponding operators in a separate thread by using an insertion event, and respectively counting central processing unit time consumption and device time consumption of the executed operators; outputting the analysis results and / or the statistical results to a display interface or a document.

4. The multi-element heterogeneous computing power adaptation method of claim 1, wherein, after adapting and uniformly accessing the multi-element heterogeneous artificial intelligence chip, further comprising: performing post-training, supervised fine-tuning and parameter fine-tuning on the deep learning model according to the target deep learning framework and the accessed multi-element heterogeneous artificial intelligence chip.

5. The multi-element heterogeneous computing power adaptation method according to claim 4, characterized in that, in the process of performing supervised fine-tuning on the deep learning model, comprising: performing supervised fine-tuning on the deep learning model by using a sharding strategy or using tensor parallel distributed training, or using both tensor parallel and pipeline parallel distributed training; in the process of performing parameter fine-tuning on a large model, comprising: performing parameter fine-tuning on the deep learning model by using single-card training or using tensor parallel distributed training.

6. The multi-element heterogeneous computing power adaptation method of claim 1, wherein, after adapting and uniformly accessing the multi-element heterogeneous artificial intelligence chip, further comprising: obtaining computing resource information of the multi-element heterogeneous artificial intelligence chip; uniformly managing the computing resource information of the multi-element heterogeneous artificial intelligence chip.

7. The multi-element heterogeneous computing power adaptation method of claim 6, wherein, uniformly managing the computing resource information of the multi-element heterogeneous artificial intelligence chip, comprising: The computing resource information of the multi-element heterogeneous artificial intelligence chip is cached and / or synchronized in memory; In the memory caching, according to the memory size of the application, the memory application of this time is forwarded to different memory allocators, and according to the memory pointer of the incoming memory release function, the corresponding target memory allocator is determined to release the memory.

8. A multi-element heterogeneous computing power adaptation apparatus, characterized in that, The device comprises: a framework docking module for docking the calling interface of the target deep learning framework by using a framework unified interface; An operator processing module is configured to register an operator related to a deep learning model by using a decoupling plug-in manner of the target deep learning framework itself, to decouple the target deep learning framework and the registered operator by using a first operator uniform interface, to construct a virtual device and register the virtual device into the target deep learning framework, and to reuse and convert parameters corresponding to the target deep learning framework to make the parameters uniform. In the process of operator basification, a tree structure used to represent a composite operator is constructed according to text information in a configuration file. A node of the tree structure includes a basic operator and an operator. Each node in the tree structure is parsed by traversal. A current node is operated by a function for matrix operation by using a first formula, and / or the current node is operated by a corresponding activation function by using a second formula. An output value shape information of the current node is automatically derived, and a temporary tensor storage area for the current node is created. The basic operators in all nodes are executed by layer-by-layer recursion until the basified operator information is obtained by completing all calculations. The first formula is: Out1 = alpha * mat1 + beta * mat2; wherein Out1 is a result matrix output by the function for matrix operation, input is an input element of the function for matrix operation, mat1 and mat2 are two tensors for matrix multiplication, and alpha and beta are weight coefficients. The second formula is: Out2 = sqrt(x^3 + 3 * x^2 + 2 * x + 1); wherein Out2 is an output result of the activation function, x is an input value, tanh is a hyperbolic tangent function, sqrt is a square root function, and represents a cube. The basified operator information is transmitted to a second operator uniform interface.​​ an operator library docking module for docking the interface of the operator library by using the second operator unified interface, and importing the basic operator information into the operator library; a chip adaptation module for adapting and uniformly accessing the multi-element heterogeneous artificial intelligence chip according to the operator information in the operator library.

9. A multi-variate heterogeneous compute power adaptation device, comprising: The device comprises: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the multi-element heterogeneous computing power adaptation method according to any one of claims 1 to 7.

10. A non-volatile storage medium, comprising: The computer program is stored on the non-volatile storage medium, and the computer program is executed by the processor to implement the steps of the multi-element heterogeneous computing power adaptation method according to any one of claims 1 to 7.

11. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the multi-element heterogeneous computing power adaptation method according to any one of claims 1 to 7.

Images