AI computing power center construction method

Abstract

The invention discloses an AI computing power center construction method, which comprises the following steps of: 1, constructing an AI computing power enabling subsystem which is used for developing and training an AI algorithm model for an AI computing cluster subsystem, enabling hardware equipment and developers; step 2, constructing an AI large model for a user to do an AI model training task; and step 3, constructing an operation system, and aggregating an AI computing power subsystem, an AI computing power enabling subsystem, an AI computing power service platform, an AI cluster network and security subsystem, a basic platform service, a security protection service and an AI cluster software service through the operation system to form a client-oriented entrance portal. According to the AI computing power center construction method, through setting of the first step, the second step and the third step, construction of the AI computing power center can be simply and effectively achieved, and the constructed AI computing power center is accurate and rapid.

Description

Technical Field

[0001] The present invention relates to a construction method, and more specifically to a construction method for an AI computing power center. Background Art

[0002] The construction of the artificial intelligence computing power center platform is a miniaturized artificial intelligence computing power center built for the company's own use and leasing by upstream and downstream enterprises in the business chain to meet the computing power needs of 3D digital asset creation. Its construction content includes artificial intelligence computing power platform and AI technical service platform. The artificial intelligence computing power platform mainly purchases and installs hardware such as servers to provide basic computing power for the entire project; the AI ​​technical service platform mainly integrates and develops software suitable for 3D digital asset creation, and provides services to customers by calling the computing power of the artificial intelligence computing power platform hardware. The computing power platform and the technical service platform are complementary and together constitute the artificial intelligence computing power center.

[0003] The construction of the current AI computing power center mainly includes the construction of hardware platforms and software platforms. There is no need to say much about the construction of the hardware platform. You only need to connect the required hardware and then turn on the power. The core lies in the construction of the software platform. The computing power of the computing power center is allocated through the constructed software platform to better serve the enterprise. However, there is currently no more suitable software platform construction method to accurately and quickly realize the construction of the AI ​​computing power center. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for constructing an AI computing power center that can accurately and quickly realize the construction of an AI computing power center.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for constructing an AI computing center, characterized in that it comprises the following steps:

[0006] Step 1: Build an AI computing power enabling subsystem for the AI ​​computing cluster subsystem and enabling hardware devices and developers for the development and training of AI algorithm models.

[0007] Step 2: Build a large AI model for users to perform AI model training tasks;

[0008] Step three: Build an operation system, which aggregates the AI ​​computing power subsystem, AI computing power enabling subsystem, AI computing power service platform, AI cluster network and security subsystem, basic platform services, security protection services and AI cluster software services to form a customer-oriented entry portal.

[0009] As a further improvement of the present invention, the AI ​​computing power enabling subsystem constructed in step 1 includes: heterogeneous computing architecture software for supporting the development of custom operators by developers;

[0010] AI framework software is used to provide step 2 for building a large AI model.

[0011] As a further improvement of the present invention, the heterogeneous computing architecture software includes:

[0012] The unified programming interface Ascend computing language library is used to provide unified and standardized interface components, including device management, context management, stream management, memory management, model loading and execution, operator loading and execution, media data processing, and graph management API libraries, providing users with programming interfaces that enable the capabilities of each component for users to develop deep neural network applications;

[0013] The graph optimization and compilation module is used to provide a unified IR interface for different front-ends, support the parsing / optimization / compilation of the computational graphs expressed by TensorFlow / Caffe / MindSpore, and provide the ability to optimize the deployment of the back-end computing engine to give full play to the performance of the device;

[0014] The computing execution engine is responsible for the scheduling and execution of various types of operators in the neural network, memory and other resources, communication management, etc., and outputs the computing results;

[0015] Operator compilation module and operator library, used to provide Ascend operator development and compilation capabilities, enabling chip capabilities; computing library, used to provide computing libraries such as neural networks, basic linear algebra, collective communication, and media data preprocessing, to support users to use and build AI models;

[0016] The computing tuning engine includes two plug-ins: the AutoTune plug-in, which supports both single operator and whole-graph tuning modes. It can automatically generate a knowledge base based on the model and data features, and apply it to the actual deployment process to obtain high performance. The model compression plug-in provides model quantization and sparsity capabilities in the form of a framework plug-in, supports Calibration and ReTrain, and realizes model miniaturization.

[0017] As a further improvement of the present invention, the graph optimization and compilation module includes:

[0018] GE module: As the control center for graph compilation and operation, the GE module provides graph operation environment management, graph execution engine management, operator library management, subgraph optimization management, graph operation management, and graph execution control;

[0019] FE module: This module is used to manage the fusion rules of operators, match IRGraph and perform operator fusion operations according to the fusion rules, manage operators supported by the system, maintain the operator list, and support custom operators;

[0020] AICPUE module, which is responsible for AICPU operator information management;

[0021] HCCL module: This module is responsible for the operator information management of HCCL, and is used to provide HCCL to implement the gradient aggregation function of all workers involved in parallel computing, and provide a data parallel solution for Ascend multi-machine and multi-card training.

[0022] As a further improvement of the present invention, the calculation library includes:

[0023] Neural network library, used as the Ascend neural network acceleration library, with rich built-in operators to support neural network training and reasoning acceleration;

[0024] Basic Linear Algebra Set, a numerical library for standardizing and publishing basic linear algebra operations;

[0025] Collective communication library, used to store HCCL operator information and to manage HCCL operator information;

[0026] The media data preprocessing module is used to implement video decoding, video encoding, JPEG decoding, JPEG encoding, PNG decoding, and VPC. The codec library performs image preprocessing through hardware acceleration, reduces CPU usage, and improves the parallel processing capability of image preprocessing.

[0027] As a further improvement of the present invention, the AI ​​big model constructed in step 2 includes:

[0028] The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries.

[0029] The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages.

[0030] The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation.

[0031] The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

[0032] As a further improvement of the present invention, the compilation optimization layer also includes an automatic hybrid parallel optimization step, specifically including data parallelism, automatic parallelism and heterogeneous parallelism. The specific steps of parallelism are as follows:

[0033] Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP;

[0034] Step 2: Split the data set in the sample dimension and send it to different cards;

[0035] Step 3, perform network mapping, wherein the writing method of the network mapping is the same as that of a single-machine network;

[0036] Step 4: perform gradient calculation and insert distributed operators after gradient calculation to implement gradient aggregation operations between cards;

[0037] Step 5: Enter the parameter update step with the same gradient value calculated in step 4.

[0038] As a further improvement of the present invention, after the distributed operator is inserted in step 4, the tensor is segmented and modeled in units of distributed operators to represent how the input and output tensors of an operator are distributed to various cards in the cluster, thereby obtaining a tensor arrangement model.

[0039] As a further improvement of the present invention, each operator in step 4 has a segmentation strategy, which is composed of two tuples, representing the segmentation forms of input and weight respectively, where the elements in the tuple correspond one-to-one to the tensor dimensions, and 2^N is the number of segments.

[0040] 1 means no splitting. When the user wants to express a data parallel splitting strategy, that is, the input batch dimension is split, and other dimensions are not split, it can be expressed as strategy = ((2^N, 1, 1), (1, 1, 1)); when expressing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, taking the channel dimension splitting as an example, other dimensions are not split, it can be expressed as strategy = ((1, 1, 1), (1, 1, 2^N)); when expressing a mixed parallel splitting strategy, one of the splitting strategies is strategy = ((2^N, 1, 1), (1, 1, 2^N)).

[0041] As a further improvement of the present invention, the specific steps of constructing the operation system in step 3 are as follows:

[0042] Step 31: Deploy the computing power module, and deploy the core components of the AI ​​cluster software service and its dependent services on the cloud host on the computing server;

[0043] Step 32: Establish a safety construction module, which contains the information of the project manager, various functional departments and construction team leaders, and marks the project manager information with the first responsible person label, and marks the information of various functional departments and construction team leaders with hierarchical responsibility labels for user query.

[0044] The beneficial effect of the present invention is that through the effects of steps 1 to 3, an AI computing center can be effectively constructed, and the AI ​​computing center meets the following requirements:

[0045] Algorithm training platform: The algorithm platform is heterogeneously compatible, that is, it can run normally under different hardware, operating systems, and network environments. In terms of general computing power CPU, it is compatible with CPU servers of different architectures of X86 / ARM; in terms of computing power, in addition to NVIDIA series GPUs, it can also be compatible with Huawei Ascend series NPU training chips Ascend910, FPGA chips and other access management, which is convenient for enterprise users to make diverse choices of underlying computing power chips when building heterogeneous computing power clusters.

[0046] Algorithm reasoning platform: The algorithm reasoning platform can perform cluster management of inference servers, uniformly issue and call algorithms, and is also compatible with different domestic and NVIDIA chip systems, so that the cluster's computing power scheduling and use can reach a project-established state, and when customers call the algorithm, it can be complete and efficient.

[0047] Algorithm warehouse platform: Algorithms developed based on the training platform are open to the market through this platform. Customers can use SAAS algorithm services according to their needs. When using services externally, the inference platform can provide computing power scheduling and support. The algorithm warehouse can be continuously added and adjusted according to customer needs. Algorithm additions and adjustments require the support of training computing power.

[0048] Data self-service modeling software: It uses visual programming software to achieve the level of low-code or even no-code development. It can perform self-service modeling, unsupervised learning and other functions based on data collected by customers or yourself. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the architecture of the AI ​​big model;

[0050] Figure 2 Schematic diagram of a three-dimensional matrix multiplication operation;

[0051] Figure 3 This is a schematic diagram of Sample 1;

[0052] Figure 4 This is a schematic diagram of Sample 2;

[0053] Figure 5 This is a schematic diagram of sample three;

[0054] Figure 6 Flowchart of the graph fusion steps. DETAILED DESCRIPTION

[0055] The present invention will be further described below in detail with reference to the embodiments shown in the accompanying drawings.

[0056] A method for constructing an AI computing center in this embodiment is characterized by comprising the following steps:

[0057] Step 1: Build an AI computing power enabling subsystem for the AI ​​computing cluster subsystem and enabling hardware devices and developers for the development and training of AI algorithm models.

[0058] Step 2: Build a large AI model for users to perform AI model training tasks;

[0059] Step three, build an operation system, through which the AI ​​computing power subsystem, AI computing power enabling subsystem, AI computing power service platform, AI cluster network and security subsystem, basic platform service, security protection service and AI cluster software service are aggregated to form a customer-oriented entry portal. In the process of using the construction method of this embodiment, you only need to execute steps one to three in sequence to complete the construction of the AI ​​computing power center simply and effectively. The constructed AI computing power center includes an artificial intelligence computing power platform and an AI technology service platform. The artificial intelligence computing power platform mainly purchases and installs hardware such as servers to provide basic computing power for the entire project; the AI ​​technology service platform mainly integrates and develops software suitable for 3D digital asset creation, and provides services to customers by calling the computing power of the artificial intelligence computing power platform hardware.

[0060] As a specific implementation of the improvement, the AI ​​computing power enabling subsystem constructed in step 1 includes: heterogeneous computing architecture software for supporting developers to develop custom operators;

[0061] The AI ​​framework software is used to provide the AI ​​big model for step 2. By setting the AI ​​computing power enabling subsystem to a combination of heterogeneous computing architecture software and AI framework software, it is possible to simply and effectively realize the capabilities of each functional unit of the heterogeneous computing architecture, the capabilities of each functional unit of the AI ​​framework, and the comprehensive performance optimization plan for the heterogeneous computing architecture and AI framework. The comprehensive composition of the AI ​​computing power enabling engine can optimize the data processing capabilities of the AI ​​chip, improve the throughput of the AI ​​computing cluster subsystem, and help developers improve development and training efficiency.

[0062] As a specific implementation of the improvement, the heterogeneous computing architecture software includes:

[0063] The unified programming interface Ascend computing language library is used to provide unified and standardized interface components, including device management, context management, stream management, memory management, model loading and execution, operator loading and execution, media data processing, and graph management API libraries, providing users with programming interfaces that enable the capabilities of each component for users to develop deep neural network applications;

[0064] The graph optimization and compilation module is used to provide a unified IR interface for different front-ends, support the parsing / optimization / compilation of the computational graphs expressed by TensorFlow / Caffe / MindSpore, and provide the ability to optimize the deployment of the back-end computing engine to give full play to the performance of the device;

[0065] The computing execution engine is responsible for the scheduling and execution of various types of operators in the neural network, memory and other resources, communication management, etc., and outputs the computing results;

[0066] Operator compilation module and operator library, used to provide Ascend operator development and compilation capabilities, enabling chip capabilities; computing library, used to provide computing libraries such as neural networks, basic linear algebra, collective communication, and media data preprocessing, to support users to use and build AI models;

[0067] The computing tuning engine includes two plug-ins: the AutoTune plug-in, which supports single operator and whole graph tuning modes. It can automatically generate a knowledge base based on the model and data features, and apply it to the actual deployment process to obtain high performance; the model compression plug-in provides model quantization and sparsity capabilities in the form of a framework plug-in, supports Calibration and ReTrain, and realizes model miniaturization. In this way, through the settings of the above modules, it is possible to simply and effectively provide a multi-level programming interface, with the advantages of full scenarios, low thresholds, and high performance, to support users to quickly build AI applications and businesses based on the Ascend platform.

[0068] As a specific implementation of the improvement, the graph optimization and compilation module includes:

[0069] GE (GraphEngine) module, which serves as the control center for graph compilation and operation, and provides graph operation environment management, graph execution engine management, operator library management, subgraph optimization management, graph operation management, and graph execution control;

[0070] FE (FusionEngine) module, which is used to manage the fusion rules of operators, match IRGraph and perform operator fusion operations according to the fusion rules, manage the operators supported by the system and maintain the operator list, and support custom operators;

[0071] AICPUE (AICPUEngine) module, which is responsible for AICPU operator information management;

[0072] HCCL (Huawei Collective Communication Library) module: This HCCL module is responsible for the operator information management of HCCL, and is used to provide HCCL to implement the gradient aggregation function of all workers involved in parallel computing, and provide a data parallel solution for Ascend multi-machine and multi-card training. Through the settings of the above modules, graph optimization and compilation operations can be effectively implemented.

[0073] As a specific implementation of the improvement, the calculation execution engine includes:

[0074] The Runtime module is used to provide resource management channels for neural network tasks, including storage (Memory) management, device (Device) management, execution flow (Stream) management, and event (Event)

[0075] Management, kernel function execution and other functions;

[0076] The TS module is responsible for the management, scheduling, and execution of the Task sequence of the computational graph. By setting up the above modules, the computation can be simply and effectively implemented, and the computation results can be output.

[0077] As a specific implementation of the improvement, the operator compilation module includes:

[0078] The TBE module is used to automatically compile and generate operators by describing the algorithm implementation and optimization scheduling of operators through DSL (domain-specific language). At the same time, this module is also used as a development tool for custom operators for ecosystem developers to use;

[0079] Among them, the operator library is the Ascend neural network acceleration library, which has rich built-in operators to support neural network training and reasoning acceleration. Through the setting of the above modules and operator libraries, it is possible to simply and effectively provide Ascend operator development and compilation capabilities and enable chip capabilities.

[0080] As a specific implementation of the improvement, the calculation library includes:

[0081] Neural network library, used as the Ascend neural network acceleration library, with rich built-in operators to support neural network training and reasoning acceleration;

[0082] Basic linear algebra set, a numerical library for standardizing the release of basic linear algebra operations. BLAS (Basic Linear Algebra Program Set) is an application program interface (API) standard that is used to standardize the release of basic linear algebra operations (such as vector or matrix multiplication). In the field of high-performance computing, BLAS is widely used. In AI chip enablement, some BLAS library implementations are also provided, and are constantly iterating and supplementing. Provide a standard BLAS library that users can call directly, shielding the underlying detail differences, and thus obtaining the best computing experience and effect on the AI ​​platform;

[0083] Collective Communication Library, which is used to store HCCL (collective communication library) operator information and is responsible for HCCL operator information management. HCCL implements the gradient aggregation (allreduce) function of all workers participating in parallel computing and provides a data parallel solution for Ascend multi-machine and multi-card training.

[0084] The media data preprocessing module is used to implement video decoding, video encoding, JPEG decoding, JPEG encoding, PNG decoding, and VPC. The codec library performs image preprocessing through hardware acceleration, reduces CPU usage, and improves the parallel processing capability of image preprocessing.

[0085] As a specific implementation of the improvement, the computing optimization engine includes:

[0086] The AutoTune module supports two tuning modes: single operator and whole graph. It can automatically generate a knowledge base based on the model and data features, and apply it to the actual deployment process to obtain high performance.

[0087] The model compression module provides model quantization and sparsity capabilities in the form of a framework plug-in, supports Calibration, ReTrain, etc., and realizes the miniaturization of the model. In this way, through the setting of the AutoTune module, Autotune can complete the deep performance optimization of the operator for specific model specifications. Users only need to make a small investment to improve the performance of the model; in the model conversion or model training stage, the user is provided with the function of automatic tuning, which can automatically complete the automatic performance optimization of the operator in the model and improve the end-to-end running speed of the model; Autotune can automatically complete the performance optimization of the model and output the knowledge base file at the same time. Users can deploy the knowledge base file to a new environment, and the same model can obtain the previously tuned performance without turning on Autotune again. The AI ​​chip enables the provision of model compression tools, which currently include model quantization and tensor decomposition functions to improve the model inference performance. Model quantization provides two quantization modes: post-training quantization and quantization-aware training. The tensor decomposition function supports decomposing some convolutional layers in the original model into several layers with smaller computational and parameter amounts. Post-training quantization refers to the INT8 quantization of the weights and data of the trained model with the help of a small amount of calibration data, thereby accelerating the model inference speed. The post-training quantization function in the model compression tool can complete the model quantization with only a small amount of data in resource-constrained scenarios, which can provide better business capabilities, better adapt to customers' hardware resources, and better serve their business applications.

[0088] In this embodiment, in order to facilitate users to perform AI model training tasks in the computing center, this solution supports the open source AI framework MindSpore. MindSpore is Huawei's AI development framework and adopts an open source release strategy. The MindSpore open source community is as follows: https: / / gitee.com / mindspore. MindSpore provides users with programming paradigms in languages ​​such as Python. Based on source code conversion, users can use native Python control syntax and some other advanced APIs, such as tuples, lists, and Lambda expressions. MindSpore is an integrated training and push AI framework that supports both training and reasoning. MindSpore builds the graph structure of the neural network based on native Python and supports both dynamic and static graphs. In the era of large models, MindSpore also provides distributed parallel capabilities. The framework provides built-in multi-dimensional distributed training strategies that users can flexibly assemble and use. It also simplifies the complexity of user parallel programming by hiding communication operations through parallel abstraction;

[0089] Reference Figure 1 As shown, the AI ​​big model constructed for this embodiment includes the following four layers:

[0090] The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries.

[0091] The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages.

[0092] The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation.

[0093] The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

[0094] As a specific implementation method for improving the above AI big model, MindSpore supports the current mainstream distributed training paradigm and develops a set of automatic hybrid parallel solutions. MindSpore supports three methods: data parallelism, automatic parallelism, and heterogeneous parallelism. The specific parallel steps are as follows:

[0095] Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP;

[0096] Step 2: Split the data set in the sample dimension and send it to different cards;

[0097] Step 3: Perform network construction. The writing method of the network construction is the same as that of the stand-alone network. This is because the models of each card are executed independently during the forward and backward propagation process, but the same network structure is maintained.

[0098] Step 4: perform gradient calculation. To ensure the consistency of the calculation logic and insert distributed operators after the gradient calculation to achieve gradient aggregation between cards, in this embodiment, MindSpore sets a mean switch. The user can choose whether to average the summed gradient values ​​or treat it as a hyperparameter. Turning on the switch is equivalent to reducing the learning rate multiple, thus completing the gradient aggregation operation.

[0099] In step 5, the same gradient value calculated in step 4 is used to enter the parameter update step. Therefore, through the above steps, hybrid parallel processing can be simply and effectively implemented. From the above steps 1 to 5, it can be seen that MindSpore implements a synchronous data parallel training method. In theory, the models trained on each card are the same. If the network contains reduction type operations in the sample dimension, the output of the network may be different, which is determined by the segmentation nature of data parallelism. Automatic parallelism is a key feature of MindSpore. It is used to implement a hybrid parallel training method of automatic data parallelism and model parallelism. It aims to help users express parallel algorithm logic with stand-alone scripts, reduce the difficulty of distributed training, and improve the efficiency of algorithm development, while maintaining high training performance.

[0100] As a specific implementation of the improvement, the specific steps of splitting the data set in step 2 in the sample dimension and sending it to different cards are as follows:

[0101] Step 21, through the two parameters num_shards and shard_id provided by the mindspore.dataset module in all dataset loading interfaces, the dataset is split into multiple parts and the batch size data is collected to each card in a circular sampling manner. When the amount of data is insufficient, sampling will start from the beginning. In this way, the two parameters num_shards and shard_id can be used to effectively realize the splitting and distribution of the dataset.

[0102] As a specific implementation of the improvement, after the distributed operator is inserted in step 4, the tensor is segmented and modeled in units of distributed operators to indicate how the input and output tensors of an operator are distributed to each card in the cluster, and the tensor layout model is obtained. During the above-mentioned automatic parallel process, the forward calculation graph (ANFGraph) of a single machine will be traversed, and the tensor is segmented and modeled in units of distributed operators (DistributedOperator) to indicate how the input and output tensors of an operator are distributed to each card in the cluster (TensorLayout). This model fully expresses the mapping relationship between tensors and devices. Users do not need to perceive which device each slice of the model is placed on to run, and the framework will automatically schedule and allocate.

[0103] In order to obtain the arrangement model of the tensor in this embodiment, each operator has a sharding strategy (ShardStrategy), which indicates the sharding of each input of the operator in the corresponding dimension. Generally, as long as the principle of 2-based and uniform distribution is met, any dimension of the tensor can be sharded. Figure 2 For example, this is a three-dimensional matrix multiplication (BatchMatMul) operation, and its splitting strategy consists of two tuples, which represent the splitting forms of input and weight respectively. The elements in the tuple correspond to the tensor dimensions one by one, 2^N is the number of splits, and 1 means no split. When the user wants to express a data parallel splitting strategy, that is, the batch dimension of the input is split, and other dimensions are not split, it can be expressed as strategy=((2^N,1,1),(1,1,1)); when expressing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, here we take the channel dimension splitting as an example, and other dimensions are not split, which can be expressed as

[0104] strategy=((1,1,1),(1,1,2^N)); When representing a hybrid parallel splitting strategy, one of the splitting strategies is strategy=((2^N,1,1),(1,1,2^N)). Based on the splitting strategy, the distributed operator defines a method for deriving the layout model of the operator input tensors and output tensors. This layout model consists of device_matrix, tensor_shape and tensormap, which respectively represent the device matrix shape, tensor shape, and the mapping relationship between the device and tensor dimensions. The distributed operator will further determine whether to insert additional calculation and communication operations into the graph based on the tensor layout model to ensure the correctness of the operator operation logic. Tensor layout transformation: When the output tensor model of the current operator is inconsistent with the input tensor model of the next operator, it is necessary to introduce calculation and communication operations to realize the change between tensor layouts. In this way, the automatic parallel process introduces the tensor redistribution algorithm (TensorRedistribution), which can derive the communication conversion method between tensors of any layout. The following three examples show the parallel computing process of the formula Z = (X × W) × V, that is, two two-dimensional matrix multiplication operations, which demonstrate how to convert between different parallel methods. Figure 3 As shown in the figure, in Example 1, the output of the first data parallel matrix multiplication is split in the row direction, while the input of the second model parallel matrix multiplication requires the full amount of tensors. The framework will automatically insert the AllGather operator to implement the arrangement transformation, such as Figure 4As shown in the figure, in Example 2, the output of the first model parallel matrix multiplication is split in the column direction, while the input of the second data parallel matrix multiplication is split in the row direction. The framework will automatically insert a communication operator equivalent to the AlltoAll operation in the set communication to implement the arrangement transformation, such as Figure 5 As shown in the figure, in Example 3, the output segmentation method of the first hybrid parallel matrix multiplication is consistent with the input segmentation method of the second hybrid parallel matrix multiplication, so there is no need to introduce a rearrangement transformation. However, since the related dimensions of the two inputs are segmented in the second matrix multiplication operation, the AllReduce operator needs to be inserted to ensure the correctness of the operation.

[0105] In summary, distributed operators and tensor arrangement models and tensor arrangement transformations are the basis for automatic parallel implementation. In general, this distributed expression breaks the boundary between data parallelism and model parallelism, and easily realizes hybrid parallelism. From the script level, users only need to construct a single-machine network to express the logic of the parallel algorithm, and the framework will automatically implement the sharding of the entire graph. Sharding strategy search: When the user is familiar with the sharding expression of the operator and manually configures the sharding strategy for the operator, this is the SEMI_AUTO_PARALLEL semi-automatic parallel mode. This method is helpful for manual tuning, but it still has certain debugging difficulties. Users need to master the parallel principle and obtain high-performance parallel solutions based on the network structure, cluster topology and other calculations and analysis. In order to help users accelerate the parallel network training process, based on the semi-automatic parallel mode, the AUTO_PARALLEL automatic parallel mode supports parallel strategy propagation (ShardingPropagation), which can effectively reduce the workload of users manually configuring operator sharding strategies. The algorithm propagates the sharding strategy from the user-configured operator to the unconfigured operator. In order to further reduce the workload of users to manually configure operator segmentation strategies, fully automatic search of segmentation strategies is supported. To this end, a corresponding cost function model (CostModel) is built around the hardware platform to calculate the computational cost (ComputationCost), memory cost (MemoryCost) and communication cost (CommunicationCost) of a certain amount of data and a certain operator under different segmentation strategies. Then, through dynamic programming algorithms (DynamicProgramming) or recursive programming algorithms (RecursiveProgramming), with the memory limit of a single card as a constraint, a segmentation strategy with better performance is efficiently searched. Distributed automatic differentiation: In addition to paying attention to forward network communication, traditional manual model segmentation also needs to consider the parallel operation of the network reverse. MindSpore packages the communication operation into an operator and uses the original automatic differentiation operation of the framework to automatically generate the reverse of the communication operator. Therefore, even when performing distributed training, users only need to pay attention to the forward propagation of the network, truly realizing fully automatic parallel training. Heterogeneous parallel training method is to analyze the memory usage and computing density of operators on the graph, divide the operators with huge memory consumption or suitable for CPU logic processing into CPU subgraphs, and divide the operators with small memory consumption and computing intensiveness into hardware accelerator subgraphs. The framework coordinates different subgraphs to perform network training, so that subgraphs on different hardware and without dependencies can be executed in parallel.

[0106] As a specific implementation method of the improvement, refer to Figure 6 As shown, the steps of performing graph-computation fusion in the compilation optimization layer are as follows:

[0107] Step a: Expand the composite operator into basic operators and form a Composite subgraph to facilitate subsequent cross-boundary optimization and operator splitting;

[0108] Step b: Aggregate adjacent basic operators or Composite subgraphs to form a larger aggregate subgraph, which is convenient for subsequent cross-boundary optimization and operator splitting;

[0109] Step c, based on the aggregated subgraphs obtained in steps a and b, a large number of cross-boundary optimizations are performed, such as algebraic simplification and common sub-expression extraction;

[0110] In step d, operators are split into subgraphs of aggregated computations based on computational features and fusion operator performance, and then the optimized computational graphs are passed to MindSporeAKG in the form of subgraphs to continue backend optimization and generate target code. Through the settings of steps a to d above, the fusion operator is accelerated and optimized. In scenarios such as deep neural network training, graph-computing fusion optimization can bring exponential performance improvements. However, with the continuous enhancement of graph-computing fusion capabilities, the development of fusion operators has become a bottleneck for further improving graph-computing fusion capabilities. The automatic generation technology of fusion operators can solve the problem of high programming thresholds for developing fusion operators based on DSA, allowing programmers to focus on the implementation logic of operators during operator development without having to pay attention to backend optimization, greatly improving their development efficiency. Especially for scenarios with complex backend hardware architectures and the presence of complex operators and fusion operators, automatic operator generation technology is even more critical.

[0111] As a specific implementation method of the improvement, the specific steps of building the operation system in step 3 are as follows: Step 31, deploying a computing power module, deploying the core components of the AI ​​cluster software service and its dependent services on the cloud host on the computing server;

[0112] Step 32: Establish a security construction module. The security construction module contains the information of the project manager, functional departments and construction team leaders, and marks the project manager information with the first responsible person label, and marks the information of the functional departments and construction team leaders with hierarchical responsibility labels for user query. Through the settings of the above steps 31 and 32, it is possible to simply and effectively realize the aggregation of AI computing power subsystem, AI computing power enabling subsystem, AI computing power service platform, AI cluster network and security subsystem, basic platform services, security protection services, and AI cluster software services, and create a one-stop AI computing power solution portal to meet the needs of artificial intelligence-related industries.

[0113] Among them, in the AI ​​cluster infrastructure construction of this embodiment, the AI ​​computing power service platform only involves the construction of basic computing servers. In this embodiment, the cloud host on the computing server is mainly used to deploy the core components of ModelArts (AI cluster software service) and its dependent services. As the AI ​​computing power is built from 140P to 240P, ModelArts, which is responsible for scheduling and arranging computing power, needs to be built accordingly. Since the service components of ModelArts are deployed on virtual machines, as the number of components increases, the number of virtual machines needs to increase, and the storage capacity also needs to increase, which requires more computing, memory and storage resources. Memory resources and computing resources are provided by computing node physical machines, and storage resources are provided by ultra-high IO storage node physical servers. Therefore, it is currently necessary to build computing server nodes and ultra-high IO storage server nodes. The construction of computing and storage servers in this period takes into account the needs of this period. The computing server resources after construction need to meet the deployment requirements of ModelArts service components in the 100PFlops@FP16 computing power scenario, specifically building 9 computing nodes and 8 ultra-high IO storage nodes. Computing servers provide CPU and memory resources, ultra-high IO servers provide storage resources, virtualization and cloud services pool resources, cloudify and then provide them to elastic cloud servers (ECS), and AI cluster software services are mainly deployed on elastic cloud servers. As the computing capacity of current AI cluster software services increases, the number of elastic cloud servers required increases. As the number of elastic cloud servers increases, the number of CPUs, memory and storage required increases accordingly. According to the connection table in the low-level design, connect the cables and optical modules of all devices. The cables include network cables, optical fibers and power cables. Pay attention to distinguish between 10G and 25G, 40G and 100G optical modules. After completing the cable and module connection and powering on the cabinet, make sure that all port indicators in the connection table are on. If there is a hardware failure, contact the manufacturer's engineer in time for troubleshooting or replacement. Secondly, the internal equipment debugging of the cloud platform is completed by the Tiangong Tianlu tool, which needs to be installed first. Tiangong is used for server BMC, BIOS, RAID, PXE configuration and connection check, etc. Tianlu is used for network device configuration generation and distribution, switch version upgrade and connection check, etc. Finally, after completing the hardware connection and power-on of the equipment, use the Tiangong Tianlu tool to automatically configure the network equipment and servers. Installation steps: clarify the connection of all servers and switches, build the hardware network, and output LLD. Use our software tools to detect the entire hardware network to ensure that all connections and networking are normal. According to the dependency relationship, first deploy the cloud platform software service to provide basic cloud services; after the deployment is completed, the basic cloud

[0114] The service is connected to the remote operation and maintenance platform, and the operation system is deployed on this basis.

[0115] In summary, the AI ​​computing power center construction method of this embodiment can accurately and quickly realize the construction of the AI ​​computing power center.

[0116] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for constructing an AI computing power center, characterized in that: The steps include: Step 1: Build an AI computing power enabling subsystem for the AI ​​computing cluster subsystem and enabling hardware devices and developers for the development and training of AI algorithm models. Step 2: Build a large AI model for users to perform AI model training tasks; Step three: Build an operation system, which aggregates the AI ​​computing power subsystem, AI computing power enabling subsystem, AI computing power service platform, AI cluster network and security subsystem, basic platform services, security protection services and AI cluster software services to form a customer-oriented entry portal.

2. The method for constructing an AI computing center according to claim 1, characterized in that: The AI ​​computing power enabling subsystem constructed in step 1 includes: Heterogeneous computing architecture software to support developers in developing custom operators; AI framework software is used to provide step 2 for building a large AI model.

3. The method for constructing an AI computing center according to claim 2, characterized in that: The heterogeneous computing architecture software includes: The unified programming interface Ascend computing language library is used to provide unified and standardized interface components, including device management, context management, stream management, memory management, model loading and execution, operator loading and execution, media data processing, and graph management API libraries, providing users with programming interfaces that enable the capabilities of each component for users to develop deep neural network applications; The graph optimization and compilation module is used to provide a unified IR interface for different front-ends, support the parsing / optimization / compilation of the computational graphs expressed by TensorFlow / Caffe / MindSpore, and provide the ability to optimize the deployment of the back-end computing engine to give full play to the performance of the device; The computing execution engine is responsible for the scheduling and execution of various types of operators in the neural network, memory and other resources, communication management, etc., and outputs the computing results; Operator compilation modules and operator libraries are used to provide Ascend operator development and compilation capabilities and enable chip capabilities; Computing library, which is used to provide computing libraries such as neural networks, basic linear algebra, collective communication, and media data preprocessing, to support users in using and building AI models; The computing tuning engine includes two plug-ins: the AutoTune plug-in, which supports both single operator and whole-graph tuning modes. It can automatically generate a knowledge base based on the model and data features, and apply it to the actual deployment process to obtain high performance. The model compression plug-in provides model quantization and sparsity capabilities in the form of a framework plug-in, supports Calibration and ReTrain, and realizes model miniaturization.

4. The method for constructing an AI computing center according to claim 3, characterized in that: The graph optimization and compilation module includes: GE module: As the control center for graph compilation and operation, the GE module provides graph operation environment management, graph execution engine management, operator library management, subgraph optimization management, graph operation management, and graph execution control; FE module: This module is used to manage the fusion rules of operators, match IRGraph and perform operator fusion operations according to the fusion rules, manage operators supported by the system, maintain the operator list, and support custom operators; AICPUE module, which is responsible for AICPU operator information management; HCCL module: This module is responsible for the operator information management of HCCL, and is used to provide HCCL to implement the gradient aggregation function of all workers involved in parallel computing, and provide a data parallel solution for Ascend multi-machine and multi-card training.

5. The method for constructing an AI computing center according to claim 4, characterized in that: The calculation library includes: Neural network library, used as the Ascend neural network acceleration library, with rich built-in operators to support neural network training and reasoning acceleration; Basic Linear Algebra Set, a numerical library for standardizing and publishing basic linear algebra operations; Collective communication library, used to store HCCL operator information and to manage HCCL operator information; The media data preprocessing module is used to implement video decoding, video encoding, JPEG decoding, JPEG encoding, PNG decoding, and VPC. The codec library performs image preprocessing through hardware acceleration, reduces CPU usage, and improves the parallel processing capability of image preprocessing.

6. The method for constructing an AI computing center according to claims 1 to 5, characterized in that: The AI ​​big model constructed in step 2 includes: The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries. The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages. The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation. The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

7. The method for constructing an AI computing center according to claim 6, characterized in that: The compilation optimization layer also includes automatic hybrid parallel optimization steps, specifically including data parallelism, automatic parallelism and heterogeneous parallelism. The specific steps of parallelism are as follows: Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP; Step 2: Split the data set in the sample dimension and send it to different cards; Step 3, perform network mapping, wherein the writing method of the network mapping is the same as that of a single-machine network; Step 4: perform gradient calculation and insert distributed operators after gradient calculation to implement gradient aggregation operations between cards; Step 5: Enter the parameter update step with the same gradient value calculated in step 4.

8. The method for constructing an AI computing center according to claim 7, characterized in that: After the distributed operator is inserted in step 4, the tensor is segmented and modeled based on the distributed operator to indicate how the input and output tensors of an operator are distributed to each card in the cluster, thereby obtaining a tensor arrangement model.

9. The method for constructing an AI computing center according to claim 8, characterized in that: In step 4, each operator has a splitting strategy, which consists of two tuples, representing the splitting forms of input and weight respectively, where the elements in the tuple correspond to the tensor dimensions one by one, 2^N is the number of splits, and 1 means no splitting. When the user wants to express a data parallel splitting strategy, that is, the batch dimension of the input is split, and other dimensions are not split, it can be expressed as strategy=((2^N,1,1),(1,1,1)); when expressing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, taking the channel dimension splitting as an example, other dimensions are not split, which can be expressed as strategy=((1,1,1),(1,1,2^N)); when expressing a hybrid parallel splitting strategy, one of the splitting strategies is strategy=((2^N,1,1),(1,1,2^N)).

10. The method for constructing an AI computing center according to claim 8, characterized in that: The specific steps for building the operation system in step 3 are as follows: Step 31: Deploy the computing power module, and deploy the core components of the AI ​​cluster software service and its dependent services on the cloud host on the computing server; Step 32: Establish a safety construction module, which contains the information of the project manager, various functional departments and construction team leaders, and marks the project manager information with the first responsible person label, and marks the information of various functional departments and construction team leaders with hierarchical responsibility labels for user query.

Images