Data classification and grading system

By introducing model layer, expression layer, compilation optimization layer and operation layer into the data classification and grading system, providing preset models and development kits, supporting native Python syntax development and global performance optimization, the problem of high difficulty in using existing systems is solved. It realizes effective call and performance optimization of AI computing.

CN119089955APending Publication Date: 2024-12-06HANGZHOU SHENJIA TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411218786.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

During the use of the existing data classification and grading system, users need to have professional programming language literacy, which makes it more difficult to use.

Method used

Provide a data classification and hierarchy system, including model layer, expression layer, compilation optimization layer and operation layer. The model layer provides preset models and development kits. The expression layer supports native Python syntax development, and the compilation optimization layer performs global performance optimization, and runs. The layer calls the underlying hardware operator through the unified runtime architecture of 'end-edge cloud', supporting federated learning, etc.

Benefits of technology

It reduces the user's requirements for programming languages, increases the difficulty of using the system, and realizes global performance optimization and effective calls of AI computing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089955A_ABST
    Figure CN119089955A_ABST
Patent Text Reader

Abstract

The invention discloses a data classification and grading system, and the system comprises a model layer which provides a user with an unpacking and ready-to-use function, and comprises a preset model, a development suite, and hotspot research field expansion libraries including a graph neural network, depth probability programming, and a scientific calculation library; the expression layer is used for providing AI model development, training and reasoning interfaces for a user; the compilation optimization layer is used as the core of the AI framework, takes the full-scene unified intermediate expression as a medium, compiles the front-end expression into an underlying language with higher execution efficiency, and performs global performance optimization at the same time; and the operation layer is in butt joint according to a compiling and optimizing result of the compiling and optimizing layer and calls a bottom layer hardware operator, and meanwhile, an end-side cloud unified runtime architecture is adopted. According to the data classification and grading system, through the arrangement of four layers, a user can conveniently do an AI model training task in a computing power center.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a model system, and more particularly to a data classification and grading system. Background Art

[0002] The construction of the artificial intelligence computing power center platform is a miniaturized artificial intelligence computing power center built for the company's own use and leasing by upstream and downstream enterprises in the business chain to meet the computing power needs of 3D digital asset creation. Its construction content includes artificial intelligence computing power platform and AI technical service platform. The artificial intelligence computing power platform mainly purchases and installs hardware such as servers to provide basic computing power for the entire project; the AI ​​technical service platform mainly integrates and develops software suitable for 3D digital asset creation, and provides services to customers by calling the computing power of the artificial intelligence computing power platform hardware. The computing power platform and the technical service platform are complementary and together constitute the artificial intelligence computing power center.

[0003] The current computing center uses a large AI model as a data classification and grading system. During use, users are required to have professional programming language literacy, which greatly increases the difficulty of using the data classification and grading system. Summary of the invention

[0004] In view of the deficiencies in the prior art, an object of the present invention is to provide a data classification and grading system that is less difficult to use.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a data classification and grading system, characterized in that it includes:

[0006] The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries.

[0007] The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages.

[0008] The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation.

[0009] The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

[0010] As a further improvement of the present invention, the compilation optimization layer also includes an automatic hybrid parallel optimization step, specifically including data parallelism, automatic parallelism and heterogeneous parallelism. The specific steps of parallelism are as follows:

[0011] Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP;

[0012] Step 2: Split the data set in the sample dimension and send it to different cards;

[0013] Step 3, perform network mapping, wherein the writing method of the network mapping is the same as that of a single-machine network;

[0014] Step 4: perform gradient calculation and insert distributed operators after gradient calculation to implement gradient aggregation operations between cards;

[0015] Step 5: Enter the parameter update step with the same gradient value calculated in step 4.

[0016] As a further improvement of the present invention, the specific steps of splitting the data set in the sample dimension and sending it to different cards in step 2 are as follows:

[0017] Step 21, through the two parameters num_shards and shard_id provided by the mindspore.dataset module, which are available in all dataset loading interfaces, the dataset is split into multiple parts and sampled in a circular manner to collect batch-sized data onto respective cards. When the amount of data is insufficient, sampling will start from the beginning.

[0018] As a further improvement of the present invention, after the distributed operator is inserted in step 4, the tensor is segmented and modeled in units of distributed operators to represent how the input and output tensors of an operator are distributed to various cards in the cluster, thereby obtaining a tensor arrangement model.

[0019] As a further improvement of the present invention, each operator in step 4 has a splitting strategy, and the splitting strategy is composed of two tuples, which respectively represent the splitting forms of input and weight, wherein the elements in the tuple correspond to the tensor dimensions one by one, 2^N is the number of splits, and 1 represents no splitting. When the user wants to represent a data parallel splitting strategy, that is, the batch dimension of input is split, and other dimensions are not split, it can be expressed as strategy=((2^N,1,1),(1,1,1)); when representing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, taking the channel dimension splitting as an example, other dimensions are not split, which can be expressed as strategy=((1,1,1),(1,1,2^N)); when representing a hybrid parallel splitting strategy, one of the splitting strategies is strategy=((2^N,1,1),(1,1,2^N)).

[0020] As a further improvement of the present invention, the steps of performing graph-computation fusion in the compilation optimization layer are as follows:

[0021] Step a: Expand the composite operator into basic operators and form a Composite subgraph to facilitate subsequent cross-boundary optimization and operator splitting;

[0022] Step b: Aggregate adjacent basic operators or Composite subgraphs to form a larger aggregate subgraph, which is convenient for subsequent cross-boundary optimization and operator splitting;

[0023] Step c, based on the aggregated subgraphs obtained in steps a and b, a large number of cross-boundary optimizations are performed, such as algebraic simplification and common sub-expression extraction;

[0024] In step d, operators are split into subgraphs based on computational features and fusion operator performance. The optimized computational graphs are then passed to MindSporeAKG in the form of subgraphs to continue backend optimization and generate target code.

[0025] The beneficial effect of the present invention is that, through the setting of the model layer, it can effectively provide users with a preset model and development kit as well as expansion libraries in hot research fields such as graph neural networks, deep probabilistic programming, and scientific computing libraries. Then, through the setting of the expression layer, it can provide users with AI model development, training, and reasoning interfaces, supporting users to develop and debug neural networks with native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios in the production and deployment stages. Through the setting of the compilation optimization layer, global performance optimization can be achieved. Finally, through the setting of the running layer, the underlying hardware operators can be effectively called to realize AI computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the architecture of the AI ​​big model;

[0027] Figure 2 Schematic diagram of a three-dimensional matrix multiplication operation;

[0028] Figure 3 This is a schematic diagram of Sample 1;

[0029] Figure 4 This is a schematic diagram of Sample 2;

[0030] Figure 5 This is a schematic diagram of sample three;

[0031] Figure 6 Flowchart of the graph fusion steps. DETAILED DESCRIPTION

[0032] The present invention will be further described below in detail with reference to the embodiments shown in the accompanying drawings.

[0033] In this embodiment, in order to facilitate users to perform AI model training tasks in the computing center, this solution supports the open source AI framework MindSpore. MindSpore is Huawei's AI development framework, which adopts an open source release strategy. The MindSpore open source community is as follows: https: / / gitee.com / mindspore. MindSpore provides users with programming paradigms in languages ​​such as Python. Based on source code conversion, users can use native Python control syntax and some other advanced APIs, such as tuples, lists, and Lambda expressions. MindSpore is an integrated training and push AI framework that supports training and reasoning functions. MindSpore builds a graph structure of a neural network based on native Python, supporting both dynamic and static graphs. In the era of large models, MindSpore also provides distributed parallel capabilities, and the framework provides built-in multi-dimensional distributed training strategies for users to flexibly assemble and use. And through parallel abstraction, communication operations are hidden to simplify the complexity of user parallel programming.

[0034] For this purpose, refer to Figure 1 As shown, a data classification and grading system of this embodiment includes the following four layers:

[0035] The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries.

[0036] The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages.

[0037] The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation.

[0038] The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

[0039] As a specific implementation method for improving the above AI big model, MindSpore supports the current mainstream distributed training paradigm and develops a set of automatic hybrid parallel solutions. MindSpore supports three methods: data parallelism, automatic parallelism, and heterogeneous parallelism. The specific parallel steps are as follows:

[0040] Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP;

[0041] Step 2: Split the data set in the sample dimension and send it to different cards;

[0042] Step 3: Perform network construction. The writing method of the network construction is the same as that of the stand-alone network. This is because the models of each card are executed independently during the forward and backward propagation process, but the same network structure is maintained.

[0043] Step 4: perform gradient calculation. To ensure the consistency of the calculation logic and insert distributed operators after the gradient calculation to achieve gradient aggregation between cards, in this embodiment, MindSpore sets a mean switch. The user can choose whether to average the summed gradient values ​​or treat it as a hyperparameter. Turning on the switch is equivalent to reducing the learning rate multiple, thus completing the gradient aggregation operation.

[0044] In step 5, the same gradient value calculated in step 4 is used to enter the parameter update step. Therefore, through the above steps, hybrid parallel processing can be simply and effectively implemented. From the above steps 1 to 5, it can be seen that MindSpore implements a synchronous data parallel training method. In theory, the models trained on each card are the same. If the network contains reduction type operations in the sample dimension, the output of the network may be different, which is determined by the segmentation nature of data parallelism. Automatic parallelism is a key feature of MindSpore. It is used to implement a hybrid parallel training method of automatic data parallelism and model parallelism. It aims to help users express parallel algorithm logic with stand-alone scripts, reduce the difficulty of distributed training, and improve the efficiency of algorithm development, while maintaining high training performance.

[0045] As a specific implementation of the improvement, the specific steps of splitting the data set in step 2 in the sample dimension and sending it to different cards are as follows:

[0046] Step 21, through the two parameters num_shards and shard_id provided by the mindspore.dataset module in all dataset loading interfaces, the dataset is split into multiple parts and the batch size data is collected to each card in a circular sampling manner. When the amount of data is insufficient, sampling will start from the beginning. In this way, the two parameters num_shards and shard_id can be used to effectively realize the splitting and distribution of the dataset.

[0047] As a specific implementation of the improvement, after the distributed operator is inserted in step 4, the tensor is segmented and modeled in units of distributed operators to indicate how the input and output tensors of an operator are distributed to each card in the cluster, and the tensor layout model is obtained. During the above-mentioned automatic parallel process, the forward calculation graph (ANFGraph) of a single machine will be traversed, and the tensor is segmented and modeled in units of distributed operators (DistributedOperator) to indicate how the input and output tensors of an operator are distributed to each card in the cluster (TensorLayout). This model fully expresses the mapping relationship between tensors and devices. Users do not need to perceive which device each slice of the model is placed on to run, and the framework will automatically schedule and allocate.

[0048] In order to obtain the arrangement model of the tensor in this embodiment, each operator has a sharding strategy (ShardStrategy), which indicates the sharding of each input of the operator in the corresponding dimension. Generally, as long as the principle of 2-based and uniform distribution is met, any dimension of the tensor can be sharded. Figure 2For example, this is a three-dimensional matrix multiplication (BatchMatMul) operation, and its splitting strategy consists of two tuples, which represent the splitting forms of input and weight respectively. The elements in the tuple correspond to the tensor dimensions one by one, 2^N is the number of splits, and 1 means no split. When the user wants to express a data parallel splitting strategy, that is, the batch dimension of the input is split, and other dimensions are not split, it can be expressed as strategy=((2^N,1,1),(1,1,1)); when expressing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, here we take the channel dimension splitting as an example, and other dimensions are not split, which can be expressed as

[0049] strategy=((1,1,1),(1,1,2^N)); When representing a hybrid parallel splitting strategy, one of the splitting strategies is strategy=((2^N,1,1),(1,1,2^N)). Based on the splitting strategy, the distributed operator defines a method for deriving the layout model of the operator input tensors and output tensors. This layout model consists of device_matrix, tensor_shape and tensormap, which respectively represent the device matrix shape, tensor shape, and the mapping relationship between the device and tensor dimensions. The distributed operator will further determine whether to insert additional calculation and communication operations into the graph based on the tensor layout model to ensure the correctness of the operator operation logic. Tensor layout transformation: When the output tensor model of the current operator is inconsistent with the input tensor model of the next operator, it is necessary to introduce calculation and communication operations to realize the change between tensor layouts. In this way, the automatic parallel process introduces the tensor redistribution algorithm (TensorRedistribution), which can derive the communication conversion method between tensors of any layout. The following three examples show the parallel computing process of the formula Z = (X × W) × V, that is, two two-dimensional matrix multiplication operations, which demonstrate how to convert between different parallel methods. Figure 3 As shown in the figure, in Example 1, the output of the first data parallel matrix multiplication is split in the row direction, while the input of the second model parallel matrix multiplication requires the full amount of tensors. The framework will automatically insert the AllGather operator to implement the arrangement transformation, such as Figure 4 As shown in the figure, in Example 2, the output of the first model parallel matrix multiplication is split in the column direction, while the input of the second data parallel matrix multiplication is split in the row direction. The framework will automatically insert a communication operator equivalent to the AlltoAll operation in the set communication to implement the arrangement transformation, such as Figure 5As shown in the figure, in Example 3, the output segmentation method of the first hybrid parallel matrix multiplication is consistent with the input segmentation method of the second hybrid parallel matrix multiplication, so there is no need to introduce a rearrangement transformation. However, since the related dimensions of the two inputs are segmented in the second matrix multiplication operation, the AllReduce operator needs to be inserted to ensure the correctness of the operation.

[0050] In summary, distributed operators and tensor arrangement models and tensor arrangement transformations are the basis for automatic parallel implementation. In general, this distributed expression breaks the boundary between data parallelism and model parallelism, and easily realizes hybrid parallelism. From the script level, users only need to construct a single-machine network to express the logic of the parallel algorithm, and the framework will automatically implement the sharding of the entire graph. Sharding strategy search: When the user is familiar with the sharding expression of the operator and manually configures the sharding strategy for the operator, this is the SEMI_AUTO_PARALLEL semi-automatic parallel mode. This method is helpful for manual tuning, but it still has certain debugging difficulties. Users need to master the parallel principle and obtain high-performance parallel solutions based on the network structure, cluster topology and other calculations and analysis. In order to help users accelerate the parallel network training process, based on the semi-automatic parallel mode, the AUTO_PARALLEL automatic parallel mode supports parallel strategy propagation (ShardingPropagation), which can effectively reduce the workload of users manually configuring operator sharding strategies. The algorithm propagates the sharding strategy from the user-configured operator to the unconfigured operator. In order to further reduce the workload of users to manually configure operator segmentation strategies, fully automatic search of segmentation strategies is supported. To this end, a corresponding cost function model (CostModel) is built around the hardware platform to calculate the computational cost (ComputationCost), memory cost (MemoryCost) and communication cost (CommunicationCost) of a certain amount of data and a certain operator under different segmentation strategies. Then, through dynamic programming algorithms (DynamicProgramming) or recursive programming algorithms (RecursiveProgramming), with the memory limit of a single card as a constraint, a segmentation strategy with better performance is efficiently searched. Distributed automatic differentiation: In addition to paying attention to forward network communication, traditional manual model segmentation also needs to consider the parallel operation of the network reverse. MindSpore packages the communication operation into an operator and uses the original automatic differentiation operation of the framework to automatically generate the reverse of the communication operator. Therefore, even when performing distributed training, users only need to pay attention to the forward propagation of the network, truly realizing fully automatic parallel training. Heterogeneous parallel training method is to analyze the memory usage and computing density of operators on the graph, divide the operators with huge memory consumption or suitable for CPU logic processing into CPU subgraphs, and divide the operators with small memory consumption and computing intensiveness into hardware accelerator subgraphs. The framework coordinates different subgraphs to perform network training, so that subgraphs on different hardware and without dependencies can be executed in parallel.

[0051] As a specific implementation method of the improvement, refer to Figure 6 As shown, the steps of performing graph-computation fusion in the compilation optimization layer are as follows:

[0052] Step a: Expand the composite operator into basic operators and form a Composite subgraph to facilitate subsequent cross-boundary optimization and operator splitting;

[0053] Step b: Aggregate adjacent basic operators or Composite subgraphs to form a larger aggregate subgraph, which is convenient for subsequent cross-boundary optimization and operator splitting;

[0054] Step c, based on the aggregated subgraphs obtained in steps a and b, a large number of cross-boundary optimizations are performed, such as algebraic simplification and common sub-expression extraction;

[0055] In step d, operators are split into subgraphs of aggregated computations based on computational features and fusion operator performance, and then the optimized computational graphs are passed to MindSporeAKG in the form of subgraphs to continue backend optimization and generate target code. Through the settings of steps a to d above, the fusion operator is accelerated and optimized. In scenarios such as deep neural network training, graph-computing fusion optimization can bring exponential performance improvements. However, with the continuous enhancement of graph-computing fusion capabilities, the development of fusion operators has become a bottleneck for further improving graph-computing fusion capabilities. The automatic generation technology of fusion operators can solve the problem of high programming thresholds for developing fusion operators based on DSA, allowing programmers to focus on the implementation logic of operators during operator development without having to pay attention to backend optimization, greatly improving their development efficiency. Especially for scenarios with complex backend hardware architectures and the presence of complex operators and fusion operators, automatic operator generation technology is even more critical.

[0056] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A data classification and grading system, characterized by: include: The model layer provides users with out-of-the-box functions. This layer includes pre-built models and development kits as well as expansion libraries in hot research areas such as graph neural networks, deep probabilistic programming, and scientific computing libraries. The expression layer provides users with interfaces for AI model development, training, and reasoning. It supports users to develop and debug neural networks using native Python syntax. At the same time, this layer provides a unified C++ / Python interface for all scenarios during the production and deployment stages. The compilation optimization layer, as the core of the AI ​​framework, uses the unified intermediate expression of all scenarios as a medium to compile the front-end expression into a low-level language with higher execution efficiency. It also performs global performance optimization, including hardware-independent optimizations such as automatic differentiation and algebraic simplification, as well as hardware-related optimizations such as graph-computing fusion and operator generation. The operation layer connects to and calls the underlying hardware operators according to the compilation and optimization results of the compilation and optimization layer. At the same time, it supports "end-edge-cloud" AI collaboration including federated learning through the unified runtime architecture of "end-edge-cloud".

2. The data classification and grading system according to claim 1, characterized in that: The compilation optimization layer also includes automatic hybrid parallel optimization steps, specifically including data parallelism, automatic parallelism and heterogeneous parallelism. The specific steps of parallelism are as follows: Step 1: Before starting parallel training each time, initialize the communication resources by calling the mindspore.communication.init interface and automatically create a global communication group WORLD_COMM_GROUP; Step 2: Split the data set in the sample dimension and send it to different cards; Step 3, perform network mapping, wherein the writing method of the network mapping is the same as that of a single-machine network; Step 4: perform gradient calculation and insert distributed operators after gradient calculation to implement gradient aggregation operations between cards; Step 5: Enter the parameter update step with the same gradient value calculated in step 4.

3. The data classification and grading system according to claim 2, characterized in that: The specific steps of splitting the data set in step 2 in the sample dimension and sending it to different cards are as follows: Step 21, through the two parameters num_shards and shard_id provided by the mindspore.dataset module, which are available in all dataset loading interfaces, the dataset is split into multiple parts and sampled in a circular manner to collect batch-sized data onto respective cards. When the amount of data is insufficient, sampling will start from the beginning.

4. The data classification and grading system according to claim 3, characterized in that: After the distributed operator is inserted in step 4, the tensor is segmented and modeled based on the distributed operator to indicate how the input and output tensors of an operator are distributed to each card in the cluster, thereby obtaining a tensor arrangement model.

5. The data classification and grading system according to claim 4, characterized in that: In step 4, each operator has a splitting strategy, which consists of two tuples, representing the splitting forms of input and weight respectively, where the elements in the tuple correspond to the tensor dimensions one by one, 2^N is the number of splits, and 1 means no splitting. When the user wants to express a data parallel splitting strategy, that is, the batch dimension of the input is split, and other dimensions are not split, it can be expressed as strategy=((2^N,1,1),(1,1,1)); when expressing a model parallel splitting strategy, that is, the non-batch dimension splitting of weight, taking the channel dimension splitting as an example, other dimensions are not split, which can be expressed as strategy=((1,1,1),(1,1,2^N)); when expressing a hybrid parallel splitting strategy, one of the splitting strategies is strategy=((2^N,1,1),(1,1,2^N)).

6. The data classification and grading system according to claim 5, characterized in that: The steps of performing graph-computation fusion in the compilation optimization layer are as follows: Step a: Expand the composite operator into basic operators and form a Composite subgraph to facilitate subsequent cross-boundary optimization and operator splitting; Step b: Aggregate adjacent basic operators or Composite subgraphs to form a larger aggregate subgraph, which is convenient for subsequent cross-boundary optimization and operator splitting; Step c, based on the aggregated subgraphs obtained in steps a and b, a large number of cross-boundary optimizations are performed, such as algebraic simplification and common sub-expression extraction; In step d, operators are split into subgraphs based on computational features and fusion operator performance. The optimized computational graphs are then passed to MindSporeAKG in the form of subgraphs to continue backend optimization and generate target code.

Citation Information

Cited By

  • MindSpore-based operator processing method and device

    CN120688561A