A dynamic data partitioning method for DNN operator library

Abstract

The present invention discloses a dynamic data segmentation method for a DNN operator library, comprising the following steps: S1, a user calls a desired operator and passes in a set of parameters consisting of multiple integers as input to S2; S2, based on the parameter size obtained in S1 and the parameter applicable range of the algorithm, the set of parameters is divided into multiple groups of parameters, and a different algorithm is selected for processing each group of parameters; S3, the parameters are obtained from S2, and according to the characteristics of the algorithm, a priority is set for the data of each dimension [N, H, W, C] in the parameters; S4, under the premise of ensuring sufficient memory, the size of the blocks of [N, H, W, C] is increased in descending order according to the priority of each data in the parameters; S5, the dynamically selected data blocks are passed to the corresponding algorithm, the algorithm is called, and a result is obtained. The present invention ensures the optimal performance of the operator, does not require the user to worry about how to choose among many algorithms, and has a good user experience.

Description

Technical Field

[0001] The present invention relates to a DNN operator library dynamic data blocking method, belonging to the field of deep learning technology. Background Art

[0002] The rapid development of artificial intelligence (AI) in recent years has profoundly altered global development patterns and human lifestyles. To seize this significant strategic opportunity, countries are striving to build first-mover advantages. Currently, the mainstream technological path for AI development is data intelligence driven by model learning. The DNN operator library is the foundation of deep learning for AI. Each operator has multiple implementation algorithms, each with its own strengths under varying use case parameters. While each architecture offers powerful computing capabilities, bandwidth resources are currently insufficient to meet these demands, making bandwidth extremely valuable. Coordinating these numerous algorithms to ensure that when users use an operator, task partitioning is optimized in real time based on user parameters, enabling the optimal algorithm to be invoked based on the current system state, is crucial for fully utilizing system bandwidth resources and plays a significant role in improving operator performance.

[0003] Currently, the DNN operator library uses a variety of optimization algorithms for each operator. When using it, users cannot dynamically adjust the task block size in real time, nor can they select the optimal task block in real time. They can only use fixed-size task blocks and fuzzy call the possible optimal algorithm. This leads to a poor user experience, is not user-friendly, and cannot fully utilize the optimal performance of the operator.

[0004] The DNN operator library is the underlying computing power support library for artificial intelligence. Each operator has multiple implementation algorithms, and each algorithm offers multiple data partitioning options. User parameters are given in real time. How to use these parameters to optimally invoke the multi-core algorithms in the operator library and maximize operator effectiveness is a pressing issue. Summary of the Invention

[0005] The purpose of the present invention is to provide a dynamic data segmentation method for a DNN operator library, which ensures the optimal performance of the operator, does not require users to worry about how to choose among numerous algorithms, and provides a good user experience.

[0006] To achieve the above object, the technical solution adopted by the present invention is to provide a DNN operator library dynamic data segmentation method, comprising the following steps:

[0007] S1, the user calls the required operator and passes in a set of parameters consisting of multiple integers as input to S2;

[0008] S2. Divide a set of parameters into multiple groups of parameters based on the parameter size obtained in S1 and the parameter applicable range of the algorithm, and select a different algorithm for processing each group of parameters. Divide the parameters into sizes suitable for processing by each algorithm based on the characteristics of the algorithm, and assign the parameters to different algorithms.

[0009] S3. Obtain parameters from S2 and set priorities for the data in each dimension [N, H, W, C] of the parameters according to the characteristics of the algorithm. The priorities are determined by the way they are stored consecutively in memory. The priority relationship is: C>W>H>N. Increasing the value of a dimension is more conducive to improving efficiency, and its priority is higher.

[0010] S4. Calculate the remaining system resources and initialize the data block size for each dimension. Under the premise of ensuring sufficient memory, increase the block size of [N, H, W, C] in descending order of the priority of each data in the parameters.

[0011] S5. Pass the dynamically selected data blocks to the corresponding algorithm, call the algorithm, and obtain the results.

[0012] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0013] The present invention provides a dynamic data blocking method for a DNN operator library. The method can preferentially select an operator algorithm based on parameters and the current memory usage of the system, and adjust the corresponding data blocking size, thereby ensuring the optimal utilization of system bandwidth resources, providing sufficient data for many cores with powerful computing power, improving the utilization rate of bandwidth resources, and ensuring the optimal performance of operators. Users do not need to worry about how to choose among numerous algorithms, and thus have a good user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Attachment Figure 1 This is a schematic diagram of dynamic data segmentation. DETAILED DESCRIPTION

[0015] Embodiment: The present invention provides a DNN operator library dynamic data segmentation method, which specifically includes the following steps:

[0016] S1, the user calls the required operator and passes in a set of parameters consisting of multiple integers as input to S2;

[0017] S2. Based on the characteristics of the algorithm itself, different algorithms are suitable for different parameter ranges. Therefore, according to the parameter size obtained in S1 and the applicable range of the algorithm parameters, a set of parameters is divided into multiple groups of parameters, and a different algorithm is selected for each group of parameters to process. There are multiple implementation algorithms for each operator, which are generally divided into methods suitable for processing larger parameters and methods with smaller parameters. According to the characteristics of the algorithm, the parameters are divided into sizes suitable for each algorithm to process, and the parameters are allocated to different algorithms to ensure the parallelism of the algorithm;

[0018] S3. Obtain parameters from S2 and set priorities for the data in each dimension [N, H, W, C] of the parameters according to the characteristics of the algorithm. The priorities are determined by the way they are stored consecutively in memory. The priority relationship is: C>W>H>N. Increasing the value of a dimension is more conducive to improving efficiency, and its priority is higher.

[0019] Because different operators implement multiple algorithms, each algorithm has the characteristic of outperforming other algorithms under different parameters. This step ensures that the optimal algorithm under these parameters can be selected;

[0020] S4. Calculate the remaining system resources and initialize the data block size for each dimension. Under the premise of ensuring sufficient memory, increase the block size of [N, H, W, C] in descending order of the priority of each data in the parameters.

[0021] This step ensures that after the blocks are divided, the system resources can be fully utilized to achieve the best performance of the algorithm;

[0022] S5. Pass the dynamically selected data blocks to the corresponding algorithm, call the algorithm, and obtain the results.

[0023] The above embodiment is further explained as follows:

[0024] The present invention provides users with a good encapsulation before implementing algorithms with numerous operators. Users only need to specify the required operators and assign parameters. The best algorithm can be selected based on the user's parameters, and the optimal data block can be dynamically selected to ensure real-time maximum utilization of the system's bandwidth resources and give full play to the best performance of the operators.

[0025] In response to the numerous implementations of different operator algorithms in the DNN operator library, a dynamic data partitioning method is proposed. This method organically combines user parameters with the numerous algorithms in the operator library, can adjust the partition size in real time according to the user parameters, and can also divide the parameters into tasks, assigning different tasks to the corresponding optimal algorithms based on the task scale and parameter characteristics. Different algorithms for the same operator can be executed in parallel. Compared with assigning tasks to a single algorithm, this method improves the parallelism of the algorithms and is more in line with the architectural characteristics of heterogeneous many-core processors.

[0026] At the same time, for different task parameters, the present invention can maximize the use of memory resources according to the current system memory usage, adjust the task block size in real time, ensure the maximum use of the system's bandwidth resources, and provide reliable protection for the performance of the algorithm.

[0027] 1. The user calls the required operator and passes in relevant parameters.

[0028] 2. Divide tasks into groups based on parameters and algorithm characteristics, dividing them into sizes suitable for algorithm processing. Because different algorithms are suitable for processing different parameters, this invention divides a set of parameters into multiple groups based on parameter size and algorithm, and then selects a different algorithm to process each group of parameters, thus ensuring the parallelism of the algorithm.

[0029] 3. Select the best algorithm based on the task size. Different algorithms are suitable for different parameters. Therefore, it is necessary to select the best algorithm for calculation processing based on the parameter size.

[0030] 4. Different algorithms have different characteristics. Based on the characteristics of the algorithm and the current memory usage, the data is dynamically segmented to ensure that the size of the data blocks is most suitable for the current system status and can fully utilize the system's bandwidth resources.

[0031] 5. Pass the dynamically selected data blocks to the corresponding algorithm, call the algorithm, and get the results.

[0032] When using the above-mentioned DNN operator library dynamic data blocking method, it can select the operator algorithm based on user parameters and the current memory usage of the system, and adjust the corresponding data block size, ensuring the optimal utilization of system bandwidth resources, providing sufficient data for the multi-core with powerful computing power, improving the utilization rate of bandwidth resources, and ensuring the optimal performance of the operator. Users do not need to worry about how to make decisions among many algorithms, and have a good user experience.

[0033] To facilitate a better understanding of the present invention, the following briefly explains the terms used herein:

[0034] Operator: A function that implements a specific function.

[0035] DNN: Deep neural network, the basis of deep learning.

[0036] DNN Operator Library: A mathematical library that implements common DNN operators.

[0037] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A dynamic data partitioning method for a DNN operator library, characterized in that: The following steps are involved: S1, the user calls the required operator and passes in a set of parameters consisting of multiple integers as input to S2; S2. Divide a set of parameters into multiple groups of parameters based on the parameter size obtained in S1 and the parameter applicable range of the algorithm, and select a different algorithm for processing each group of parameters. Divide the parameters into sizes suitable for processing by each algorithm based on the characteristics of the algorithm, and assign the parameters to different algorithms. S3. Obtain parameters from S2 and set priorities for the data of each dimension [N, H, W, C] in the parameters according to the characteristics of the algorithm. The priority is determined by the way it is stored continuously in memory. The more the value of a dimension is conducive to improving efficiency, the higher its priority. The priority relationship is: C>W>H>N; S4. Calculate the remaining system resources and initialize the data block size for each dimension. Under the premise of ensuring sufficient memory, increase the block size of [N, H, W, C] in descending order of the priority of each data in the parameters. S5. Pass the dynamically selected data blocks to the corresponding algorithm, call the algorithm, and obtain the results.

Images