Data chunking method, computing method, and computing device

Abstract

The application relates to a data block method, a calculation method and a calculation device. The method comprises the following steps: obtaining user problem description information; wherein the user problem description information comprises attribute information of to-be-blocked data, a maximum number of calculation cores for calculating the to-be-blocked data in an NPU and alignment conditions; the alignment conditions are used for restricting the size of a data block and the starting address of the data block; a target block constraint model is obtained based on the attribute information; and a target solver is used for solving based on the attribute information, the maximum number, the alignment conditions and the target block constraint model, so as to obtain a block result of the to-be-blocked data. Thus, a data block algorithm which needs to be manually written by a user can be converted into a process which is automatically solved by a data block method provided by an embodiment of the application and only needs to describe a problem, so that the human consumption for writing the block algorithm can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to data segmentation methods, calculation methods and computing devices. Background Technology

[0002] Operator programming is a dedicated programming language for the NPU (neural network processing unit), designed to fully leverage the parallelism of the Ascend NPU hardware to accelerate data-intensive computational scenarios such as AI (artificial intelligence) model training and inference. Related operator programming provides various parallel capabilities, including multi-core parallelism, vector parallelism, and pipelining parallelism.

[0003] The relevant operators can run on AI COREs (computing cores) to perform related operations. An NPU often has multiple AI COREs. Operator programming uses SPMD (single program multiple data) to achieve multi-core parallel computing. Specifically, an operator can run simultaneously on multiple AI COREs. Although each AI CORE runs the same program, the data it processes is different, thus achieving block-based parallel computing.

[0004] However, when using multiple AI cores for parallel computing, the related technologies require developers to manually write data partitioning algorithms in the operators, which consumes a lot of manpower and results in low efficiency in generating partitioning algorithms in the related technologies. Summary of the Invention

[0005] This application provides a data segmentation method, a calculation method, and a computing device, which can improve the generation efficiency of the segmentation algorithm.

[0006] According to a first aspect of the embodiments of this application, a data segmentation method is provided, the method being applied to a CPU in a server, the server further comprising an NPU, the method comprising:

[0007] Obtain user problem description information; the user problem description information includes: attribute information of the data to be partitioned, the maximum number of computing cores in the NPU used to compute the data to be partitioned, and alignment conditions; the alignment conditions are used to constrain the size of the data block and the starting address of the data block;

[0008] The target block constraint model is obtained based on attribute information. The target block constraint model is used to constrain the data to be divided into first-class data blocks and second-class data blocks based on alignment conditions. The target block constraint model includes first-class data block constraints, second-class data block constraints, and optimization objectives. The first-class data block constraints are used to constrain the number and length of first-class data blocks, the second-class data block constraints are used to constrain the number and length of second-class data blocks, and the optimization objective is to minimize the number of computing cores occupied when the length of the first-class data blocks is minimized.

[0009] Based on attribute information, maximum quantity, alignment conditions, and target partitioning constraints, the model is solved using a target solver to obtain the partitioning results of the data to be partitioned.

[0010] The attribute information of the data to be segmented includes: data dimension, data shape, and data type.

[0011] This solution, by obtaining user problem description information, can transform the data partitioning algorithm that originally required manual writing by the user into a process where the data partitioning method provided in this application embodiment automatically solves the problem simply by requiring the user to describe it, which can effectively reduce the manpower consumption of writing partitioning algorithms.

[0012] In one possible implementation provided in the embodiments, the target block constraint model includes a one-dimensional data block constraint model and a two-dimensional data block constraint model;

[0013] Obtaining a block-based constraint model based on attribute information includes:

[0014] Based on the data dimension of the data to be partitioned, when the data to be partitioned is determined to be one-dimensional, a one-dimensional data partitioning constraint model is obtained; or, when the data to be partitioned is determined to be two-dimensional, a two-dimensional data partitioning constraint model is obtained.

[0015] This solution can simultaneously meet the segmentation requirements for both one-dimensional and two-dimensional data by constructing one-dimensional data segmentation constraint models and two-dimensional data segmentation constraint models respectively.

[0016] In one possible implementation provided by the embodiment, when the data to be segmented is one-dimensional data, the segmentation result is quadruple data, which includes: the length of the first type of data block, the number of the first type of data blocks, the length of the second type of data block, and the number of the second type of data blocks.

[0017] This scheme uses quadruple data to describe the block results, which makes it easier for the NPU to quickly split the one-dimensional block data based on the quadruple data, thereby improving data processing efficiency.

[0018] In one possible implementation provided by the embodiment, when the data to be segmented is two-dimensional data, the segmentation result is octet data, which includes: the length of the first type of data block on the horizontal axis, the number of the first type of data blocks on the horizontal axis, the length of the second type of data block on the horizontal axis, the number of the second type of data blocks on the horizontal axis, the length of the first type of data block on the vertical axis, the number of the first type of data blocks on the vertical axis, the length of the second type of data block on the vertical axis, and the number of the second type of data blocks on the vertical axis.

[0019] This scheme uses octet data to describe the block results, which makes it easier for the NPU to quickly divide the two-dimensional block data based on the octet data, thereby improving data processing efficiency.

[0020] In one possible implementation provided in the embodiments, the method further includes:

[0021] Store the block results into the target structured data;

[0022] In response to the NPU's call request, send the target structured data to the NPU.

[0023] After obtaining the segmented results using the methods described above, this scheme can store the segmented results in a target structured data set. This target structured data set can be a TilingContext. The CPU can receive segmented result call requests from the NPU, and in response to these requests, the CPU sends the TilingContext to the NPU, thus facilitating data transfer between the CPU and the NPU.

[0024] In one possible implementation provided in the embodiments, the method further includes:

[0025] Determine whether the cache contains a target chunking strategy that matches the attribute information;

[0026] If no target partitioning strategy matching the attribute information exists in the cache, execute the step of obtaining the partitioning constraint model based on the attribute information;

[0027] Alternatively, if a target chunking strategy that matches the attribute information exists in the cache, the target chunking strategy is used as the chunking result for the data to be chunked.

[0028] For repeated calculations of the same shape, this scheme can directly reuse historical block-based schemes, thereby avoiding repeated runs and improving processing efficiency.

[0029] In one possible implementation provided in the embodiments, determining whether the cache contains a target block strategy that matches the attribute information includes:

[0030] Generate hash keys based on attribute information;

[0031] The cache is searched to see if a target hash value that matches the hash key exists; the cache stores a table showing the relationship between keywords and hash values, where the hash value represents the generated block splitting strategy.

[0032] When a target hash value matching the hash key exists in the cache, determine that the cache contains a target chunking strategy that matches the attribute information.

[0033] This scheme uses hash keys to match target hash values ​​in the cache, enabling fast retrieval of cached data.

[0034] According to a second aspect of the embodiments of this application, a computing method is provided, the method being applied to an NPU in a server, the method comprising:

[0035] Obtain the target structured data sent by the CPU in the server; wherein, the target structured data includes the block division result of the data to be block divided, the block division result includes a first type of data block and a second type of data block, the first type of data block satisfies the alignment condition of the NPU, and the second type of data block includes data blocks other than the first type of data block in the block division result;

[0036] Based on the block division results, the computing cores in the NPU are divided into a first computing core for computing the first type of data blocks and a second computing core for computing the second type of data blocks.

[0037] The first type of data block is calculated by loading the operator kernel through the first computing core, and the second type of data block is calculated by loading the operator kernel through the second computing core to obtain the calculation result;

[0038] Send the calculation results to the CPU.

[0039] This scheme utilizes a data partitioning optimization algorithm and, by designing appropriate optimization objectives, can automatically tune operator performance and core usage. This allows the NPU to improve data processing efficiency when performing parallel operations across multiple computing cores.

[0040] According to a third aspect of the embodiments of this application, a data segmentation apparatus is provided. The apparatus may be a CPU in a server, and the server further includes an NPU. The apparatus includes:

[0041] The information acquisition module is used to acquire user problem description information, which includes: attribute information of the data to be partitioned, the maximum number of computing cores in the NPU used to compute the data to be partitioned, and alignment conditions; the alignment conditions are used to constrain the size of the data block and the starting address of the data block.

[0042] The block constraint module is used to obtain the target block constraint model based on attribute information. The target block constraint model is used to constrain the data to be divided into first-class data blocks and second-class data blocks based on alignment conditions. The target block constraint model includes first-class data block constraints, second-class data block constraints, and optimization objectives. The first-class data block constraints are used to constrain the number and length of first-class data blocks, the second-class data block constraints are used to constrain the number and length of second-class data blocks, and the optimization objective is to minimize the number of computing cores occupied when the length of the first-class data blocks is minimized.

[0043] The block result generation module is used to solve the block results of the data to be partitioned by using the target solver based on attribute information, maximum quantity, alignment conditions and target block constraint model.

[0044] According to a fourth aspect of the embodiments of this application, a computing device is provided, which may be an NPU in a server, the method comprising:

[0045] The data acquisition module is used to acquire the target structured data sent by the CPU in the server. The target structured data includes the block division result of the data to be divided. The block division result includes a first type of data block and a second type of data block. The first type of data block satisfies the alignment condition of the NPU. The second type of data block includes data blocks other than the first type of data block in the block division result.

[0046] The computation core partitioning module is used to divide the computation cores in the NPU into a first computation core for computing the first type of data blocks and a second computation core for computing the second type of data blocks, based on the partitioning results.

[0047] The operator kernel loading module is used to load the operator kernel through the first computing core to calculate the first type of data block, and to load the operator kernel through the second computing core to calculate the second type of data block, so as to obtain the calculation result;

[0048] The result sending module is used to send the calculation results to the CPU.

[0049] According to a fifth aspect of the embodiments of this application, a computing device is provided. The computing device includes: a CPU and an NPU;

[0050] A memory for storing executable instructions for the CPU and NPU; wherein the CPU is configured to execute instructions to implement the method of the first aspect, and the NPU is configured to execute instructions to implement the method of the second aspect.

[0051] According to a sixth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described in the embodiments of this application.

[0052] According to a seventh aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described in the embodiments of this application. Attached Figure Description

[0053] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0054] Figure 1 A schematic diagram of a scenario system architecture provided for an exemplary embodiment of this application;

[0055] Figure 2 A schematic diagram of the engineering operator development process provided for an exemplary embodiment of this application;

[0056] Figure 3 A schematic diagram of a data segmentation process provided on the CPU side, as provided in an exemplary embodiment of this application;

[0057] Figure 4 A flowchart illustrating a data segmentation method provided for an exemplary embodiment of this application;

[0058] Figure 5 A flowchart illustrating a data segmentation method provided as another exemplary embodiment of this application;

[0059] Figure 6 A schematic diagram of two-dimensional block AI CORE scheduling provided for an exemplary embodiment of this application;

[0060] Figure 7 A flowchart illustrating a calculation method provided in yet another exemplary embodiment of this application;

[0061] Figure 8 A schematic diagram of two-dimensional data segmentation results provided as another exemplary embodiment of this application;

[0062] Figure 9 A schematic block diagram of the functional modules of a data segmentation device provided in an exemplary embodiment of this application;

[0063] Figure 10 A schematic block diagram of the functional modules of a data computing device provided in an exemplary embodiment of this application;

[0064] Figure 11 A structural block diagram of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0065] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0066] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0067] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0068] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0069] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0070] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0071] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0072] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application; other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0073] To improve the efficiency of data partitioning algorithm generation, this application modeles the data partitioning problem as an integer programming algorithm. This transforms the original manual coding of data partitioning algorithms into an automatic solution where developers only need to input the user's problem description information, and the data partitioning method provided in this application can solve the problem automatically. This effectively reduces the human effort required to write data partitioning algorithms. Furthermore, the integer programming algorithm used in this application is an optimization algorithm. By designing a suitable optimization objective, mathematical algorithms can automatically tune the operator performance and the number of computing cores used in the NPU.

[0074] Specifically, such as Figure 1 As shown, Figure 1 This application provides a schematic diagram of a scenario system architecture. The system may include a terminal 10 and a server 20. The server 20 is equipped with a CPU (central processing unit) 21 and an NPU 22, wherein:

[0075] Terminal 10 is used to receive user problem description information input by the user and send the user problem description information to server 20. The user problem description information describes the user's computational needs for the segmented data.

[0076] For example, the user's problem description information may include: Calculate the cumulative sum of 43 float data, with a computing resource of up to 4 computing cores, and require that the starting address of all blocks be 32 bytes aligned (i.e., the first element address is known to be 32 bytes aligned, so this requirement is equivalent to the block length being 32 bytes aligned). What is the optimal block scheduling scheme?

[0077] Server 20 receives user problem description information sent by terminal 10, and obtains a data segmentation algorithm through CPU 21. It then segments the data to be calculated into multiple large and small blocks to obtain a segmentation result. CPU 21 sends this segmentation result to NPU 22. Based on the result, NPU 22 allocates a corresponding computing core to each data block. Each computing core performs parallel computation on each data block by calling corresponding operators and obtains the computation result. Server 20 also sends the obtained computation results to terminal 10.

[0078] The large data block refers to the block data that meets the above alignment conditions, such as the block data that meets the 32-byte alignment. The small data block refers to the data in the block result other than the large data block. For example, the small data block does not meet the 32-byte alignment, but both the large data block and the small data block meet the 32-byte alignment of the starting address.

[0079] Specifically, CPU21 acquires user description information, which may include attribute information of the data to be partitioned, the maximum number of computation cores in the NPU used to compute the data to be partitioned, and alignment conditions. The alignment conditions constrain the size of the data blocks and their starting addresses. For example, the alignment condition might require large partitioned data blocks to be 32-byte aligned, and all block start addresses to be 32-byte aligned. The attribute information of the partitioned data may include: data dimension, data shape, and data type. The data dimension can be one-dimensional or two-dimensional; for one-dimensional data, the data shape can be [s0, s1, ..., s...]. n [], corresponding to two-dimensional data, the data shape can be [M, N], such as [18432, 7168]; data type, such as float data, etc.

[0080] CPU21 can call the software interface provided in this application embodiment to solve for the optimal block parameters based on the user description information, and set the number of computing cores to be started according to the solved parameters, and transmit the optimal block parameters and the number of computing cores as the block result to NPU22.

[0081] The NPU22 receives the block results from the CPU21 and divides the computing cores into cores for computing large blocks of data and cores for computing small blocks of data. It then calls an aligned computing interface for large blocks of data and either an aligned or unaligned computing interface for small blocks of data, depending on the situation. Specifically, the aligned computing interface is used to call the computing cores for large blocks of data, while the unaligned computing interface is used to call the computing cores for small blocks of data.

[0082] In this embodiment, the computing cores in NPU22 used for computing large blocks of data and those used for computing small blocks of data will each make single operator API (application programming interface) calls to perform parallel operations on each block of data. Furthermore, the data segmentation method provided in this embodiment can be used as part of an operator; CPU21 can call the API interface through a single operator to perform segmentation processing on the data to be segmented, thereby obtaining the segmentation results.

[0083] Specifically, Figure 2 This is a schematic diagram of the engineering operator development process provided in the embodiments of this application, combined with Figure 2 As shown, the process may include:

[0084] (1) Environmental preparation.

[0085] In this embodiment, the CANN (compute architecture for neural networks) software is first installed, including the compiler and runtime library, and the NPU driver and firmware are configured.

[0086] The purpose of creating an operator project is to initialize the project structure, including using relevant tools to generate project templates, including: operator prototype definitions, Kernel-side (i.e., NPU22-side) code, Host-side (i.e., CPU21-side) Tiling (blocking algorithm) code, and compilation scripts.

[0087] (2) Operator implementation.

[0088] The purpose of defining an operator prototype is to declare the operator interface specification, which includes defining: the number and type of input / output tensors, operator properties (such as convolution kernel size), and data arrangement constraints.

[0089] The purpose of implementing the Kernel-side operators is to write NPU execution code, using relevant programming languages ​​to implement: data movement (from Global to Local memory), AI Core computation logic (vector instruction optimization), and result write-back.

[0090] The purpose of the tiling implementation on the host side is to design a data chunking strategy (i.e., a data chunking method), which includes implementing on the host side: obtaining global data attributes from TilingContext, designing a data chunking method (such as requiring chunked data to meet 32-byte alignment), and setting TilingData (chunking method), BlockDim (number of kernel function instances) / Workspace (memory space).

[0091] (3) Compile and deploy.

[0092] The purpose of operator project compilation and deployment is to generate executable operators. The specific process may include compiling code, generating .sh files, generating binary files, packaging into .so libraries, and deploying to the NPU environment.

[0093] (4) Operator call.

[0094] The purpose of single operator API calls is to verify operator functionality, including: initializing the programming environment, loading custom operator libraries, performing asynchronous computations, and verifying the correctness of results.

[0095] In this embodiment, data partitioning and resource scheduling can be resolved on the Host side, while AICore hardware-level optimization can be focused on the Kernel side. Tiling serves as a bridge connecting the Host and Kernel, ensuring that data partitioning meets NPU hardware constraints, i.e., alignment conditions.

[0096] In the embodiments provided in this application, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the data segmentation process provided on the CPU side in an embodiment of this application. The data segmentation process may include the following steps:

[0097] Step 310: Obtain the input, output, and attribute information of the operator from the TilingContext.

[0098] In this embodiment, during operator development, all key information (such as input tensor shape, output tensor specifications, and operator-defined attributes) is uniformly obtained through the `TilingContext*context` object, providing a data foundation for subsequent block-based decisions. Here, `TilingContext*context` is a pointer to a structure.

[0099] Step 320, Tiling is invoked.

[0100] In this embodiment, the block-splitting method provided in this application embodiment can be used as Tiling. By decomposing large-scale computing tasks into multiple data blocks (Tile), it can adapt to the parallel processing process of NPU hardware. This can solve the problem that a single core cannot process ultra-large tensors at once and improve parallel efficiency.

[0101] Specifically, step 320 may include steps 321 and 322.

[0102] Step 321 describes the configuration and alignment conditions for running the block method.

[0103] In the embodiment, the specific configuration rules of the block partitioning algorithm (such as block size, number, etc.) need to be clearly defined. The alignment condition is to ensure that each block meets the hardware alignment requirements (such as 32-byte alignment), otherwise it will lead to DMA (Direct Memory Access) transfer failure or performance degradation.

[0104] For example, for half-precision floating-point data (2 bytes / element), the number of elements in each block must be a multiple of 16 (because 16 elements × 2 bytes = 32 bytes).

[0105] Step 322: Call the interface of the block method.

[0106] In this embodiment, development can be simplified by calling a pre-generated generic chunking interface. Developers only need to obtain the input / output / attributes from the TilingContext, the alignment rules (alignment conditions) of the user input, and related configurations, and then automatically generate the chunking scheme by calling this interface. This avoids repeatedly implementing the chunking logic and ensures that the chunking conforms to the hardware constraints of the NPU.

[0107] Step 330: Set TilingData, BlockDim, TilingKey, and Workspace size to TilingContext.

[0108] In this embodiment, TilingData is used to store the specific data of the data partitioning method (i.e., the partitioning algorithm) (such as the starting position, size, and step size of each partition). BlockDim is used to define the dimension of parallel computing (such as how many AICores are used for parallel processing). TilingKey is used as a unique key value to identify different partitioning schemes, and is used to dynamically select the optimal partitioning strategy at runtime. workspace size is the size allocated to the operator's temporary memory space (Workspace), used to store intermediate computing results.

[0109] This application's embodiments can adapt tensors of arbitrary shapes to limited hardware resources (such as 8 AI Cores) through a block-based mechanism. Hardware alignment requirements are enforced, such as blocks needing to be 32-byte aligned; otherwise, DMA cannot be used efficiently or runtime errors may occur. Furthermore, the generic interface encapsulates the underlying complexity, allowing developers to focus solely on business logic.

[0110] In the embodiments provided in this application, such as Figure 4 As shown, Figure 4 This is a schematic diagram of a data segmentation method provided in an embodiment of this application. The segmentation process may include the following steps:

[0111] Step 410, Host side preparation.

[0112] In this embodiment, the step includes:

[0113] Initialize the environment: AscendCL is initialized, and resources are requested.

[0114] Memory management:

[0115] Request input / output memory and calculate workspace size.

[0116] Data transfer: Copy the input data to the NPU.

[0117] Step 420, CPU block decision.

[0118] Specifically, step 420 may include:

[0119] Step 421: Determine if the cache is hit. If the cache is hit, execute step 422 to return the cached block partitioning scheme; otherwise, execute step 423 to run the data block solver.

[0120] Step 424: Add the chunking scheme to the cache.

[0121] In this embodiment, the system can check in the cache whether a partitioning result corresponding to the data to be partitioned already exists. If it does, it can be retrieved directly without needing to solve it again through the solver. If the partitioning result corresponding to the data to be partitioned does not exist in the cache, it can be solved again through the solver, and the obtained partitioning scheme can be added to the cache. This partitioning scheme is the data partitioning method or partitioning algorithm.

[0122] The CPU transmits the block partitioning scheme and data addresses to the NPU.

[0123] Step 430: Parallel computation is performed in the NPU and the results are returned.

[0124] The NPU can receive a data partitioning scheme through the data partitioning engine and distribute the scheme to multiple corresponding AI Cores. Each AI Core executes an operator kernel. The NPU then aggregates the computation results from each AI Core and returns the result to the CPU.

[0125] Step 440: CPU resources are released.

[0126] This step may include: waiting for the NPU to complete, obtaining the computation results transferred by the NPU, and releasing resources. Releasing resources includes memory release and environment destruction.

[0127] For repeated calculations of the same shape, the implementation example can directly reuse historical block-based schemes to avoid repeated runs and improve efficiency.

[0128] Based on the above embodiments, this application also provides a data segmentation method, which can be applied to the CPU 21 in the server 20 described above. The server 20 also includes an NPU 22, such as... Figure 5 As shown, the method may include the following steps:

[0129] In step S510, the user's problem description information is obtained.

[0130] The user problem description information includes: attribute information of the data to be partitioned, the maximum number of computing cores in the NPU used to compute the data to be partitioned, and alignment conditions; the alignment conditions are used to constrain the size of the data block and the starting address of the data block.

[0131] In this embodiment, as can be seen from the above embodiments, a user can send a user problem containing a user problem description to the server 20 through the terminal 10. The CPU can use the user problem description information to convert the data block algorithm that originally required the user to write manually into a process that only requires the user to describe the problem and is automatically solved by the data block method provided in this application embodiment.

[0132] In step S520, the target block constraint model is obtained based on the attribute information.

[0133] The target block constraint model is used to constrain the data to be divided into first-class data blocks and second-class data blocks based on alignment conditions. The target block constraint model includes first-class data block constraints, second-class data block constraints, and optimization objectives. The first-class data block constraints are used to constrain the number and length of first-class data blocks, the second-class data block constraints are used to constrain the number and length of second-class data blocks, and the optimization objective is to minimize the number of computing cores occupied when the length of the first-class data blocks is minimized.

[0134] In step S530, the target solver is used to solve the block results of the data to be partitioned based on the attribute information, maximum quantity, alignment conditions and target block constraint model.

[0135] In this embodiment, by obtaining the user's problem description information, the data partitioning algorithm that originally required the user to write manually can be transformed into a process in which the data partitioning method provided by this application embodiment can automatically solve the problem simply by the user describing the problem, which can effectively reduce the manpower consumption of writing partitioning algorithms.

[0136] In this embodiment, the first type of database can be large blocks of data, and the second type of database can be small blocks of data. For example, given the length of the data to be divided, if the data to be divided can be perfectly and evenly divided while satisfying a given alignment condition, then all data blocks will have the same length and satisfy the alignment condition, and all the resulting data blocks will be called large blocks. If the data to be divided cannot be perfectly and evenly divided, then the given data to be divided can be split into several large blocks of data that satisfy the given alignment condition, and the remaining data blocks will be called small blocks of data. In addition, the length of the small blocks of data is limited to be less than the length of the large blocks of data.

[0137] For example, if there are 1999 half-precision floating-point numbers, with a given alignment condition of 32 bytes and a given maximum number of available cores of 8, and the data is divided into 7 blocks of length 256 elements and 1 block of length 207 elements, then the block of length 256 is called the large block of data, and the block of length 207 is called the small block of data.

[0138] For two-dimensional data, the dimension containing the rows of a two-dimensional matrix is ​​usually called the M-axis (horizontal axis), and the dimension containing the columns is called the N-axis (vertical axis). Treating the length of a specific axis as a one-dimensional partitioning problem, we can define large blocks along the M-axis, small blocks along the M-axis, large blocks along the N-axis, and small blocks along the N-axis. After partitioning the two-dimensional matrix along all axes, the two-dimensional block whose rows (M-axis) and columns (N-axis) are respectively large blocks along the M-axis and large blocks along the N-axis is defined as a large two-dimensional block, and all other two-dimensional blocks are small two-dimensional blocks. For example... Figure 6 The above, Figure 6 This is a schematic diagram of two-dimensional block AI CORE scheduling provided in an embodiment of this application. Cores 0, 1, 3, and 4 represent large data blocks, while the other blocks represent small data blocks.

[0139] In this embodiment, the attribute information of the data to be segmented includes: data dimension, data shape, and data type.

[0140] Therefore, in this embodiment, the data to be segmented can be determined to be one-dimensional or two-dimensional based on its data dimension. A target segmentation constraint model can be pre-constructed, which includes a one-dimensional data segmentation constraint model and a two-dimensional data segmentation constraint model. Thus, based on the data dimension of the data to be segmented, when it is determined to be one-dimensional, a one-dimensional data segmentation constraint model is obtained; or, when it is determined to be two-dimensional, a two-dimensional data segmentation constraint model is obtained.

[0141] Specifically, in constructing the one-dimensional data block constraint model, the research problem for one-dimensional data block partitioning is how to achieve load balancing among AI cores while ensuring that large blocks meet given alignment conditions. Given the total length N (in terms of the number of elements) of the input data to be partitioned, the maximum number of computing cores C used for parallel computation in the NPU, and A as the alignment constraint for large blocks (i.e., the alignment condition, which requires that the partitioned large blocks be integer multiples of A), where N, C, and A are all positive integers.

[0142] In the embodiment, the objective to be solved is <N f N t C f C t The quadruples > represent the length N of the large data block. f The length N of the small data block t The number of large data blocks, C f The number of small data blocks C t N f N t C f C t All are natural numbers. The number of small data blocks, C... t Limiting the range to 0 to 1 simplifies kernel-side (i.e., NPU-side) programming without affecting performance. Therefore, the solution space range for this problem can be represented as follows:

[0143] 0≤ N f ≤ N (1)

[0144] 0≤ N t ≤ N (2)

[0145] 0≤ C f ≤ C (3)

[0146] 0≤ C t ≤ 1 (4)

[0147] Due to limitations imposed by the total length of the data blocks to be partitioned and the maximum number of computing cores in the NPU, additional constraints are as follows:

[0148] N f * C f + N t * C t = N (5)

[0149] C f + C t ≤C (6)

[0150] N f ≥Nt +1 (7)

[0151] Describe the length N of a large block of data f The alignment to A is as follows:

[0152] Add an auxiliary variable k, a positive integer, as one of the variables to be solved, and add constraints:

[0153] N f =k * A (8)

[0154] Finally, set the optimization objective to minimize the number of computing cores used while ensuring the smallest possible length of large data blocks:

[0155] minimize N f +(C f +C t ) / C(9)

[0156] Thus, the one-dimensional partitioning problem has been transformed into an integer programming problem with quadratic constraints and linear optimization objectives, namely MIQCP (Mixed Integer Quadratically Constrained Program). The constructed one-dimensional data partitioning constraint model can include the constraints in (1) to (9) above.

[0157] In the embodiment, during the construction of the two-dimensional data block constraint model, for two-dimensional data block, given the two-dimensional shape of the block to be divided as [M,N], and the maximum number of available computational cores as C, for an expression representing the axis length or block length E∈{M f M t N f N t M f *N f M f *N t M t *N f M t *N t The alignment constraint of} is A E M f M t N f N t These represent the lengths of the large data blocks on the M-axis, the small data blocks on the M-axis, the large data blocks on the N-axis, and the small data blocks on the N-axis, respectively. M, N, C, A E All are positive integers.

[0158] The objective is to solve for: These represent the length of the large data block on the M-axis, the length of the small data block on the M-axis, the number of large data blocks on the M-axis, the number of small data blocks on the M-axis, the length of the large data block on the N-axis, the length of the small data block on the N-axis, the number of large data blocks on the N-axis, and the number of small data blocks on the N-axis, respectively. The solution space is described as follows:

[0159] 0≤ M f ≤ M (10)

[0160] 0≤ M t ≤ M (11)

[0161]

[0162] 0≤ N f ≤ N (14)

[0163] 0≤ N t ≤ N (15)

[0164]

[0165] Due to limitations imposed by the total length of the data blocks to be partitioned and the maximum number of computing cores in the NPU, additional constraints are as follows:

[0166]

[0167] M f ≥M t +1 (20)

[0168] N f ≥N t +1 (21)

[0169] Describe a length expression E aligned to A E The method is as follows:

[0170] Add an auxiliary variable k for positive integers E To add a constraint to one of the variables to be solved:

[0171] E = k E * A E (twenty two)

[0172] Finally, the optimization objective is set to minimize the number of computational cores used while ensuring the minimum length of large two-dimensional data blocks:

[0173]

[0174] Thus, the two-dimensional partitioning problem has been transformed into an integer programming problem with quadratic constraints and a quadratic optimization objective, namely MIQCP. The constructed two-dimensional data partitioning constraint model can include the constraints in (10) to (23) above.

[0175] In the embodiment, when the target solver is used to solve the data to be partitioned, based on the attribute information, maximum number of computing cores, alignment conditions and target partitioning constraint model, for example, the MIQCP solver can be used.

[0176] In this embodiment, based on the constructed target block constraint model, the inputs of the target block constraint model are assembled into the input format of an integer programming solver with quadratic constraints and quadratic objectives, and the solver is invoked to solve the problem. The solver can be self-developed based on existing mathematical research reports, or it can use open-source or commercial software interfaces. For one-dimensional data, the solver ultimately returns a quadruple; for two-dimensional data, the solver returns an octet.

[0177] Specifically, when the data to be segmented is one-dimensional, the segmentation result is a quadruple data set, which includes: the length of the first type of data block, the number of the first type of data blocks, the length of the second type of data block, and the number of the second type of data blocks. For example, the quadruple data set in the above embodiment... <N f N t C f C t >

[0178] When the data to be segmented is two-dimensional, the segmentation result is an octet data set. The octet data set includes: the length of the first type of data block on the horizontal axis, the number of the first type of data blocks on the horizontal axis, the length of the second type of data block on the horizontal axis, the number of the second type of data blocks on the horizontal axis, the length of the first type of data block on the vertical axis, the number of the first type of data blocks on the vertical axis, the length of the second type of data block on the vertical axis, and the number of the second type of data blocks on the vertical axis. For example, the octet data set in the above embodiment...

[0179] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps:

[0180] In step S540, the block results are stored in the target structured data.

[0181] In step S550, in response to the NPU's call request, the target structured data is sent to the NPU.

[0182] In this embodiment, after obtaining the segmentation results in the above manner, the segmentation results can be stored in the target structured data. This target structured data can be the TilingContext in the above embodiment. The CPU can receive a segmentation result call request sent by the NPU, and in response to the request, the CPU sends the TilingContext to the NPU.

[0183] Based on the above embodiments, in another embodiment provided in this application, the method can further determine whether the cache contains a target partitioning strategy that matches the attribute information; when there is no target partitioning strategy that matches the attribute information in the cache, the step of obtaining a partitioning constraint model based on the attribute information is executed; or, when there is a target partitioning strategy that matches the attribute information in the cache, the target partitioning strategy is used as the partitioning result of the data to be partitioned.

[0184] In the embodiments, it is possible to combine Figure 4 As shown, by checking the cache to see if a target partitioning strategy corresponding to the data to be partitioned already exists, if it does, it is directly retrieved and used as the partitioning result, without needing to solve it again through the solver. If the target partitioning strategy corresponding to the data to be partitioned does not exist in the cache, it can be solved again through the solver, and the obtained partitioning scheme is added to the cache. In this way, for repeated calculations of the same shape, historical partitioning schemes can be directly reused, thereby avoiding repeated runs and improving processing efficiency.

[0185] In this embodiment, when determining whether the cache contains a target chunking strategy that matches the attribute information, a hash key can be generated based on the attribute information; the cache can be searched for a target hash value that matches the hash key; if a target hash value that matches the hash key exists in the cache, it is determined that the cache contains a target chunking strategy that matches the attribute information. The cache stores a table showing the relationship between keys and hash values, where the hash value represents a generated chunking strategy.

[0186] In this embodiment, a tiling method based on the MIQCP solver is employed to further reduce the overhead of the tiling method. For each operator, the input shape of the tiling can be used as the hash key, and the output quadruple or octule can be used as the cached hash value.

[0187] Based on the above embodiments, in another embodiment provided in this application, such as Figure 7 As shown, a calculation method is also provided, which can be applied to the NPU22 in server 20. The method may include the following steps:

[0188] In step S710, the target structured data sent by the CPU in the server is obtained.

[0189] The target structured data includes the block division results of the data to be divided. The block division results include a first type of data block and a second type of data block. The first type of data block satisfies the alignment conditions of the NPU, and the second type of data block includes data blocks other than the first type of data blocks in the block division results.

[0190] As can be seen from the above embodiments, the first type of data block can be a large data block, and the second type of data block can be a small data block. The alignment condition is an NPU hardware constraint condition, which can be referred to in the description of the above embodiments and will not be repeated here.

[0191] In step S720, based on the block division results, the computing cores in the NPU are divided into a first computing core for computing the first type of data blocks and a second computing core for computing the second type of data blocks.

[0192] In this embodiment, the segmentation result may include the number of large blocks and the number of small blocks of data into which the data to be segmented is divided. This allows the computational cores in the NPU used to compute these segmented data to be divided into two parts: a first computational core used to compute the first type of data blocks and a second computational core used to compute the second type of data blocks.

[0193] In step S730, the first type of data block is calculated by loading the operator kernel through the first computing core, and the second type of data block is calculated by loading the operator kernel through the second computing core, so as to obtain the calculation result.

[0194] In step S740, the calculation result is sent to the CPU.

[0195] In this embodiment, the computing core is divided into cores for computing large blocks of data and cores for computing small blocks of data. For large blocks of data, an aligned computing interface can be called, while for small blocks of data, aligned and unaligned computing interfaces are called as appropriate. Specifically, the aligned computing interface is used to call the computing core for computing large blocks of data, and the unaligned computing interface is used to call the computing core for computing small blocks of data. Thus, the first computing core loads an operator kernel to compute a first type of data block, and the second computing core loads an operator kernel to compute a second type of data block, obtaining the computation result, which is then sent to the CPU.

[0196] This application's embodiments utilize a data partitioning optimization algorithm and, by designing a suitable optimization objective, can automatically tune operator performance and core usage. This allows the NPU to improve data processing efficiency when performing parallel operations across multiple computing cores.

[0197] Based on the above embodiments, when the data to be segmented is one-dimensional data, the following data segmentation example for the data to be segmented is provided:

[0198] User problem description: Calculate the cumulative sum of 43 float data, with a maximum of 4 computing cores. All block start addresses must be 32 bytes aligned (it is known that the first element address is 32 bytes aligned, so this requirement is equivalent to the block length being 32 bytes aligned). What is the optimal block scheduling scheme?

[0199] The one-dimensional data block constraint model provided in the above embodiments can be:

[0200] 0≤N f ≤43

[0201] 0≤N t ≤43

[0202] 0≤C f ≤4

[0203] 0≤C t ≤1

[0204] N f *C f +N t *C t =43

[0205] C f +C t ≤4

[0206] N f ≥N t +1

[0207] N f =k*8

[0208] minimize N f +(C f +C t ) / C

[0209] The solver yields the following block partitioning results: 2 large blocks, 1 small block, 16 floats in length for each large block, and 11 floats in length for each small block. The calculation uses 3 cores. This can be represented by a quadruple: <16, 2, 11, 1>, where <16, 2, 11, 1> represents the length of the large block, the number of large blocks, the length of the small block, and the number of small blocks, respectively.

[0210] Based on the above embodiments, it can be seen that, while ensuring that the length of large data blocks does not increase, the embodiments also regard the number of computing cores as one of the optimization objectives. The number of computing cores used (3 computing cores) will be less, and the optimization algorithm can ensure that the number of computing cores occupied will be minimized.

[0211] Furthermore, the parameters obtained in this application embodiment support complex operations such as reduction. Specifically, for example, related technologies calculate parameters where two large data blocks each have 16 floats, and two small data blocks each have 8 floats, totaling 48 floats, exceeding the 43 floats given in the problem. Therefore, the computational core processing the data at the very end of memory needs to ensure that when using the parameter of 8 (floats), the subsequent memory exceeding 43 floats has been padded with 0s to avoid affecting the reduction operation; or it can calculate the length of the padding data (48-43) to obtain a mask and call the unaligned summation interface to mask the influence of the padding data. The parameters given in this application embodiment are: 2 large blocks, 1 small block, 16 floats long for the large blocks, and 11 floats long for the small blocks, whose total length is exactly equal to the 43 floats given in the problem. The computational core processing the data at the end of memory uses the parameter of 11 (floats), which can be directly used as a mask, and the calculation can be completed using the unaligned summation interface.

[0212] Based on the above embodiments, in another embodiment provided in this application, when the data to be segmented is two-dimensional data, the following data segmentation example for the data to be segmented is provided:

[0213] User problem description: For the up_proj weight matrix of the MLP structure in DeepSeekR1, which has a shape of [18432, 7168], and assuming the input x has a shape of [12345, 7168], the desired operation is to right-multiply the x matrix by the transpose of the weight matrix. The expectation is to perform this computation on up to 40 cores, dividing the resulting matrix [12345, 18432] into blocks, and testing the optimal parameters within a given set. The M-axis alignment parameter is searched in [16, 32, 64], and the N-axis alignment parameter is searched in [32, 64, 128].

[0214] Solving using the two-dimensional data block constraint model and solver provided in the above embodiments yields the following results: Figure 8 The block division results are shown. Among them, Figure 8 This is a two-dimensional data segmentation result provided for an exemplary embodiment of this application. Wherein, Figure 8 Each column (from right to left) represents the shape of the data to be divided into blocks, the value of the optimization objective, the length of the large M-axis data block, the number of large M-axis data blocks, the length of the small M-axis data block, the number of small M-axis data blocks, the length of the large N-axis data block, the number of large N-axis data blocks, the length of the small N-axis data block, and the number of small N-axis data blocks.

[0215] Figure 8Each line in the table represents multiple segmentation results obtained when splitting the data to be segmented, and each segmentation result includes corresponding segmentation parameters. Subsequently, operator developers can develop multiple versions of the operator based on these parameters and test their performance, selecting the optimal version as the final version adopted. This example demonstrates the powerful flexibility and convenience of the embodiments of this application in operator development.

[0216] As can be seen from the embodiments provided in this application above, the embodiments of this application can support the complex scenario of two-dimensional block division and support arbitrary alignment methods. Furthermore, the interface is universal and flexible, effectively saving human resources for operator development.

[0217] The implementation example models the data partitioning problem using an optimization algorithm, theoretically guaranteeing the optimality of multi-core parallelism. This is primarily achieved by minimizing large data blocks and minimizing the number of cores used, ultimately improving operator performance and increasing the utilization of computing cores. Furthermore, the implementation example provides a general data partitioning method that supports alignment constraints for arbitrary axes and arbitrary partitions. This enables the implementation example to support the development of data partitioning code for the vast majority of operators, ultimately improving operator development efficiency and effectively saving human resources.

[0218] In the case of dividing each functional module according to its corresponding function, this application provides a data segmentation device, which can be a server or a CPU chip applied to a server. Figure 9 A schematic block diagram of the functional modules of a data segmentation device provided for an exemplary embodiment of this application. For example... Figure 9 As shown, the data segmentation device includes:

[0219] The information acquisition module 91 is used to acquire user problem description information; wherein, the user problem description information includes: attribute information of the data to be partitioned, the maximum number of computing cores in the NPU used to compute the data to be partitioned, and alignment conditions; the alignment conditions are used to constrain the size of the data block and the starting address of the data block;

[0220] The block constraint module 92 is used to obtain the target block constraint model based on attribute information. The target block constraint model is used to constrain the data to be divided into first-class data blocks and second-class data blocks based on alignment conditions. The target block constraint model includes first-class data block constraints, second-class data block constraints, and optimization objectives. The first-class data block constraints are used to constrain the number and length of the first-class data blocks, the second-class data block constraints are used to constrain the number and length of the second-class data blocks, and the optimization objective is to minimize the number of computing cores occupied when the length of the first-class data blocks is minimized.

[0221] The block result generation module 93 is used to solve the block result of the data to be blocked by the target solver based on attribute information, maximum quantity, alignment conditions and target block constraint model.

[0222] The attribute information of the data to be segmented includes: data dimension, data shape, and data type.

[0223] This solution, by obtaining user problem description information, can transform the data partitioning algorithm that originally required manual writing by the user into a process where the data partitioning method provided in this application embodiment automatically solves the problem simply by requiring the user to describe it, which can effectively reduce the manpower consumption of writing partitioning algorithms.

[0224] In another embodiment provided in this application, the target block constraint model includes a one-dimensional data block constraint model and a two-dimensional data block constraint model;

[0225] Obtaining a block-based constraint model based on attribute information includes:

[0226] Based on the data dimension of the data to be partitioned, when the data to be partitioned is determined to be one-dimensional, a one-dimensional data partitioning constraint model is obtained; or, when the data to be partitioned is determined to be two-dimensional, a two-dimensional data partitioning constraint model is obtained.

[0227] This solution can simultaneously meet the segmentation requirements for both one-dimensional and two-dimensional data by constructing one-dimensional data segmentation constraint models and two-dimensional data segmentation constraint models respectively.

[0228] In another embodiment provided in this application, when the data to be segmented is one-dimensional data, the segmentation result is quadruple data, which includes: the length of the first type of data block, the number of the first type of data blocks, the length of the second type of data block, and the number of the second type of data blocks.

[0229] This scheme uses quadruple data to describe the block results, which makes it easier for the NPU to quickly split the one-dimensional block data based on the quadruple data, thereby improving data processing efficiency.

[0230] In another embodiment provided in this application, when the data to be segmented is two-dimensional data, the segmentation result is octet data, which includes: the length of the first type of data block on the horizontal axis, the number of the first type of data blocks on the horizontal axis, the length of the second type of data block on the horizontal axis, the number of the second type of data blocks on the horizontal axis, the length of the first type of data block on the vertical axis, the number of the first type of data blocks on the vertical axis, the length of the second type of data block on the vertical axis, and the number of the second type of data blocks on the vertical axis.

[0231] This scheme uses octet data to describe the block results, which makes it easier for the NPU to quickly divide the two-dimensional block data based on the octet data, thereby improving data processing efficiency.

[0232] In another embodiment provided in this application, the device further includes:

[0233] The storage module is used to store the block results into the target structured data;

[0234] The sending module is used to send target structured data to the NPU in response to the NPU's call request.

[0235] After obtaining the segmented results using the methods described above, this scheme can store the segmented results in a target structured data set. This target structured data set can be a TilingContext. The CPU can receive segmented result call requests from the NPU, and in response to these requests, the CPU sends the TilingContext to the NPU, thus facilitating data transfer between the CPU and the NPU.

[0236] In another embodiment provided in this application, the device further includes a cache determination module, used for:

[0237] Determine whether the cache contains a target chunking strategy that matches the attribute information;

[0238] If no target partitioning strategy matching the attribute information exists in the cache, execute the step of obtaining the partitioning constraint model based on the attribute information;

[0239] Alternatively, if a target chunking strategy that matches the attribute information exists in the cache, the target chunking strategy is used as the chunking result for the data to be chunked.

[0240] For repeated calculations of the same shape, this scheme can directly reuse historical block-based schemes, thereby avoiding repeated runs and improving processing efficiency.

[0241] In another embodiment provided in this application, the cache determination module is further configured to:

[0242] Generate hash keys based on attribute information;

[0243] The cache is searched to see if a target hash value that matches the hash key exists; the cache stores a table showing the relationship between keywords and hash values, where the hash value represents the generated block splitting strategy.

[0244] When a target hash value matching the hash key exists in the cache, determine that the cache contains a target chunking strategy that matches the attribute information.

[0245] This scheme uses hash keys to match target hash values ​​in the cache, enabling fast retrieval of cached data.

[0246] By dividing each functional module according to its corresponding function, this application provides a computing device, which can be a server, a terminal, or a chip applied to a server. Figure 10 A schematic block diagram of the functional modules of a computing device provided for an exemplary embodiment of this application. For example... Figure 10 As shown, the computing device includes:

[0247] The data acquisition module 101 is used to acquire the target structured data sent by the CPU in the server; wherein, the target structured data includes the block division result of the data to be block divided, the block division result includes a first type of data block and a second type of data block, the first type of data block satisfies the alignment condition of the NPU, and the second type of data block includes data blocks other than the first type of data block in the block division result;

[0248] The computing core partitioning module 102 is used to partition the computing cores in the NPU into a first computing core for computing a first type of data block and a second computing core for computing a second type of data block based on the partitioning results.

[0249] The operator kernel loading module 103 is used to load the operator kernel through the first computing core to calculate the first type of data block, and to load the operator kernel through the second computing core to calculate the second type of data block, so as to obtain the calculation result;

[0250] The result sending module 104 is used to send the calculation results to the CPU.

[0251] This application also provides a computing device, including: at least one processor; a memory for storing at least one processor-executable instruction; wherein the at least one processor is configured to execute instructions to implement the methods disclosed in the embodiments of this application.

[0252] The aforementioned processor can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by integrated logic circuits in the processor's hardware or by software instructions. The aforementioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in memory, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0253] Furthermore, various operations / processes according to embodiments of this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computing device with a dedicated hardware architecture, such as... Figure 11 The computing device 1900 shown is equipped with programs that constitute the software. When various programs are installed, the computing device is able to perform various functions, including functions such as those mentioned above. Figure 11 A structural block diagram of a computing device provided for an exemplary embodiment of this application.

[0254] The computing device 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0255] like Figure 11As shown, the computing device 1900 includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computing device 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904. The computing device may also include a CPU 1910 and an NPU 1911.

[0256] Multiple components in computing device 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information to computing device 1900. Input unit 1906 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the computing device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disk and optical disk. Communication unit 1909 allows computing device 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0257] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (NPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on a computing device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0258] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by the processor of a computing device, the computing device is able to perform the methods disclosed in the embodiments of this application.

[0259] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0260] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0261] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0262] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0263] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0264] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0265] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0266] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0267] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A data segmentation method, characterized in that, The method is applied to the CPU in a server, the server further comprising an NPU, and the method includes: Obtain user problem description information; wherein, the user problem description information includes: attribute information of the data to be partitioned, the maximum number of computing cores in the NPU used to calculate the data to be partitioned, and alignment conditions; the alignment conditions are used to constrain the size of the data block and the starting address of the data block; A target block constraint model is obtained based on the attribute information; wherein, the target block constraint model is used to constrain the data to be divided into a first type of data block and a second type of data block based on the alignment condition, the target block constraint model includes a first type of data block constraint condition, a second type of data block constraint condition and an optimization objective, the first type of data block constraint condition is used to constrain the number and length of the first type of data block, the second type of data block constraint condition is used to constrain the number and length of the second type of data block, and the optimization objective is used to minimize the number of computing cores occupied when the length of the first type of data block is minimized; Based on the attribute information, the maximum quantity, the alignment condition, and the target block constraint model, the target solver is used to solve the block result of the data to be blocked.

2. The method according to claim 1, characterized in that, The attribute information of the data to be segmented includes: data dimension, data shape, and data type.

3. The method according to claim 2, characterized in that, The target block constraint model includes a one-dimensional data block constraint model and a two-dimensional data block constraint model; The process of obtaining the block constraint model based on the attribute information includes: Based on the data dimension of the data to be segmented, when it is determined that the data to be segmented is one-dimensional data, the one-dimensional data segmentation constraint model is obtained. Alternatively, when it is determined that the data to be segmented is two-dimensional data, the two-dimensional data segmentation constraint model can be obtained.

4. The method according to claim 3, characterized in that, When the data to be segmented is one-dimensional data, the segmentation result is a quadruple data, which includes: the length of the first type of data block, the number of the first type of data blocks, the length of the second type of data block, and the number of the second type of data blocks.

5. The method according to claim 3, characterized in that, When the data to be segmented is two-dimensional data, the segmentation result is an octet data, which includes: the length of the first type of data block on the horizontal axis, the number of the first type of data blocks on the horizontal axis, the length of the second type of data block on the horizontal axis, the number of the second type of data blocks on the horizontal axis, the length of the first type of data block on the vertical axis, the number of the first type of data blocks on the vertical axis, the length of the second type of data block on the vertical axis, and the number of the second type of data blocks on the vertical axis.

6. The method according to claim 1, characterized in that, The method further includes: Store the segmentation results to the target structured data; In response to the NPU's call request, the target structured data is sent to the NPU.

7. The method according to claim 1, characterized in that, The method further includes: Determine whether the cache contains a target block strategy that matches the attribute information; If no target partitioning strategy matching the attribute information exists in the cache, the step of obtaining the partitioning constraint model based on the attribute information is executed. Alternatively, if a target partitioning strategy matching the attribute information exists in the cache, the target partitioning strategy is used as the partitioning result for the data to be partitioned.

8. The method according to claim 7, characterized in that, The step of determining whether the cache contains a target block strategy that matches the attribute information includes: Generate hash keys based on the attribute information; The cache is searched to see if a target hash value matching the hash key exists; wherein, the cache stores a table relating keywords and hash values, and the hash value represents the generated block splitting strategy; When a target hash value matching the hash key exists in the cache, it is determined that the cache contains a target block strategy matching the attribute information.

9. A calculation method, characterized in that, The method is applied to the NPU in a server, and the method includes: Obtain the target structured data sent by the CPU in the server; wherein, the target structured data includes the block division result of the data to be block divided, the block division result includes a first type of data block and a second type of data block, the first type of data block satisfies the alignment condition of the NPU, and the second type of data block includes data blocks other than the first type of data block in the block division result; Based on the block division results, the computing cores in the NPU are divided into a first computing core for computing the first type of data blocks and a second computing core for computing the second type of data blocks. The first type of data block is calculated by loading an operator kernel through the first computing core, and the second type of data block is calculated by loading an operator kernel through the second computing core to obtain the calculation result; The calculation result is sent to the CPU.

10. A computing device, characterized in that, include: CPU and NPU; A memory for storing executable instructions of the CPU and the NPU; Wherein, when the CPU is configured to execute the instructions, it implements the method as described in any one of claims 1-8, and when the NPU is configured to execute the instructions, it implements the method as described in claim 9.

Images