Dense matrix data processing method and device based on NPU

Abstract

The invention relates to an NPU-based dense matrix data processing method and device, and relates to the technical field of high-performance computing, and the method comprises the steps: carrying out the format conversion processing and dimension reduction mapping conversion processing of at least one group of input first sparse matrix and second sparse matrix based on an NPU processing unit, and obtaining a corresponding first dense matrix and second dense matrix; according to NPU kernel configuration information in the NPU processing unit, adaptive partitioning processing is carried out on at least one group of input corresponding first dense matrixes and second dense matrixes, a plurality of matrix partitioning groups are obtained, and each matrix partitioning group comprises a first matrix partitioning block used for multiplication and a corresponding second matrix partitioning block; distributing the plurality of matrix block groups to a plurality of NPU cores of an NPU processing unit for multiplication operation to obtain a matrix block multiplication result; and carrying out integration processing on the block multiplication results to obtain matrix multiplication results corresponding to the first dense matrix and the second dense matrix. According to the scheme, the energy efficiency ratio is increased.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of high-performance computing, and particularly relates to a dense matrix data processing method and device based on an NPU (Neural Network Processor). BACKGROUND

[0002] In various application scenarios, for example, as the scale of data volume is larger and larger, the requirement of calculation precision is higher and higher, and the requirement of calculation timeliness is higher and higher, how to realize high-performance calculation of a calculation device becomes a research focus in various fields.

[0003] Taking the field of geophysical exploration as an example, in the fields of seismic data processing and electromagnetic prospecting, a large amount of matrix operations are involved, and these operations usually have high calculation complexity and high memory demand, which puts extremely high requirements on the calculation performance of a calculation device. For example, the Reverse Time Migration (RTM) algorithm is a mainstream high-precision imaging algorithm and is widely used in seismic data processing, and the core calculation process of the algorithm mainly consists of dense matrix multiplication.

[0004] In the process of implementing the present disclosure, the inventors have found that the related art at least has the following technical problems: the traditional CPU (Central Processing Unit) and GPU (Graphics Processing Unit) still have certain limitations in high-performance calculation, especially when processing calculation-intensive tasks such as dense matrix multiplication, in the case of meeting the calculation precision, usually a large calculation resource consumption or hardware consumption is needed as a trade-off, and the calculation efficiency also needs to be improved, and the comprehensive advantages are not possessed. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a dense matrix data processing method and device based on an NPU.

[0006] In a first aspect, embodiments of the present disclosure provide a dense matrix data processing method based on an NPU. The method comprises: performing format conversion processing and dimension reduction mapping conversion processing on at least one group of input first sparse matrices and second sparse matrices based on an NPU processing unit, to obtain corresponding first dense matrices and second dense matrices; performing adaptive blocking processing on the at least one group of input corresponding first dense matrices and second dense matrices according to NPU kernel configuration information in the NPU processing unit, to obtain a plurality of matrix block groups, each matrix block group comprising: a first matrix block and a corresponding second matrix block for performing multiplication operations; distributing the plurality of matrix block groups to a plurality of NPU kernels of the NPU processing unit for multiplication operation processing, to obtain matrix block multiplication results; each NPU kernel comprising one or more matrix multiplication operation units; and performing integration processing on the block multiplication results, to obtain a matrix multiplication result corresponding to the first dense matrix and the second dense matrix.

[0007] In some embodiments, the matrix dimension of the first sparse matrix corresponding to the i-th group of input is Mi rows by Ki columns, and the matrix dimension of the second sparse matrix corresponding to the i-th group of input is Ki rows by Ni columns; i is a positive integer; Mi, Ki, and Ni are positive integers; in the case of multiple groups of input, the matrix dimensions of some groups of input are different from those of other groups of input. The format conversion processing of the at least one group of input first sparse matrices and second sparse matrices based on the NPU processing unit comprises: for each group of input first sparse matrix and second sparse matrix in the at least one group of input, performing integer multiple padding and splitting conversion processing on the corresponding dimensions of the columns of the first sparse matrix and the rows of the second sparse matrix according to the array dimension SxS of the matrix multiplication operation unit in the NPU processing unit, to obtain a first target sparse matrix and a second target sparse matrix that match the format of the matrix multiplication operation unit; the dimension of the first target sparse matrix is a first dimension [Ri, Mi, S], and the dimension of the second target sparse matrix is a second dimension [Ri, Ni, S], where Ri=Ki 补 / S, Ri is a positive integer, Ki 补 represents the minimum dimension corresponding to the columns of the first sparse matrix and the rows of the second sparse matrix after padding by S.

[0008] In some embodiments, the dimension reduction mapping conversion processing comprises: converting the first target sparse matrix into a first dense matrix with a lower dimension and converting the second target sparse matrix into a second dense matrix with a lower dimension based on the NPU processing unit according to a preset mapping rule; the preset mapping rule has an inverse mapping rule for recovery.

[0009] In some embodiments, the method further comprises: based on the inverse mapping rule, performing inverse mapping conversion on the matrix multiplication result to obtain a target matrix multiplication result corresponding to the first sparse matrix and the second sparse matrix of each group of inputs.

[0010] In some embodiments, the NPU kernel configuration information comprises input specifications of the matrix multiplication operation of each NPU kernel. The method of adapting and dividing the first dense matrix and the second dense matrix corresponding to each group of inputs based on the input specifications of the NPU kernel configuration information to obtain a plurality of matrix block groups comprises: dividing the first dense matrix and the second dense matrix corresponding to each group of inputs in each group of inputs based on the input specifications of the matrix multiplication operation of each NPU kernel to obtain a plurality of matrix block groups corresponding to each group of inputs.

[0011] In some embodiments, the NPU kernel configuration information further comprises: the number of NPU kernels, and the processing capability limit of each NPU kernel. The method further comprises: determining multi-core scheduling information according to the number of matrix block groups corresponding to each group of inputs, the number of NPU kernels, the processing capability limit of each NPU kernel, and state information of the NPU kernel; wherein the state information of the NPU kernel comprises: whether each matrix multiplication unit in each NPU kernel is in an idle state; the multi-core scheduling information is used to indicate that a current matrix block group to be processed is allocated to a target matrix multiplication unit in an NPU kernel that is in an idle state and is of a suitable specification; and the method of allocating the plurality of matrix block groups to the plurality of NPU kernels in the NPU processing unit for multiplication operation processing to obtain a matrix block multiplication result comprises: based on the multi-core scheduling information, allocating the plurality of matrix block groups to the plurality of NPU kernels in the NPU processing unit for multiplication operation processing to obtain a matrix block multiplication result.

[0012] In some embodiments, the method is applied to a matrix multiplication operation task involved in data processing of geophysical exploration; the method further includes: carrying the at least one group of inputs from the global memory of the CPU to a unified buffer in the NPU processing unit for processing by the NPU processing unit; wherein the format conversion processing and the dimension reduction mapping conversion processing are performed based on a vector operation unit in the NPU processing unit, and the corresponding processing results are stored in the unified buffer. Wherein, the plurality of matrix block groups obtained by the adaptive block processing are stored in a first buffer; the matrix block multiplication results obtained by the multiplication operation processing of the plurality of NPU kernels are stored in a second buffer. The integration processing is performed based on the vector operation unit, the matrix multiplication result is obtained, and after the mapping conversion inverse processing is performed on the matrix multiplication result, the target matrix multiplication result corresponding to the first sparse matrix and the second sparse matrix of the at least one group of inputs is written to the global memory.

[0013] In a second aspect, embodiments of the present disclosure provide an NPU-based dense matrix data processing device. The device includes an NPU processing unit, a matrix block unit, a scheduling and processing unit, and an integration processing unit. The NPU processing unit is configured to perform format conversion processing and dimension reduction mapping conversion processing on at least one group of input first sparse matrix and second sparse matrix to obtain corresponding first dense matrix and second dense matrix. The matrix block unit is configured to perform adaptive block processing on the first dense matrix and the second dense matrix corresponding to the at least one group of input based on NPU kernel configuration information in the NPU processing unit to obtain a plurality of matrix block groups, each matrix block group including a first matrix block and a corresponding second matrix block for multiplication operation. The scheduling and processing unit is configured to assign the plurality of matrix block groups to a plurality of NPU kernels of the NPU processing unit for multiplication operation processing to obtain matrix block multiplication results; each NPU kernel includes one or more matrix multiplication operation units. The integration processing unit is configured to perform integration processing on the block multiplication results to obtain a matrix multiplication result corresponding to the first dense matrix and the second dense matrix.

[0014] In a third aspect, embodiments of the present disclosure provide an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; the memory is configured to store a computer program; and the processor is configured to execute the program stored on the memory to implement the NPU-based dense matrix data processing method as described above.

[0015] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the NPU-based dense matrix data processing method is implemented.

[0016] The above technical solutions provided by the embodiments of the present disclosure have at least some or all of the following advantages: The format conversion processing adapts the input to the input format in the NPU processing unit, and the dimension reduction mapping conversion processing can make the matrix data more concentrated and improve the processing efficiency. At the same time, the format conversion processing and the dimension reduction mapping conversion processing based on the NPU processing unit can utilize the efficient data storage and handling mechanism of the NPU (Neural Network Processing Unit) processing unit to quickly batch process the matrix elements of higher dimensions and efficiently handle data, improve processing timeliness, and fully utilize the hardware resources of the NPU. Subsequently, the first dense matrix and the second dense matrix corresponding to at least one group of inputs are adaptively processed according to the NUP kernel configuration information in the NPU processing unit. Regardless of the different dimensions of the dense matrix corresponding to each group of inputs, the adaptive block processing is performed to adapt to the input format of the NPU kernel, and the plurality of matrix block groups obtained by the block processing can be implemented in parallel and efficiently by one or more matrix multiplication units in the NPU kernel to obtain matrix multiplication results. The block multiplication results are integrated to obtain the matrix multiplication results corresponding to the first dense matrix and the second dense matrix. The low-power characteristics of the NPU are fully utilized to significantly reduce the energy consumption of the matrix multiplication operation while ensuring high-performance computing, thereby improving the energy efficiency ratio. The above method can be applied to the matrix multiplication operation tasks involved in the data processing of geophysical exploration. Compared with the scheme based on CPU and GPU combination operation, the energy efficiency ratio can be improved, and it can also be extended to other scenarios requiring efficient matrix multiplication operation, such as artificial intelligence, deep learning, and large-scale scientific computing, and has universality. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or related technical descriptions will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Figure 1An architecture schematic diagram of an NPU processing unit suitable for the NPU-based dense matrix data processing method according to an embodiment of the present disclosure is schematically shown.

[0020] Figure 2 A flowchart of the NPU-based dense matrix data processing method according to an embodiment of the present disclosure is schematically shown.

[0021] Figure 3 A flowchart of the NPU-based dense matrix data processing method according to another embodiment of the present disclosure is schematically shown.

[0022] Figure 4 A structure block diagram of the NPU-based dense matrix data processing device according to an embodiment of the present disclosure is schematically shown.

[0023] Figure 5 A structure block diagram of the electronic device provided by an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0025] Term Explanation A sparse matrix is a common matrix type in the field of numerical calculation and engineering, which has a number of non-zero elements far less than zero elements, and the non-zero elements are distributed in no fixed pattern. Generally, when the proportion of non-zero elements is less than or equal to a set value (for example, 5%), it can be called a sparse matrix.

[0026] A dense matrix refers to a matrix in which each element has a corresponding value, and most elements are not zero. It is stored in the form of a two-dimensional array in computer memory, and each element can be directly accessed and operated. Since each element has storage space, a dense matrix is suitable for cases where matrix elements are relatively dense.

[0027] The first exemplary embodiment of the present disclosure provides an NPU-based dense matrix data processing method.

[0028] With the rapid development of artificial intelligence and special processor technology, NPU, as a new type of computing architecture, is gradually emerging in the field of high-performance computing. Compared with traditional CPU and GPU, NPU has higher parallel computing efficiency and lower energy consumption in processing large-scale matrix operations, and is particularly suitable for high-performance computing scenarios in geophysical exploration, especially the optimization implementation of RTM algorithm.

[0029] Taking the field of geophysics as an example, a large number of matrix multiplication operations are involved in data processing in the field of geophysics, such as performing reverse time migration algorithm (RTM), prestack depth migration method, forward and inversion of potential field data, and extension processing of potential field data. However, the current optimization research on the combination of geophysical exploration algorithm and NPU is still in its infancy. How to fully utilize the hardware characteristics of NPU and design an efficient processing flow to further improve the computing efficiency and accuracy of related matrix multiplication operation tasks (such as the matrix multiplication operation tasks involved in the execution of RTM) has become a technical problem to be solved.

[0030] Figure 1 The architecture of the NPU processing unit of the NPU-based dense matrix data processing method suitable for the embodiments of the present disclosure is schematically shown. In Figure 1 the main part of hardware and software is divided by a dashed line, the upper half is mainly the hardware architecture, and the lower half is mainly the software architecture.

[0031] Referring to Figure 1 , the architecture of the NPU processing unit of the NPU-based dense matrix data processing method suitable for the embodiments of the present disclosure is schematically shown. In the NPU processing unit 100, the following hardware is included: a plurality of operation units, a plurality of buffer zones, and a carrying unit (MTE); also includes an interface for interacting with the CPU, corresponding software modules and their supporting cache units, such as: bus interface unit (BIU), system controller (system control), instruction cache (I Cache), scalar instruction scheduler (Scalar PSD), instruction distribution unit (Instru. Dispatch), operation unit data queue (CubeQueue), vector data queue (Vector Queue), and carrying unit queue (MTE Queue).

[0032] The plurality of operation units includes: a matrix multiplication operation unit (such as a CUBE unit), a vector (Vector) operation unit, a scalar (Scalar) operation unit, an addition operation unit, and an addition counter.

[0033] The plurality of buffers include a unified buffer, a buffer L1, a buffer L0A, a buffer L0B, and a buffer L0C. The buffer corresponding to the input of the matrix multiplication operation unit includes the buffer L1, the buffer L0A, and the buffer L0B. The buffer corresponding to the output of the matrix multiplication operation unit includes the buffer L0C. The input and the output can store data through the unified buffer.

[0034] In the embodiments of the present disclosure, the dense matrix data processing method based on the NPU is suitable for data processing in various fields and is a general solution. The following describes the data processing of the matrix multiplication operation task involved in the data processing of geophysical exploration, but the present disclosure is not limited to this scenario.

[0035] During the data processing of geophysical exploration, the dense matrix data processing method based on the NPU can be applied to the matrix multiplication operation involved in the reverse-time migration algorithm, the prestack depth migration method, the forward and inverse of the potential field data, the continuation processing of the potential field data, and the like.

[0036] Figure 2 A flowchart of the dense matrix data processing method based on the NPU according to an embodiment of the present disclosure is schematically shown.

[0037] Referring to Figure 2 The dense matrix data processing method based on the NPU provided by the embodiments of the present disclosure includes the following steps: S210, S220, S230, and S240.

[0038] In step S210, the NPU processing unit performs format conversion processing and dimension reduction mapping conversion processing on at least one group of input first sparse matrix and second sparse matrix to obtain corresponding first dense matrix and second dense matrix.

[0039] In some embodiments, the matrix dimension of the first sparse matrix corresponding to the i-th group of input is Mi rows x Ki columns, and the matrix dimension of the second sparse matrix corresponding to the i-th group of input is Ki rows x Ni columns; i is a positive integer; Mi, Ki, and Ni are positive integers; in the case where there are multiple groups of input, the matrix dimensions corresponding to some groups of input are different from the matrix dimensions corresponding to other groups of input.

[0040] In step S210, the NPU processing unit performs format conversion processing on at least one group of input first sparse matrix and second sparse matrix, including: The column of the first sparse matrix and the row of the second sparse matrix corresponding to each of the at least one group of inputs are subjected to an integer multiple padding split conversion process according to the array dimension SxS (for example, a fractal basic unit determined by the Cube operation unit (hardware)) of the matrix multiplication operation unit in the NPU processing unit, and a first target sparse matrix and a second target sparse matrix corresponding to the matrix multiplication operation unit are obtained in format matching; the corresponding dimension of the first target sparse matrix is the first dimension [Ri, Mi, S], and the corresponding dimension of the second target sparse matrix is the second dimension [Ri, Ni, S], Ri=Ki 补 / S, Ri is a positive integer, and Ki 补 indicates that the column of the first sparse matrix and the row of the second sparse matrix are padded according to S and the corresponding minimum dimension is obtained.

[0041] In order to realize efficient calculation of matrix multiplication on the NPU, the input matrix needs to be converted in format first to adapt to the hardware characteristics of the Cube operation unit.

[0042] As an example, the input matrix format conversion: the first sparse matrix (as the left matrix of the multiplication operation, the matrix dimension (which can also be described as the matrix shape) is Mi rows x Ki columns) and the second sparse matrix (as the right matrix of the multiplication operation, the matrix dimension is: Ki rows x Ni columns) corresponding to the i-th group of inputs are respectively converted into the format supported by the Cube operation unit, for example, the matrix multiplication operation unit is a Cube unit, and the corresponding array dimension SxS is 16x16, then the left matrix is converted into a first target sparse matrix, and the corresponding first dimension is: [Ri=Ki 补 / 16, Mi, 16], and the right matrix is converted into a second target sparse matrix, and the corresponding second dimension is: [Ri=Ki 补 / 16, Ni, 16]. Wherein, if Ki itself is an integer multiple of S, then Ki 补 =Ki, and the integer result of Ri=Ki / 16 is directly used; if Ki is not an integer multiple of S, then the number of bits of the column of the first sparse matrix and the row of the second sparse matrix need to be padded, that is, Ki is adjusted, and the corresponding dimension Ki 补 is obtained after padding the least number of bits, and then Ki 补 / 16 is calculated to obtain an integer Ri. The above Ri corresponds to the number of blocks.

[0043] In some embodiments, the at least one group of inputs is transferred from a global memory of a central processing unit (CPU) to a unified buffer in a neural processing unit (NPU) processing unit using an efficient data storage and transfer mechanism of the NPU processing unit, for processing by the NPU processing unit.

[0044] Referring to Figure 1 In some embodiments, in the NPU processing unit, the format conversion processing and the dimension reduction mapping conversion processing are performed based on a vector operation unit, and the corresponding processing results are stored in the unified buffer.

[0045] Since the vector operation unit can perform fast batch processing on matrix elements of higher dimensions, and the results of the format conversion processing and the dimension reduction mapping conversion processing performed by the vector operation unit are stored in the unified buffer, and the plurality of matrix block groups obtained after subsequent block processing can be efficiently transferred to the matrix multiplication operation units corresponding to the plurality of NPU kernels based on a transfer unit (MTE) for processing.

[0046] In some embodiments, the dimension reduction mapping conversion processing includes: based on the NPU processing unit, converting the first target sparse matrix into a first dense matrix with lower dimensions and converting the second target sparse matrix into a second dense matrix with lower dimensions according to a preset mapping rule; the preset mapping rule has an inverse mapping rule for restoration.

[0047] The dimension reduction mapping conversion processing involves matrix decomposition or dimension reduction technology. In the case where the S dimension in the first dimension and the second dimension is unchanged, the original high-dimensional sparse matrix is represented by a low-dimensional dense matrix through a preset transformation (such as projection or matrix decomposition), and then the operation (such as matrix multiplication operation) is performed in the low-dimensional space, which can significantly reduce the calculation and storage overhead.

[0048] For example, as an example of matrix decomposition transformation, the first target sparse matrix and the second target sparse matrix can be respectively decomposed into the product of several small matrices based on singular value decomposition (SVD), to obtain a first dense matrix and a second dense matrix with lower dimensions. Since the first target sparse matrix corresponds to the left matrix role of matrix multiplication, and the second target sparse matrix corresponds to the right matrix role of matrix multiplication, when singular value decomposition is used, the number of columns of the left matrix and the number of rows of the right matrix are required to remain unchanged before and after decomposition. Alternatively, as an example of projection transformation, high-dimensional data can also be projected into a low-dimensional space based on a random matrix while approximately preserving distance information (based on the Johnson-Lindenstrauss lemma, a set of high-dimensional data points is projected into a low-dimensional space through linear mapping while maintaining the approximate invariance of the distance relationship between points), to obtain a first dense matrix and a second dense matrix after dimension reduction.

[0049] In this embodiment, the first target sparse matrix is subjected to dimension reduction mapping processing, and is assumed to be reduced from the first dimension [Ri, Mi, S] to the third dimension [Ri 密 , Mi 密 , S]; the second target sparse matrix is subjected to dimension reduction mapping processing, and is assumed to be reduced from the second dimension [Ri, Ni, S] to the fourth dimension [Ri 密 , Ni 密 , S], wherein Ri 密 , Mi 密 are the reduced dimensions of the first dense matrix, and are positive integers; and Ri 密 , Ni 密 are the reduced dimensions of the second dense matrix, and are positive integers.

[0050] In the above step S210, the format conversion processing adapts the input to the input format in the NPU processing unit, and the dimension reduction mapping conversion processing can make the matrix data more concentrated and improve the processing efficiency; at the same time, the format conversion processing and the dimension reduction mapping conversion processing based on the NPU processing unit (for example, based on the vector operation unit in the NPU processing unit) can utilize the efficient data storage and handling mechanism of the NPU (neural network processor) processing unit to quickly batch process the matrix elements with higher dimensions and perform efficient data handling, improve the processing timeliness, and fully utilize the hardware resources of the NPU.

[0051] In step S220, according to the NPU kernel configuration information in the NPU processing unit, at least one group of input corresponding first dense matrix and second dense matrix are subjected to adaptive block processing to obtain a plurality of matrix block groups, each matrix block group comprising: a first matrix block and a corresponding second matrix block for performing multiplication operation.

[0052] In some embodiments, in the above step S220, the plurality of matrix block groups obtained by performing the adaptive block processing are stored in the first buffer, for example Figure 1 Buffer L1 in the example architecture.

[0053] Generally speaking, the input dimensions are in the hundreds, thousands, or even tens of thousands of levels, and the dimensions are still very large after format conversion and dimensionality reduction mapping conversion.

[0054] In some embodiments, the NPU core configuration information includes but is not limited to: input specifications for matrix multiplication operations performed by each NPU core, the number of cores of the NPU core, and the processing capability limit of each NPU core.

[0055] In some embodiments, the input specifications corresponding to different NPU cores can be the same; the input specifications corresponding to different NPU cores can also be different. The input specifications of one or more matrix multiplication units included in the same NPU core can be the same or different.

[0056] The number of cores in an NPU core is X. Different computing devices may have different numbers of cores based on the differences in NPU processing units. Generally speaking, the number of NPU cores is greater than 1. The processing power limit of each NPU core refers to the sum of the processing power corresponding to all matrix multiplication units in each NPU core.

[0057] For different input groups, since at least one of Mi, Ni and Ki may be different, the first dimension and second dimension corresponding to different input groups may be different. That is, for the NPU processing unit, the shape of the input matrix is ​​different.

[0058] Among them, in the above-mentioned step S220, according to the NPU core configuration information in the above-mentioned NPU processing unit, the first dense matrix and the second dense matrix corresponding to at least one group of inputs are adaptively blocked to obtain multiple matrix block groups, including: according to the input specifications of the above-mentioned each NPU core for matrix multiplication operation, the first dense matrix and the second dense matrix corresponding to each group of inputs in at least one group of inputs are divided based on the above-mentioned input specifications to obtain multiple matrix block groups corresponding to each group of inputs.

[0059] In the first dense matrix and the second dense matrix, since they have been processed by format conversion and dimensionality reduction mapping conversion in advance, the third dimension [Ri 密 ,Mi 密 , S], the fourth dimension [Ri 密 , Ni 密, S] is adapted to the array dimension SxS of the matrix multiplication operation unit in the NPU processing unit, so that the first dense matrix and the second dense matrix can be quickly divided from the dimension of Ri 密 , Mi 密 , Ni 密 to obtain a plurality of matrix block groups.

[0060] In some embodiments, the above method further comprises: determining multi-core scheduling information according to the number of groups of the plurality of matrix block groups corresponding to each group of inputs, the above number of kernels, the processing capability limit of each NPU kernel, and the state information of the above NPU kernel. The state information of the NPU kernel includes: whether each matrix multiplication operation unit in each NPU kernel is in an idle state; and the multi-core scheduling information is used to indicate that the current matrix block group to be processed is allocated to the target matrix multiplication operation unit in the idle state and the specification adaptation in the NPU kernel.

[0061] In some embodiments, the number of matrix block groups corresponding to a group of inputs (corresponding to a group of left matrices and right matrices, i.e., a group of first dense matrices and second dense matrices corresponding to a group of inputs) or some input groups can be very large, far exceeding the processing capability limit of all NPU kernels, so time division multiplexing is used, i.e., after all NPU kernels process part of the matrix block groups, the remaining matrix block groups are processed.

[0062] Therefore, based on the above considerations, each matrix block group corresponding to the same group of inputs is preferentially divided into a plurality of corresponding NPU kernels for parallel processing.

[0063] In each NPU kernel, as long as a matrix multiplication operation unit is detected to be in an idle state and the specification is adapted, a new matrix block group (which can be a matrix block group in the same group of inputs that has not been processed, or a matrix block group corresponding to other groups of inputs) is input for corresponding processing, improving the time occupancy rate of the matrix multiplication operation unit, thereby improving the processing timeliness of all groups of inputs; when selecting a new matrix block group, for a new group of inputs and the remaining matrix block groups after processing part of the matrix block groups, the remaining matrix block groups after processing part of the matrix block groups are preferentially selected for processing to ensure that the total processing time of the plurality of matrix block groups in the same group of inputs is as compact as possible, reducing the processing delay.

[0064] In step S220, the first dense matrix and the second dense matrix corresponding to at least one group of inputs are adaptively block-processed according to the NUP kernel configuration information in the NPU processing unit. Regardless of the different dimensions of the dense matrices corresponding to each group of inputs, the adaptive block processing is performed to adapt to the input format of the NPU kernel, and the plurality of matrix block groups obtained by the block processing can be implemented in parallel and efficiently by one or more matrix multiplication operation units in the NPU kernel to obtain the matrix block multiplication result.

[0065] In step S230, the plurality of matrix block groups are distributed to the plurality of NPU kernels of the NPU processing unit for multiplication operation processing to obtain the matrix block multiplication result. Each NPU kernel includes one or more matrix multiplication operation units.

[0066] The correspondence between the plurality of matrix block groups obtained by dividing the same group of inputs and the NPU kernel includes one or more of the following cases: The plurality of matrix block groups correspond to different NPU kernels; or There are two matrix block groups corresponding to different matrix multiplication operation units of the same NPU kernel; or There are two matrix block groups corresponding to different processing time periods of the same matrix multiplication operation unit of the same NPU kernel.

[0067] In step S230, the plurality of matrix block groups are distributed to the plurality of NPU kernels of the NPU processing unit for multiplication operation processing to obtain the matrix block multiplication result, including: based on the multi-core scheduling information, the plurality of matrix block groups are distributed to the plurality of NPU kernels of the NPU processing unit for multiplication operation processing to obtain the matrix block multiplication result.

[0068] In some embodiments, in step S230, the matrix block multiplication result obtained by the multiplication operation processing of the plurality of NPU kernels is temporarily stored in a second buffer, for example Figure 1 The buffer L0C in the example architecture.

[0069] In step S240, the block multiplication result is integrated to obtain the matrix multiplication result corresponding to the first dense matrix and the second dense matrix.

[0070] For example, the integration processing can include: performing element addition operation on corresponding positions for column-split results; performing splicing processing according to corresponding positions for row-split results, etc.

[0071] In some embodiments, based on Figure 1The vector operation unit in the example architecture performs the above integration processing, and the obtained matrix multiplication result is stored in the above unified buffer.

[0072] In some embodiments, the above integration processing is performed based on the vector operation unit, the obtained matrix multiplication result is stored in the above unified buffer, and after inverse mapping conversion processing is performed on the above matrix multiplication result, the target matrix multiplication result corresponding to the first sparse matrix and the second sparse matrix of at least one group of inputs is written to the above global memory.

[0073] In the embodiments including steps S210-S240, the format conversion processing adapts the input to the input format in the NPU processing unit, and the dimension reduction mapping conversion processing can make the matrix data more concentrated and improve the processing efficiency; at the same time, the format conversion processing and the dimension reduction mapping conversion processing based on the NPU processing unit can utilize the efficient data storage and transfer mechanism of the NPU (Neural Network Processing Unit) processing unit to quickly process the matrix elements of higher dimensions in batches and perform efficient data transfer, improve the processing timeliness, and fully utilize the hardware resources of the NPU. Subsequently, the first dense matrix and the second dense matrix corresponding to at least one group of inputs are adaptively divided and processed according to the NUP kernel configuration information in the NPU processing unit. Regardless of the different dimensions of the dense matrices corresponding to each group of inputs, the adaptive division and processing is performed to adapt to the input format of the NPU kernel, and the plurality of matrix block groups obtained by the division and processing can be implemented in parallel and efficiently by one or more matrix multiplication operation units in the NPU kernel to obtain matrix block multiplication results; subsequently, the above block multiplication results are integrated to obtain the matrix multiplication results corresponding to the first dense matrix and the second dense matrix; the low-power consumption characteristics of the NPU are fully utilized to significantly reduce the energy consumption of the matrix multiplication operation while ensuring high-performance computing, and the energy efficiency ratio is improved. The above method can be applied to the matrix multiplication operation tasks involved in the data processing of geophysical exploration. Compared with the scheme based on CPU and GPU combined operation, the energy efficiency ratio can be improved, and it can also be extended to other scenarios requiring efficient matrix multiplication operation, such as artificial intelligence, deep learning, and large-scale scientific computing, and has universality.

[0074] Figure 3 A flowchart of an NPU-based dense matrix data processing method according to another embodiment of the present disclosure is schematically shown.

[0075] In some embodiments, referring to Figure 3 As shown in the above method, in addition to steps S210-S240, the following step S310 is included: performing inverse mapping conversion processing on the above matrix multiplication result based on an inverse mapping rule to obtain target matrix multiplication results corresponding to the first sparse matrix and the second sparse matrix of at least one group of inputs.

[0076] By converting the matrix multiplication output corresponding to the ith group of inputs into the format of Mi rows x Ni columns based on the above mapping conversion inverse processing, the dimension of the target matrix multiplication result is [Mi, Ni].

[0077] The matrix multiplication result obtained in the above step S240 is carried from the format supported by the Cube operation unit to the Unified Buffer by taking the Unified Buffer as an intermediate buffer, and then the mapping conversion inverse processing is performed according to the original format requirement, and written back to the global memory.

[0078] Through the above method, the performance optimization and energy efficiency improvement of the embodiments of the present disclosure are realized as follows: Computing efficiency improvement: By adapting the Cube operation unit, multi-core parallel computing and dynamic scheduling, the computing efficiency of large-scale dense matrix multiplication is significantly improved.

[0079] Energy consumption reduction: The low-power consumption characteristics of the NPU are fully utilized to reduce the energy consumption of matrix multiplication operation.

[0080] Real-time enhancement: By optimizing the computing process and reducing the data carrying overhead, the real-time or quasi-real-time requirement of the scheme applied to high-precision imaging tasks such as RTM algorithm in geophysical exploration is met.

[0081] The second exemplary embodiment of the present disclosure provides a dense matrix data processing device based on NPU.

[0082] Figure 4 The structure block diagram of the dense matrix data processing device based on NPU according to an embodiment of the present disclosure is schematically shown.

[0083] Referring to Figure 4 The dense matrix data processing device based on NPU 400 provided by the embodiments of the present disclosure includes an NPU processing unit 410, a matrix blocking unit 420, a scheduling and processing unit 430, and an integration processing unit 440.

[0084] The NPU processing unit 410 is configured to perform format conversion processing and dimension reduction mapping conversion processing on at least one group of input first sparse matrix and second sparse matrix, to obtain corresponding first dense matrix and second dense matrix.

[0085] The matrix blocking unit 420 is configured to perform adaptive blocking processing on the first dense matrix and the second dense matrix corresponding to at least one group of input according to the NPU kernel configuration information in the NPU processing unit, to obtain a plurality of matrix blocking groups, each matrix blocking group including a first matrix block and a corresponding second matrix block for performing multiplication operation.

[0086] The scheduling and processing unit 430 is configured to allocate the plurality of matrix block groups to a plurality of NPU cores of the NPU processing unit for multiplication operation processing to obtain a matrix block multiplication result. Each NPU core includes one or more matrix multiplication operation units.

[0087] The integration processing unit 440 is configured to perform integration processing on the block multiplication result to obtain a matrix multiplication result corresponding to the first dense matrix and the second dense matrix.

[0088] In some embodiments, the dense matrix data processing apparatus 400 further includes an inverse mapping processing unit configured to perform inverse mapping conversion processing on the matrix multiplication result based on an inverse mapping rule to obtain a target matrix multiplication result corresponding to at least one group of input first sparse matrix and second sparse matrix.

[0089] Further details of the present embodiment can be referred to the related description of the first embodiment, which will not be repeated here.

[0090] Any plurality of the functional modules included in the NPU-based dense matrix data processing apparatus 400 can be combined in one module, or any one of the modules can be split into a plurality of modules. Alternatively, at least part of the function of one or more of the modules can be combined with at least part of the function of another module, and implemented in one module. At least one of the functional modules included in the NPU-based dense matrix data processing apparatus 400 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system in package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware by integrating or packaging circuits, or implemented in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the functional modules included in the NPU-based dense matrix data processing apparatus 400 can be at least partially implemented as a computer program module which can perform corresponding functions when executed.

[0091] A third exemplary embodiment of the present disclosure provides an electronic device.

[0092] Figure 5 A structural block diagram of an electronic device provided by an embodiment of the present disclosure is schematically shown.

[0093] Reference is made to Figure 5As shown, the electronic device 500 provided by the embodiments of the present disclosure includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete mutual communication through the communication bus 504; the memory 503 is used for storing a computer program; and the processor 501 is used for executing the program stored on the memory to implement the NPU-based dense matrix data processing method as described above.

[0094] The fourth exemplary embodiment of the present disclosure also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the NPU-based dense matrix data processing method as described above.

[0095] The computer readable storage medium can be included in the device or apparatus described in the above embodiments; or can exist separately and not be assembled into the device or apparatus. The computer readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0096] According to the embodiments of the present disclosure, the computer readable storage medium can be a non-volatile computer readable storage medium, which can include but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus or device.

[0097] It should be noted that, in the present document, the relationship terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0098] The foregoing is merely illustrative of the various implementations of the present disclosure and the general principles thereof. Numerous modifications can be made to these illustrations, and equivalents can be substituted therefor, without departing from the scope of the present disclosure. Accordingly, the disclosure is not to be limited to that shown, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dense matrix data processing method based on NPU, characterized in that: include: Based on the NPU processing unit, performing format conversion processing and dimensionality reduction mapping conversion processing on at least one set of input first sparse matrices and second sparse matrices to obtain corresponding first dense matrices and second dense matrices; According to NPU core configuration information in the NPU processing unit, adapting and blocking the first dense matrix and the second dense matrix corresponding to at least one group of inputs to obtain a plurality of matrix block groups, each matrix block group including: a first matrix block for performing a multiplication operation and a corresponding second matrix block; Allocating the plurality of matrix block groups to a plurality of NPU cores of the NPU processing unit for multiplication processing to obtain matrix block multiplication results; each NPU core includes one or more matrix multiplication operation units; The block multiplication results are integrated to obtain matrix multiplication results corresponding to the first dense matrix and the second dense matrix.

2. The method according to claim 1, characterized in that The matrix dimensions of the first sparse matrix corresponding to the i-th group of inputs are: Mi rows × Ki columns, and the matrix dimensions of the second sparse matrix corresponding to the i-th group of inputs are: Ki rows × Ni columns; i is a positive integer; Mi, Ki, and Ni are all positive integers; When there are multiple groups of inputs, the matrix dimensions corresponding to some groups of inputs are different from the matrix dimensions corresponding to other groups of inputs; The NPU processing unit is used to perform format conversion processing on at least one set of input first sparse matrix and second sparse matrix, including: For the first sparse matrix and the second sparse matrix of each input in the at least one group of inputs, the corresponding dimensions of the columns of the first sparse matrix and the rows of the second sparse matrix are padded, split and converted by integer multiples according to the array dimension S×S of the matrix multiplication operation unit in the NPU processing unit, and the first target sparse matrix and the second target sparse matrix whose formats match the matrix multiplication operation unit are obtained; the dimension corresponding to the first target sparse matrix is ​​the first dimension [Ri, Mi, S], and the dimension corresponding to the second target sparse matrix is ​​the second dimension [Ri, Ni, S], where Ri=Ki 补 / S, Ri is a positive integer, Ki 补 Indicates the minimum dimension corresponding to padding the columns of the first sparse matrix and the rows of the second sparse matrix according to S divisibility.

3. The method according to claim 2, characterized in that The dimension reduction mapping conversion process includes: Based on the NPU processing unit, according to a preset mapping rule, the first target sparse matrix is ​​converted into a first dense matrix with a lower dimension, and the second target sparse matrix is ​​converted into a second dense matrix with a lower dimension; the preset mapping rule has an inverse mapping rule corresponding to restoration.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: Based on the inverse mapping rule, the matrix multiplication result is subjected to an inverse mapping transformation process to obtain a target matrix multiplication result corresponding to at least one set of input first sparse matrix and second sparse matrix.

5. The method according to any one of claims 1 to 3, characterized in that The NPU core configuration information includes: input specifications for each NPU core to perform matrix multiplication operations; According to the NPU core configuration information in the NPU processing unit, the first dense matrix and the second dense matrix corresponding to at least one group of inputs are adapted and block processed to obtain multiple matrix block groups, including: According to the input specifications of the matrix multiplication operations performed by the respective NPU cores, a first dense matrix and a second dense matrix corresponding to each group of inputs in at least one group of inputs are divided based on the input specifications to obtain a plurality of matrix block groups corresponding to each group of inputs.

6. The method according to claim 5, characterized in that The NPU core configuration information also includes: the number of NPU cores and the processing capacity limit of each NPU core; The method further comprises: Determining multi-core scheduling information based on the number of matrix block groups corresponding to each group of inputs, the number of cores, the processing capability limit of each NPU core, and status information of the NPU cores; The state information of the NPU core includes: whether each matrix multiplication operation unit in each NPU core is idle; the multi-core scheduling information is used to indicate that the current matrix block group to be processed is allocated to the target matrix multiplication operation unit in the NPU core that is idle and has an adapted specification; Among them, the multiple matrix block groups are allocated to multiple NPU cores in the NPU processing unit for multiplication operation processing to obtain matrix block multiplication results, including: based on the multi-core scheduling information, the multiple matrix block groups are allocated to multiple NPU cores in the NPU processing unit for multiplication operation processing to obtain matrix block multiplication results.

7. The method according to any one of claims 1 to 3, characterized in that The method is applied to matrix multiplication tasks involved in data processing for geophysical exploration; The method further comprises: Moving the at least one set of inputs from the global memory of the CPU to a unified buffer in the NPU processing unit for processing by the NPU processing unit; The format conversion process and the dimensionality reduction mapping conversion process are performed based on the vector operation unit in the NPU processing unit, and the corresponding processing results are stored in the unified buffer; The plurality of matrix block groups obtained by performing the adaptive block processing are stored in the first buffer; the matrix block multiplication results obtained by performing the multiplication operation processing by the plurality of NPU cores are stored in the second buffer; The integration processing is performed based on the vector operation unit, and the obtained matrix multiplication result is stored in the unified buffer. After the mapping transformation inverse processing is performed on the matrix multiplication result, the target matrix multiplication result corresponding to the first sparse matrix and the second sparse matrix of at least one set of inputs is written to the global memory.

8. A dense matrix data processing device based on NPU, characterized in that: include: An NPU processing unit is used to perform format conversion processing and dimensionality reduction mapping conversion processing on at least one set of input first sparse matrices and second sparse matrices to obtain corresponding first dense matrices and second dense matrices; a matrix block unit, configured to perform adaptive block processing on a first dense matrix and a second dense matrix corresponding to at least one set of inputs according to NPU core configuration information in the NPU processing unit, to obtain a plurality of matrix block groups, each matrix block group including: a first matrix block for multiplication operation and a corresponding second matrix block; A scheduling and processing unit, configured to distribute the plurality of matrix block groups to the plurality of NPU cores of the NPU processing unit for multiplication processing to obtain matrix block multiplication results; each NPU core includes one or more matrix multiplication operation units; An integration processing unit is used to integrate the block multiplication results to obtain matrix multiplication results corresponding to the first dense matrix and the second dense matrix.

9. An electronic device, characterized in that: The processor, the communication interface, the memory and the communication bus are connected to each other via the communication bus. Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images