19 results about "Matrix partitioning" patented technology

A data processing architecture, chip, matrix multiplication and neural network calculation method, the data processing architecture (20) comprises: a node array of P rows and Q columns; P×Q nodes (10) in the node array respectively store a first submatrix obtained by dividing a first matrix, and a second submatrix obtained by dividing a second matrix; each node (10) is configured to send the second submatrix stored in the node to other P-1 nodes in the column where the node is located, respectively, to perform multiplication calculation on the first submatrix stored in the node and the transposed matrix of the plurality of second submatrices, respectively, to obtain a plurality of first type submatrix calculation results; according to a preset first correspondence relationship, the first type submatrix calculation results corresponding to other nodes in the row where the node is located are respectively sent to the corresponding nodes, and the first type submatrix calculation result corresponding to the node calculated is added to the first type submatrix calculation result received from other nodes to obtain a first result submatrix of the node.

A computing network comprising an orchestrator controller and multiple nodes is used to execute a distributed application across these nodes, each node including a node controller. The orchestrator controller determines whether a matrix is ​​symmetric, the matrix being divided into partial matrices, each partial matrix residing on one of the multiple nodes. The orchestrator controller causes each node controller to apply an antisymmetric function to the cells of its partial matrices and aggregates the results of the antisymmetric function application, thereby providing a sum of the partial matrices. The orchestrator controller determines whether the sum of all partial matrix sums is zero; if it is zero, the matrix is ​​determined to be symmetric. This computing network reduces the amount of communication between the multiple nodes required to determine the symmetry of the matrix.

The invention provides a data processing method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of large models.The method comprises the steps that an input matrix is divided into a plurality of sub-matrixes, and the number and the position index of each sub-matrix are determined based on a grouping staggering method; executing corresponding sub-matrix operation based on at least one calculation thread block in the thread block network, and transmitting an operation result of the corresponding sub-matrix based on at least one communication thread block corresponding to the at least one calculation thread block; based on the serial number sequence corresponding to each sub-matrix, storing the operation result of each sub-matrix in a cache, and based on the serial number and the position index corresponding to each sub-matrix and the operation results of the sub-matrixes sequentially stored in the cache, determining an output matrix; therefore, the communication thread blocks are arranged between the calculation thread blocks, so that the calculation thread blocks can synchronously perform sub-matrix operation in the communication process of the communication thread blocks, and the resource utilization rate of the graphics processor is improved.

This invention discloses a heterogeneous edge distributed collaborative parallel inference method for a large language model, relating to the field of distributed edge computing technology. The method includes: determining the calculation formulas for the parameter quantity and inference time of the self-attention layer allocated to each edge device; determining the calculation formulas for the parameter quantity and inference time of the multilayer perceptron allocated to each edge device; obtaining the total memory constraint; optimizing the minimum computation time of the self-attention layer and the multilayer perceptron under the total memory constraint to obtain the attention head allocation ratio of the self-attention layer and the parameter matrix partitioning ratio of the multilayer perceptron; distributing the parameters of the large language model to each edge device according to the allocation and partitioning ratios, and performing parallel inference of the large language model through each edge device. This method improves the segmentation accuracy of the large language model parameters on edge devices.

The application discloses a shared memory allocation method and device for matrix calculation, a medium, equipment and products, and the method comprises the following steps: when the second storage body with smaller storage capacity is sufficient to accommodate the first multiplier matrix, allocating the first multiplier matrix with larger memory requirement to the second storage body, and allocating the second multiplier matrix to the first storage body; checking whether the storage body with the largest remaining space is sufficient to accommodate the bias matrix; if yes, allocating the bias matrix to the storage body with the largest remaining space and performing matrix fusion calculation; if not, if the memory requirement of the bias matrix is not greater than the total remaining space, dividing the bias matrix into two sub-matrices, and allocating the two sub-matrices to the first storage body and the second storage body respectively to perform matrix fusion calculation; and allocating the result matrix obtained through matrix fusion calculation to the memory occupation space of the bias matrix through multiplexing operation. The application can maximize the utilization of the remaining space of the shared memory, and significantly improve the success rate of memory allocation.

The application relates to the technical field of artificial intelligence, and provides a matrix multiplication implementation method and device, electronic equipment, a storage medium and a program product, the method comprising the following steps: based on multiple matrix multiplication operations to be simultaneously executed, dividing computing cores on a chip into multiple computing groups, each computing group comprising at least two computing cores, and each computing group corresponding to one matrix multiplication operation; controlling each computing group to respectively execute the corresponding matrix multiplication operation, and obtaining a result matrix of each matrix multiplication operation. According to the application, the computing cores are grouped, each computing group executes one matrix multiplication operation in parallel, and the function of executing multiple matrix multiplication operations on the chip in parallel is realized; moreover, the computing cores in each computing group are reduced relative to the entire chip, for one matrix in each matrix multiplication operation, the matrix is divided into a smaller number of sub-matrices, the number of rows of the sub-matrices is relatively large, the number of rows of the sub-matrices can cover the minimum calculation granularity of the computing cores, and the utilization rate of the computing cores is improved.

This application relates to the field of quantum computing technology, and in particular to a method, apparatus, and electronic device for compressed storage of quantum logic gates, comprising: creating a decision graph and using the root node of the decision graph as the current node; dividing the matrix into four sub-matrices with the same number of rows and columns, creating a child node corresponding to each sub-matrix, and associating the pointer of the current node with each child node; when a sub-matrix satisfies a preset termination condition, using the child node corresponding to the sub-matrix as a leaf node and storing the leaf node as a scalar value of the corresponding sub-matrix; recursively decomposing the quantum gate matrix and shared sub-matrices by creating a decision tree, decomposing complex matrix operations into smaller sub-problems, reducing redundant computations; and using the decision graph to explore the sparsity and regularity of the quantum gate matrix, efficiently representing the quantum gate matrix and state vector, thereby achieving compressed storage of matrices in quantum circuits and avoiding redundant computations.

The invention discloses a data processing method and device and related equipment, which can perform LU decomposition of R columns of sub-matrixes on a to-be-decomposed matrix needing to be decomposed in each iteration process in a process of performing LU decomposition calculation on applied to-be-processed data, namely a dense matrix, and calculate a matrix multiplication taking a product of R and sub-matrix dimensions as a dimension. The dimension of matrix multiplication calculation is large, and efficient calculation of matrix multiplication can be achieved. In this way, the dense matrix can be divided into the sub-matrixes with the small granularity, it is guaranteed that the calculation efficiency of matrix multiplication is high on the premise that process loads are balanced, and then the overall efficiency of matrix decomposition calculation is improved.

A data processing architecture, a chip, a matrix multiplication computation method, and a neural network computation method. The data processing architecture (20) comprises a node array having P rows and Q columns. P×Q nodes (10) in the node array respectively store first sub-matrices obtained by partitioning a first matrix X on which matrix multiplication is to be performed, and second sub-matrices obtained by partitioning a second matrix W on which matrix multiplication is to be performed. Each node (10) is configured to separately send the first sub-matrix stored in the node to the other Q-1 nodes in the same row as the node, separately send the second sub-matrix stored in the node to the other P-1 nodes in the same column as the node, and on the basis of sub-matrices stored in and received by the node, compute a result sub-matrix corresponding to the node, wherein the sub-matrices include the first sub-matrices and the second sub-matrices, and the result sub-matrices of the P×Q nodes constitute a result matrix Y obtained by performing matrix multiplication on the first matrix X and the second matrix W.

The invention belongs to assembly structure division in the field of assembly sequence planning, and particularly relates to a method for automatically dividing sub-assemblies, which comprises the following specific steps of: 1, acquiring six interference matrixes of an assembly in the direction of an overall coordinate axis; 2, acquiring a static interference matrix, namely a connection matrix, of the assembly; 3, performing C + + programming on the connection matrix and the gravity direction interference matrix, and generating a support matrix based on an intersection taking mode; 4, firstly defining a basic part of the assembly body by utilizing the support matrix based on an actual assembly site; step 5, dividing sub-assemblies based on the support matrix; and 6, for the obtained sub-assemblies and basic parts, the assemblies are defined into an assembly hierarchy tree form conforming to an assembly site. The invention defines a sub-assembly division method based on an actual assembly site, the method can be used as a reference in a subsequent assembly sequence planning process, the assembly sequence of the sub-assembly layers can be planned firstly, and then the sequence of parts in each sub-assembly is planned.

This application discloses a parity check matrix storage method, data decoding method, apparatus, and electronic device, relating to the field of data processing technology. The parity check matrix storage method includes: obtaining the base matrix corresponding to the parity check matrix; dividing the base matrix into multiple sub-base matrices with the same data format; for each sub-base matrix, determining the shift value corresponding to each non-zero item in each row of the sub-base matrix, and the column index in the sub-base matrix, forming multiple sets of data and row end marker information corresponding to each row; storing the multiple sets of data and row end marker information corresponding to each row by column to obtain the corresponding storage matrix. Thus, since the number of columns and / or rows in each sub-base matrix is ​​reduced relative to the base matrix, the redundant space required for alignment when storing the column index and shift value of each row of the base matrix can be reduced. Compared with the scheme of storing based on the column index and shift value of the non-zero item in each row of the base matrix, the storage space can be reduced.

The invention discloses a matrix multiplication hardware acceleration method and a hardware acceleration circuit, and belongs to the field of artificial neural network acceleration calculation and chips. Dividing the matrix into a plurality of coarse blocks in advance according to the size of a data cache module; the data reading module reads the plurality of coarse blocks from the external storage module according to a matrix multiplication sequence, allocates different fine blocks of the matrix A to each matrix multiplication unit in the matrix multiplication array, transpose the matrix B and allocates one fine block to each matrix multiplication unit in the matrix multiplication array; the matrix multiplication array can obtain a plurality of parts and data in each calculation; and each matrix multiplication unit in the matrix multiplication array obtains the part of the input matrix, which should be multiplied by the distributed fine blocks, and the part is subjected to matrix multiplication so as to complete the matrix multiplication of the input matrix and the model parameters of the large model.

The embodiment of the invention provides a large-scale matrix QR decomposition multi-core parallel computing method and device, equipment and a storage medium. The method comprises the steps that a target large-scale matrix is divided into multiple matrix blocks according to the fast memory capacity of each core; arranging the plurality of cores according to the capacity of the cores to obtain a two-dimensional core arrangement; distributing the plurality of matrix blocks into a two-dimensional core arrangement according to a circulation mode in the transverse direction and the longitudinal direction; a task execution dependency graph between matrix blocks in each core is constructed according to a QR decomposition principle, so that each core executes a QR decomposition task and a transmission task of the matrix blocks according to the task execution dependency graph, and a 2D cyclic distribution mode of the matrix blocks converts dependency of column vectors into dependency between matrix column blocks; the cyclic distribution mode can relieve the problem of unbalanced storage calculation of each node to a certain extent.

This invention discloses a multi-core computing method and apparatus for QR decomposition that integrates column-block TSQR and tile update. The method includes: for a large-scale matrix to be decomposed, horizontally dividing it into blocks based on the number of cores in the multi-core processor, and further dividing the large-scale matrix into multiple layers according to the tile update method; constructing a multi-stage pipeline with a binary tree structure, performing parallel triangular decomposition on the initial layers of the large-scale matrix through the multi-core processor to obtain the corresponding upper triangular matrix and orthogonal matrix; replacing the original layer block matrix with the upper triangular matrix, performing conjugate transpose on the orthogonal matrix to obtain the update matrix, and implementing parallel updates for the remaining layers through the multi-core processor; based on the reflection vector implicitly stored by the multi-core processor during the parallel triangular decomposition process, solving for the overall orthogonal matrix corresponding to the large-scale matrix, and simultaneously outputting the overall upper triangular matrix obtained after layer update. This invention can realize QR decomposition of large-scale matrices.

The application provides a large language model length extrapolation method and device without training, and the method comprises the following steps: acquiring an input sequence, dynamically determining an optimal mapping length through a parameterized function based on the length of the input sequence; dividing a relative position matrix of the input sequence into three regions, and applying different position encoding granularity strategies to different regions; using Flash Attention to perform attention calculation by taking the relative position matrix of the input sequence as input, wherein, for the head and middle regions, sliding window attention is realized by indexing different KV Cache, and for the tail region, lower triangular attention is realized. The application does not need to retrain the model, and the calculation overhead is low. Only a small amount of code modification is needed to directly embed into mainstream large models such as Llama and Qwen, and it is compatible with efficient attention implementation methods such as Flash Attention2.

This disclosure relates to the field of computer technology and provides a method for a processor comprising multiple computing units to perform matrix multiplication. In this method: a quantized parameter matrix is ​​divided into multiple parameter blocks, which are then sequentially and non-repeatingly assigned to multiple computing units, such that each computing unit is assigned a first number of consecutive parameter blocks, except for at most one computing unit which is assigned fewer than a first number of consecutive parameter blocks from the multiple parameter blocks; each computing unit dequantizes its assigned parameter blocks; each computing unit obtains the portion of the input matrix that should be multiplied by the assigned parameter blocks, and performs matrix multiplication on the dequantized assigned parameter blocks and the portion of the input matrix to complete the matrix multiplication operation between the input matrix and the model parameters. Therefore, globally, only one dequantization is required on the quantized parameter matrix, saving computational resources and improving inference efficiency.

The invention discloses an automatic design method for a large-model accelerator, and belongs to the field of large-model chip design. Constructing a matrix multiplication processing unit and a vector processing unit as basic hardware modules; completely streamlined scheduling is adopted, a synchronization barrier is judged and eliminated through periodic-level data dependence, and operator-level streamlined execution is achieved; the KV cache is directly written into downstream storage according to a column vector by using an SRAM-Free data stream, so that redundant cache is avoided; a double vector quantization technology is adopted, and the inverse quantization operand is reduced through row-column segmentation quantization; on the basis of the model operator set and hardware resource constraints, executing automatic granularity pipeline search, dynamically determining the degree of parallelism and a matrix division factor, and generating an optimal pipeline structure; and a reconfigurable interconnection network is configured to realize flexible connection and deployment of multiple operators. According to the method, the problems of insufficient flexibility, high reasoning delay, waste of hardware resources and the like caused by existing template design are effectively solved, and the reasoning delay and power consumption of a large language model on platforms such as an FPGA (Field Programmable Gate Array) and an ASIC (Application Specific Integrated Circuit) can be remarkably reduced.

The present disclosure provides a data processing method and device, electronic equipment, storage medium and program product, relating to the technical field of large model, the method comprises: dividing an input matrix into a plurality of sub-matrices, determining the number and position index of each sub-matrix based on a grouping interleaving method; performing corresponding sub-matrix operation based on at least one computing thread block in a thread block network, and transmitting the operation result of the corresponding sub-matrix based on at least one communication thread block corresponding to the at least one computing thread block; storing the operation result of each sub-matrix in the cache based on the number sequence corresponding to each sub-matrix, determining the output matrix based on the number and position index corresponding to each sub-matrix and the operation result of the sub-matrix stored in the cache in sequence; in this way, the communication thread block is arranged between the computing thread blocks, so that the computing thread blocks can perform sub-matrix operation synchronously in the process of communication, improving the utilization rate of graphics processor resources.