454 results about "Matrix multiplication" patented technology

The invention discloses a processor and electronic equipment. The processor includes a computing unit including a tensor core configured to perform a matrix multiplication operation using a scaling factor, and a memory, the computing unit further including a scaling factor processing module configured to determine and cache a scaling factor for each tensor associated with the matrix multiplication operation, the computing unit further comprises at least one storage module arranged on a data path between the tensor core and the memory, and the at least one storage module is exclusively occupied by the tensor core when the tensor core executes tensor related operation. And the scaling factor processing module is arranged on the at least one storage module. At present, scaling factors and floating-point number quantization are completed by a vector calculation core, so that performance is reduced, and delay becomes high, and a scaling factor processing module arranged on a storage module in a calculation unit can improve the overall execution efficiency of low-precision matrix multiplication using the scaling factors.

Provided in the present invention is a parameter-efficient large-language fine-tuning federated learning framework, comprising the following steps: performing modeling on LoRA adapters of different edge clouds; since different weights exhibit different average performances on the LoRA adapters, using singular values to quantify the importance of the weights, and therefore, before each round of independent training of the LoRA adapters using N edge clouds, using a matrix singular value to decompose a BA matrix in the LoRA adapter for each trainable weight; configuring heterogeneous LoRA adapters on the basis of the importance of the weights; and using different numbers of quantization bits to quantize a pre-trained model, and performing high-precision inverse quantization on the pre-trained model only when matrix multiplication is executed, wherein the pre-trained model is quantized to the maximum number of quantization bits on the basis of the memory budget of the edge clouds. The present invention has the following beneficial effects: the present invention determines the optimal fine-tuning model structure, thereby improving the performance of LLM fine-tuning, and adapts to heterogeneous and resource-constrained edge clouds.

An apparatus providing tensor core matrix multiplication and accumulation (MMA) with hardware-based statistics collection and outlier suppression is disclosed. The apparatus includes processor circuitry comprising at least one processor core comprising matrix multiplication circuitry to: execute a matrix multiplication operation on first input data from a first set of registers and on second input data from a second set of registers; collect, as part of executing the matrix multiplication operation via statistics collection hardware circuitry of the matrix multiplication circuitry, output statistics data corresponding to the matrix multiplication operation; and output the output statistics data along with a result of the matrix multiplication operation; and output statistics storage to store the output statistics data.

The invention provides a mixed data precision matrix multiplication and addition unit and a calculation method, the matrix multiplication and addition unit comprises a calculation unit, and the calculation unit comprises a format division module, a multiplication array module, an addition tree module, an accumulator module, a normalization module and a shift register module. The calculation unit converts the first input matrix and the second input matrix into input data in a middle floating point format; executing parallel multiplication operation on the input data to generate an intermediate product result; performing index alignment and accumulation on the intermediate product result to generate an intermediate accumulated value; accumulating the intermediate product result and the value of the third input matrix in a form of accumulating an intermediate accumulated value, and outputting an accumulated result; and converting an accumulation result into a normalized result and outputting the normalized result. The format division module supports various precisions and converts data with different widths into an intermediate floating point format, so that other hardware units can be reused, and the problems that hardware resources are complex and different model reasoning scenes are difficult to meet are solved.

The invention discloses a tensor core, a processor, a data processing method, electronic equipment and a storage medium, and is applied to the field of tensor processing. The tensor kernel comprises a first dot multiplication unit and a scaling factor matrix multiplication processing module, the scaling factor matrix multiplication processing module comprises a second dot multiplication unit, and the first dot multiplication unit and the second dot multiplication unit support dot multiplication operations of different floating-point number precisions. The tensor core is configured to receive a first tensor, a second tensor, a scaling factor of the first tensor, a scaling factor of the second tensor, an offset term of the first tensor, and an offset term of the second tensor, perform a matrix multiplication operation using the scaling factor and the offset term using a first dot multiplication unit and a scaling factor matrix multiplication processing module, and obtaining a matrix multiplication operation result of the first tensor and the second tensor. Matrix multiplication operation using scaling factors is executed by multiplexing dot multiplication units with different precisions in a tensor kernel, extra hardware area cost is reduced, and existing hardware resources are fully multiplexed.

The invention relates to an optimization method of a hybrid expert system, computer equipment, a readable storage medium and a program product. A plurality of experts contained in the MOE are deployed in a plurality of artificial intelligence chips in groups, and the method comprises the following steps: carrying out routing calculation on an original input tensor to obtain a routing calculation result; determining input element grouping information corresponding to each expert based on expert index information and expert weight information in the routing calculation result; taking the expert dimension as a parallel dimension, executing rearrangement operation for the original input tensor in parallel based on the input element grouping information, taking a rearrangement result as input information of an expert, executing matrix multiplication and accumulation operation, and taking the expert dimension as the parallel dimension, and based on the input element grouping information, executing anti-rearrangement operation of the matrix multiplication and accumulation operation result in parallel to obtain a final operation result. By adopting the method, the MOE reasoning performance can be improved.

The invention discloses an attention mechanism calculation method and device, a storage medium and a product, and the method comprises the steps: carrying out matrix multiplication operation through employing a query matrix block of a first register block and a key matrix block of a shared memory, obtaining a first product matrix block, and writing the first product matrix block into a second register block; performing exponential operation by using the first product matrix blocks to obtain sub-matrix blocks, writing the sub-matrix blocks into a second register block in a covering manner, and writing the sub-matrix blocks into a third register block in the form of a target precision type; performing matrix multiplication operation by using the sub-matrix blocks of the third register block and the value matrix blocks of the shared memory to obtain second product matrix blocks, and writing the second product matrix blocks into a second register block; performing softmax operation by using the second product matrix blocks to obtain attention result matrix blocks, and writing the attention result matrix blocks into a fourth register block; and writing the attention result matrix of the fourth register group into the shared memory in blocks. According to the embodiment of the invention, overflow of the register can be avoided, and the utilization of hardware resources is maximized.

The invention discloses an attention mechanism calculation method and device, a medium and a product. The method comprises the steps that a second thread bundle group is controlled to load an ith query matrix block; controlling the second thread bundle group to perform matrix multiplication operation and exponential operation by using the ith query matrix block and the transposed jth key matrix block to obtain a jth attention score matrix block; and controlling the first thread bundle group and the second thread bundle group to alternately use different sub-blocks of the jth value matrix block to perform attention mechanism operation of the jth attention score matrix block until the last sub-block of the jth attention result matrix block is obtained. By adopting the embodiment of the invention, sufficient register resources can be provided for the calculation of the attention mechanism, so that the calculation efficiency is improved.

The embodiment of the invention provides a matrix multiplication and accumulation operation unit and method, a hardware accelerator and electronic equipment, and the matrix multiplication and accumulation operation unit comprises a data loading storage engine, a tensor register file and a matrix multiplication engine. The data loading and storage engine is used for loading data of a plurality of matrixes to be subjected to matrix multiplication and accumulation calculation; the tensor register file is used for storing data of a plurality of matrixes acquired from the data loading storage engine; the tensor register file comprises at least three tensor register groups, each tensor register group comprises a plurality of tensor registers, and different tensor register groups are used for storing data of different matrixes in the plurality of matrixes; and the matrix multiplication engine is used for carrying out matrix multiplication accumulation calculation based on the data of the plurality of matrixes stored in the tensor register file. According to the embodiment of the invention, more efficient MMA calculation is realized under the conditions of low cost, low power consumption and less occupied space.

The invention discloses a multi-precision matrix calculation unit and a use method thereof, and relates to the technical field of integrated circuits, the multi-precision matrix calculation unit comprises a control logic unit, a cache module and a calculation array; wherein the control logic unit is used for configuring a computing array and controlling the cache module to read in and read out data; the cache module comprises three first cache sub-modules and one second cache sub-module; wherein the three first cache sub-modules are respectively used for caching a first to-be-processed matrix, a second to-be-processed matrix and a third to-be-processed matrix, and the second cache sub-module is used for caching a result matrix; and the calculation array is used for carrying out multiplication and addition operation on different types of to-be-processed matrixes to obtain a result matrix and writing the result matrix back to the cache module. The method can serve as a basic operation core for integrated processing of large-scale matrix multiplication and can also be integrated in processors such as RISC-V for matrix operation, universality is higher, and the method can adapt to fast and efficient calculation scenes.

The invention discloses a multi-modal data driven dynamic pressure test scene generation method and system, and the method comprises the steps: carrying out the event extraction and matching of a user operation log and a full-link API call sequence, and carrying out the feature enhancement of each alignment event pair, and obtaining a local feature vector; statistical features are extracted from each user session, a graph attention network is constructed, and a group feature matrix is obtained; obtaining a global incidence matrix according to the mapping between the service load and the resource consumption; and splicing the user session feature matrix and the group feature matrix to generate group enhancement features, performing matrix multiplication on the group enhancement features and the global incidence matrix to obtain system-level risk features, splicing local feature vectors and the encoded user behavior logs, splicing the system-level risk features and the encoded performance indexes, and obtaining the user behavior log. And finally, carrying out multi-source fusion to generate a pressure measurement scene. The simulation precision, the dynamic adaptive capacity, the abnormal reproduction capacity, the resource utilization rate and the like are remarkably improved.

The invention discloses an input data sharing and cache optimization method in matrix multiplication calculation. The method comprises the steps of 1, segmenting and distributing an input data matrix and a weight matrix according to the number N of calculation cores on a chip; 2, sequentially connecting the plurality of calculation cores end to end to form a data transmission annular structure; step 3, calculating the distributed matrix multiplication by each calculation core, and transmitting the current input sub-matrix of the calculation core to the next calculation core; step 4, performing matrix multiplication operation on the transmitted input sub-matrix and the weight sub-matrix in the next calculation kernel; and 5, iterating transmission and calculation of the input sub-matrixes, and carrying out N rounds of matrix multiplication of the input sub-matrixes and the weight sub-matrixes to complete the whole operation process. The invention further discloses a system for implementing the method, and the system has wide application value.

The invention discloses a matrix multiplication task execution method and device, equipment, a medium and a program product, the method is applied to a processor unit in a graphics processor, and the method comprises the steps that in the process that the processor unit executes a matrix multiplication task, the processor unit executes the matrix multiplication task; distributing a target register for the matrix multiplication task in the processor unit, wherein the target register comprises a shared register; generating at least two types of second thread bundles for executing the matrix multiplication task; the at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the common register; and the common register is used for storing an intermediate result of the matrix multiplication task.

The invention relates to the technical field of artificial intelligence, and provides a matrix multiplication implementation method and device, electronic equipment, a storage medium and a program product.The method comprises the steps that calculation cores on a chip are divided into a plurality of calculation groups based on a plurality of matrix multiplication operations needing to be executed at the same time, each calculation group at least comprises two calculation cores, each calculation group corresponds to one matrix multiplication operation; and controlling each calculation group to execute the corresponding matrix multiplication operation, and obtaining a result matrix of each matrix multiplication operation. According to the method, the calculation cores are grouped, and each calculation group executes one matrix multiplication operation in parallel, so that the function of executing a plurality of matrix multiplication operations in parallel on the chip is realized; the number of calculation cores in each calculation group is reduced relative to the whole chip, one matrix in each matrix multiplication operation is divided into a smaller number of sub-matrixes, and the number of rows of the sub-matrixes is large, so that the number of rows of the sub-matrixes can cover the minimum calculation granularity of the calculation cores, and the utilization rate of the calculation cores is improved.

To reduce programming noise for matrix values stored in a memory array for use in an in-array vector-matrix multiplication, such as for a neural network, the matrix is partitioned into a linear combination of matrices that will preserve the output after the combination, with the small value component matrices being normalize to the lager, full range of values before being programmed into the memory arrays. After multiplying each matrix of the combination with the vector by applying a set of bias values, the outputs are rescaled to undo the normalization before adding the individual outputs back together for the final output. This re-scaling causes the effective noise of small weights to be reduced, providing large noise tolerance for these small weight values.

The invention belongs to the technical field of data processing. The invention discloses a matrix multiplication calculation task processing system, method and device, a storage medium, a program product and a chip, and the system comprises a storage module which is used for storing a preset weight matrix in advance; the linear transformation control module comprises a finite-state machine used for matrix multiplication operation scheduling and is used for generating an input request signal so that the matrix multiplication calculation task processing system can receive an input matrix; the operation module is used for executing matrix multiplication operation on the input matrix and a preset weight matrix to obtain a matrix multiplication operation result and generate an output matrix; and the data flow control module is used for controlling a plurality of groups of data flows formed by the input data in the input matrix according to rows to sequentially enter the operation module according to the input request signal until all the data in the input matrix execute and complete the matrix multiplication operation. The problems that in the prior art, a matrix multiplication accelerator lacks flexibility in a linear layer implementation task of artificial intelligence hardware, the data migration cost is high, and the energy efficiency ratio is low are solved, and the method is suitable for self-defined extension based on an RISC-V instruction set architecture, can be used as a coprocessor of an RISC-V processor, and can be used as a coprocessor of the RISC-V processor. And linear layer calculation in the neural network is accelerated, especially in an artificial intelligence task, the calculation efficiency can be remarkably improved, the power consumption can be reduced, and the real-time processing requirement can be met.

The invention relates to a method for calculating matrix multiplication, an artificial intelligence chip, a calculation device, a medium and a program product. The method comprises the following steps of: accumulating a calculation result of a current cycle calculation and a calculation result of a previous cycle calculation of matrix multiplication executed in a thread bundle group granularity by utilizing a buffer which is configured in a calculation core and is used for accumulation operation; determining whether the last cycle calculation of the matrix multiplication performed at the thread bundle group granularity is completed; and in response to determining that the last cycle computation of the matrix multiplication performed at the thread bundle group granularity is completed, writing the computation results accumulated via the buffer to the register file. According to the invention, the write bandwidth of the register and the occupation of the register space can be obviously reduced.

The invention discloses an attention mechanism calculation method and device, a storage medium and a product, and the method comprises the steps: carrying out the matrix multiplication operation of a query matrix block and a key matrix block, obtaining a first product matrix block of a first precision type, writing the first product matrix block into a second register group, and then carrying out the index operation, and obtaining a molecular matrix block; writing the quantized sub-matrix blocks into a third register block according to a second precision type; writing the sub-matrix blocks into a fourth register block according to a third precision type, and performing layout conversion through a shared memory; performing softmax operation by using the molecular matrix blocks and the value matrix blocks after layout conversion to obtain attention result matrix blocks of a first precision type, and writing the attention result matrix blocks into a fifth register block; and writing the attention result matrix blocks into the shared memory according to the third precision type. The embodiment of the invention can reduce the accumulative error of the intermediate operation result.

The invention provides an acceleration processing method and device for sparse matrix vector multiplication. The method comprises the following steps: acquiring a sparse matrix; dividing the sparse matrix into segments, and distributing threads for the segments; a matrix multiplication-accumulation instruction in a preset instruction set architecture is called, a tensor calculation core Tensor Core is used for carrying out matrix multiplication calculation of small blocks on the fragments, result data are obtained, and the result data are used for representing vector data objects of the linear equation set solution vectors. According to the method, the obtained sparse matrix is divided into the fragments and the threads are distributed, so that Tensor Core concurrent calculation is realized, the calculation efficiency is greatly improved, and the calculation time is remarkably shortened especially for solving a large-scale linear equation set.

The invention belongs to the technical field of wind turbine fault diagnosis. According to the mechanism data fusion-based wind turbine fault diagnosis method and system, a feature sequence, a time sequence feature, a residual error and a differential residual error are spliced and fused to obtain a fused feature vector; linearly mapping the fused feature vector to a high-dimensional hidden space, performing matrix multiplication among the features in the high-dimensional hidden space to obtain similarity distribution among the features, and normalizing to obtain an adaptive adjacent matrix; and carrying out matrix multiplication on the adaptive adjacency matrix and the features in the high-dimensional hidden space again, carrying out normalization to obtain a coupling feature matrix, adding the adaptive adjacency matrix and the coupling feature matrix to obtain an interactive feature vector, and obtaining a wind turbine fault diagnosis result according to the interactive feature vector. According to the method, adaptive modeling of different feature importance is realized, and misjudgment caused by environmental noise and working condition fluctuation is reduced.

The invention discloses a spatial data management method and system based on artificial intelligence, and relates to the technical field of spatial data management, and the method comprises the steps: defining the number of wavelet decomposition layers, calculating a self-adaptive threshold value, screening edges of an incidence matrix, generating a sparse incidence matrix, constructing a projection matrix, and projecting a feature matrix to a feature vector space through employing a matrix multiplication method. Generating a comprehensive feature matrix; and using a logistic function to define a nonlinear stream function, forming state vectors, calculating a mean value of the state vectors, fusing the mean value with the node feature matrix to form a dynamic feature matrix, and constructing a graph convolutional network model to predict the fault probability of the nodes. According to the method, fine-grained dynamic features of spatial data are accurately mined through combination of multi-scale wavelet decomposition and adaptive threshold screening, feature energy screening and a cosine distance matrix are introduced, the discrimination ability of feature selection and dimension reduction is improved, and the spatial data are extracted by using the graph convolutional network and combining a sparse embedding mechanism. And the generalization ability of the model in node fault prediction is obviously enhanced.

A computer processor may include a number of cores, a shared cache shared among the cores, and a local cache associated with each core and used by that core only. Input data for a neural network (NN) layer may be partitioned into a set of tiles of size T×T, and the tile set may be partitioned into blocks of R tiles. For each block, a core may perform a transform operation on the tiles to produce transformed data matrices fitting in a local cache, and a set of multiply operations, each multiply operation using a transformed data matrix and a transformed kernel matrix from a set of transformed kernel matrices. The set of transformed kernel matrices may fit in the shared cache. The result of at least one of the multiply operations may be stored in a location used to store a transformed data matrix.

The invention provides a matrix multiplication operation method and device and a matrix transposition operation method and device of a processor. The matrix multiplication operation method of the processor comprises the following steps: taking a first matrix of which the storage sequence is inconsistent with the loading sequence for carrying out element-by-element multiplication operation as a second matrix, loading the second matrix through a register set in the processor, and dividing the second matrix into a plurality of third matrixes, replacing elements in a plurality of nth register groups included in the register set; for each nth register group, reading elements in the nth register group according to an alternating sequence of the elements of the first register and the elements of the second register through the (n + 1) th register group; when iteration is carried out for n to N-1, the obtained sixth matrix is stored in the memory; and performing element-by-element multiplication and accumulation on the elements at each position of the first matrix and the sixth matrix. According to the invention, the time required by matrix multiplication operation can be reduced.

A data processing apparatus includes a first vector register and a second vector register, both of which are dynamically spatially and dynamically temporally divisible. A decode circuit receives one or more matrix multiplication instructions indicating a set of first elements in the first vector register and a set of second elements in the second vector register, and generates a matrix multiplication operation in response to receiving the matrix multiplication instructions. The matrix multiplication operation causes one or more execution units to perform a matrix multiplication of the set of first elements and the set of second elements, and an average bit width of the first elements is different from an average bit width of the second elements.

A simulation system and method for implementing a model based on an iterative neural network, the system comprising: a simulation vector-matrix multiplication circuit that encodes a weight matrix of the model based on the iterative neural network; and an analog non-linear circuit that encodes a non-linear function arranged in a feedback loop configured to return an output signal from the non-linear circuit as input to the vector-matrix multiplication circuit, wherein the system is configured to output a solution vector of values of the model based on the iterative neural network upon convergence of the system.

A video bitstream including coded information of a current block in a current picture is received. The coded information indicates a plurality of candidate intra prediction modes for the current block. Two or more predictors are determined based on the plurality of candidate intra prediction modes for the current block according to a pre-defined condition. At least one of the two or more predictors is generated based on a matrix-multiplication mode of the plurality of candidate intra prediction modes such that the at least one of the two or more predictors is obtained by a matrix multiplication of a matrix of weight coefficients and neighboring reconstructed samples in a template of the current block. The current block is reconstructed based on a weighted combination of the two or more predictors.

The invention provides an operation method of an attention mechanism in a chip, the chip, electronic equipment, a storage medium and a program product. In a forward stage of attention model training, forward calculation is performed on a query matrix, a key matrix and a value matrix in each thread of a calculation engine based on a first instruction pipeline to obtain a forward output matrix, in algorithm implementation, matrix multiplication is executed by calling a matrix multiplication unit through a first thread and a third thread, and a forward output matrix is obtained. The vector calculation is processed by a second thread calling vector calculation unit; in a reverse phase, performing reverse calculation on the query matrix, the key matrix, the value matrix and the output gradient matrix in each thread based on a second instruction pipeline to obtain a target gradient matrix; the matrix multiplication unit is called in the first thread and the third thread to execute matrix multiplication, the vector calculation unit is called in the second thread to execute vector calculation, and the data carrying unit is called in the idle first thread or the third thread to obtain all matrixes. According to the invention, the operation performance of the chip can be improved.

A video coder is configured to receive a first block of video data to be coded using intra prediction, and code the first block of a video data using neural network-based intra prediction model, wherein the neural network-based intra prediction uses a model that includes one or more of: dense matrix multiplications, decoupled training and inference, fusion of normalization layers into matrix multiplication, or rectified linear activations.

The invention provides a quantum circuit system based on an AES cryptosystem and an encryption method. The quantum circuit system based on the AES cryptosystem comprises a byte replacement module which adopts a pre-optimized S-box query circuit to realize non-linear byte replacement; the row shift module completes in-row byte cyclic shift through quantum bit exchange operation; the column confusion module realizes a matrix multiplication circuit based on finite field operation, and a quantum circuit of the column confusion module is generated by a depth-first matrix decomposition algorithm; and the round key adding module realizes XOR operation of a round key and a state by using a CNOT gate array, and a key required by the round key adding module is dynamically generated through a key expansion circuit. And a compact Z-shaped architecture is adopted to optimize time sequence scheduling and resource allocation of each module. On the premise of keeping the quantum bit resource usage amount equivalent to that of an original Z-shaped architecture, the quantum circuit depth is remarkably reduced, and higher fidelity and shorter execution time can be realized on NISQ equipment.

Matrix multiplications in deep neural networks (DNNs) may be converted to convolutions executed by DNN accelerators. A matrix multiplication may have a first input tensor and a second input tensor. The first input tensor may be converted to an activation tensor of a convolutional operation. The second input tensor may be converted to a weight tensor of a convolutional operation. The conversion of the first input tensor to the activation tensor or the second input tensor to the weight tensor may include tensor transposing followed by tensor reshaping. The DNN accelerator may perform the convolutional operation on the activation tensor and weight tensor. An output tensor of the convolutional operation may be converted to an output tensor of the matrix multiplication. The conversion from the output tensor of the convolutional operation to the output tensor of the matrix multiplication may include tensor reshaping followed by tensor transposing.