18 results about "K matrix" patented technology

The invention discloses a storage-efficient large model reasoning method for many-core supercomputing, and relates to the field of machine learning. The overall architecture of the method is based on video memory optimization of Attention and FFN modules in a Transform model, and high storage efficiency in a reasoning stage is realized through matrix block calculation and dynamic video memory management. The method comprises the following steps: firstly, carrying out parameter partitioning and serial calculation: in an Attention module, vertically cutting a Q and K parameter matrix into a plurality of sub-blocks along a column direction, and keeping the input complete; serially calculating the product of each sub-block and the input to obtain a local Q matrix and a local K matrix, and immediately performing QKT block multiplication to obtain a partial attention score; aggregating calculation results of all the sub-blocks to obtain a complete attention score matrix; afterwards, normalizing the attention score by using Softmax, carrying out serial multiplication on the normalized attention score and a V block subjected to delay calculation, and splicing a result to obtain Attention output; optimizing an FFN module, vertically cutting parameters of a full connection layer into sub-blocks, inputting complete data, sequentially carrying out serial calculation on the complete data and the sub-blocks, and splicing nonlinear transformation results of all the sub-blocks in real time.

The embodiment of the invention provides a method and device for calculating a matrix multiplied by a vector, computing equipment and a storage medium, the matrix is a matrix of M * K, the vector is a vector of K * 1, and M and K are positive integers. The method comprises the following steps: converting an M * K matrix into a first tensor of M * (K / N) * N, N being a positive integer, Ngt; 1 and K are multiples of N; converting the vector of K * 1 into a second tensor of N * (K / N); using a tensor calculation kernel to carry out general matrix multiplication calculation of a first tensor of M * (K / N) * N and a second tensor of N * (K / N) in batches to obtain M result matrices; elements on diagonals of each of the M result matrices are added to result in a matrix of M * K multiplied by each element on a result vector of M * 1 of the vector of K * 1. According to the scheme, high-throughput and low-delay high-speed matrix vector multiplication (MMV) operation is realized by utilizing the tensor calculation kernel.

Embodiments of the present application relate to the technical field of artificial intelligence. Disclosed are a method and apparatus for attention calculation in an artificial intelligence model. The method comprises: acquiring a plurality of elements requiring attention calculation; generating a Q matrix, a K matrix, and a V matrix required for attention calculation and corresponding to the plurality of elements; determining a position of valid data in a relation matrix corresponding to the Q matrix and the K matrix; on the basis of the determined position of the valid data in the relation matrix, determining calculation data for matrix multiplication calculation in the Q matrix and the K matrix; performing matrix multiplication calculation on the determined calculation data in the Q matrix and the K matrix to obtain the relation matrix; and performing an operation on the relation matrix and a V matrix to implement attention calculation. By means of the present application, invalid data can be prevented from participating in attention calculation, thereby improving the attention calculation efficiency.

The invention discloses a matrix multiplier, a chip, electronic equipment, a data processing method, a computer readable storage medium and a computer program product, and belongs to the field of artificial intelligence. The matrix multiplier comprises a multiplier array, a data preprocessing module and an accumulator; the multiplier array is configured to perform multiplication operation on an input first matrix and a second matrix; the data preprocessing module is configured to preprocess the third matrix to obtain a mask matrix; the accumulator is configured to add the multiplication result of the first matrix and the second matrix and the mask matrix; wherein the first matrix is an M * K matrix, the second matrix is a K * N matrix, the third matrix is an M * N matrix, and M, N and K are all integers greater than or equal to 1. According to the method, the mask process is fused into the matrix multiplication operation, so that the matrix multiplication operation and the mask adding operation can be synchronously carried out, and the calculation rate of the Attention (attention) operation is effectively improved.

The embodiment of the invention discloses an attention calculation method and device in an artificial intelligence model, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of elements which need to be subjected to attention calculation; generating a Q matrix, a K matrix and a V matrix required by attention calculation corresponding to the plurality of elements; and determining the positions of effective data in the relation matrix corresponding to the Q matrix and the K matrix in the relation matrix. And determining calculation data for matrix multiplication calculation in the Q matrix and the K matrix according to the determined position of the effective data in the relation matrix. And performing matrix multiplication calculation on the determined Q matrix and K matrix calculation data to obtain a relation matrix. And carrying out operation on the relation matrix and the V matrix to realize attention calculation. By adopting the data processing method and device, invalid data can be prevented from participating in the attention calculation, and the efficiency of the attention calculation can be improved.

The invention discloses a cross-water-area bridge deformation solving method based on Beidou No.3 / GNSS inversion, and particularly relates to the technical field of bridge monitoring. The method comprises the following steps: establishing an observation equation by combining all satellites based on satellites which can be observed by a bridge structure monitoring station; solving the observation equation by using a matrix method to obtain a prediction state vector of the deformation quantity of the bridge structure and a covariance matrix of the state vector; in consideration of the influence of gross error of a Beidou No.3 / GNSS observation value, a self-adaptive factor is provided to correct a predicted value of a state vector; obtaining a vector K and an updated state vector covariance; introducing a robust factor M to determine the weight of each type of observation values in consideration of gross errors in the Beidou No.3 / GNSS carrier phase observation values; updating the matrix K and the state covariance matrix by using the corrected measurement covariance matrix R; and calculating the updated state vector. According to the technical scheme, the problem that a conventional Beidou / GNSS data processing method is low in precision is solved, and the precision of bridge deformation calculation is improved.

This invention provides an improved method, system, and computer program for performing matrix multiplication computer instructions. [Solution] The method is a processor having a control register and a vector register whose length is indicated in the control register, in which a matrix operation A × B = C is performed, where A is an M × K matrix with elements of a first bit width, B is a K × N matrix with elements of a second bit width, and C is an M × N matrix with elements of a third bit width. The instruction has three parameters that encode the type of multiplication to be performed and indicate which vector register to sequentially store the elements of each matrix in. The method also obtains the length from the control register, decodes the instruction to obtain the multiplication type, determines the first and second bit widths according to the multiplication type, determines the values ​​of M, N, and K according to the length and the first and second bit widths, and performs matrix multiplication considering the three parameters, the first and second bit widths, and the values ​​of M, N, and K.

The application discloses a kind of converter electromagnetic transient simulation parallel optimization method, equipment and storage medium, belong to power system electromagnetic transient simulation technical field, by merging significantly reduces unnecessary intermediate variable, shortens simulation calculation process;And since C, D matrix and Un, k matrix multiplication operation with Itotal of A, B matrix and multiplication are carried out simultaneously, and the calculation process between row and row is independent, increase the parallel degree of electromagnetic transient simulation calculation;Since parameter matrix does not change with the change of switch state, it is calculated and stored in advance in simulation initialization stage, only needs to consume register reading operation time in the calculation of single simulation step;Coefficient matrix A, B, C, D are all square matrix of scale Nb*Nb, the formula for calculating Ub and Ib is the same in structure, to lay the algorithm foundation for saving logic resources while improving the calculation efficiency of simulation program using pipeline and other hardware development techniques.

Embodiments of the present disclosure provide a method, apparatus, computing device and storage medium for calculating matrix-vector multiplication, where the matrix is an M*K matrix and the vector is a K*1 vector, M and K are positive integers. The method comprises: converting the M*K matrix into an M*(K / N)*N first tensor, where N is a positive integer, N>1, and K is a multiple of N; converting the K*1 vector into an N*(K / N) second tensor; using a tensor computing core to batch-process the general matrix multiplication of the M*(K / N)*N first tensor and the N*(K / N) second tensor to obtain M result matrices; and adding the elements on the diagonal of each of the M result matrices to obtain each element on the M*1 result vector of the M*K matrix multiplied by the K*1 vector. The above scheme uses a tensor computing core to implement high-throughput, low-latency high-speed matrix-vector multiplication (MMV) operation.

The application provides an image segmentation method based on block-level self-attention and block correlation mechanism, comprising the following steps: 1, building a ResNet101 backbone network; 2, dividing the output features of the backbone network into blocks, and using 1*1 convolution to generate a query matrix (Q), a key matrix (K) and a value matrix (V) for each feature block, and using a block correlation mechanism (BCM) to calculate the correlation of the Q matrix of each feature block and the global features; 3, building a self-attention network with relative position offset, embedding the BCM into the self-attention calculation process of each feature block, using the output M matrix of the BCM and the K matrix and the V matrix for self-attention calculation, and introducing a relative position offset for the M, K and V matrices of each feature block to jointly form a block-level self-attention (BLSA) module; 4, splicing the output of each feature block in the spatial dimension and sending it into a segmentation head for segmentation, and performing experimental comparison.

A method for estimating delays and Doppler shifts of GNSS signals comprises obtaining respective output sequences from N correlators for a snapshot of a received GNSS signal, wherein each sequence has K values, the N correlators correspond to N different delays of a pseudorandom noise, PRN, code sequence, the K values are correlation values at K sampling time points in each sequence, N and K are positive integers. The method further comprises, for each output sequence, obtaining a frequency spectrum by computing a K-point discrete Fourier transform, DFT, of the output sequence; obtaining a N×K matrix using the N frequency spectra; determining at least one peak value in the N×K matrix; and estimating a delay and a Doppler shift of the received GNSS signal using indices of the at least one peak value in the N×K matrix.

Video coding using reduced-rank matrices for matrix intra prediction (MIP). In a video coding method, information is obtained indicating a matrix rank, k, and a matrix intra prediction mode for a current block. The current block has a block shape, and neighboring reference samples. Based on the intra prediction mode and the current block shape, a first and second matrix are selected from a stored set of intra prediction matrices. Each of the stored matrices is associated with a respective block shape and intra prediction mode. A first rank-k matrix is obtained from the first matrix, and a second rank-k matrix is obtained from the second matrix. The current block is then predicted using a product of the first rank-k matrix, the second rank- k matrix, and the reference samples.

The invention relates to the technical field of electric digital data processing, in particular to a processing method for matrix multiplication. The method comprises the steps that a first matrix A and a second matrix B to be subjected to matrix multiplication are obtained, A is an m * k matrix, and B is a k * n matrix; obtaining a logic coordinate sequence P of the matrix block when mapping is carried out according to a preset initial mapping relation; if mgt; n is greater than stoBlt; the method is characterized by comprising the following steps of: storing; if so, obtaining a logic coordinate sequence P'of the matrix block when mapping is carried out according to the target mapping relation; the target thread blocks are scheduled according to the sequence of the corresponding ordinal numbers from small to large, the thread blocks with the corresponding ordinal numbers being i are used for executing matrix multiplication operation of the matrix blocks with the logic coordinates being p'i, and when matrix multiplication operation is conducted on A and B, B is completely loaded to the cache, and A is partially loaded to the cache. According to the invention, the speed of matrix multiplication operation is improved.

A partial fixed caching method and system based on large model inference, a device, and a medium, relating to the technical fields of artificial intelligence and natural language processing. The technical problem to be solved is how to find a balance between computing resources and storage resources that is more suitable for localized inference of large models, thereby improving inference efficiency. The technical solution used is: a pre-filling stage: inputting a sequence or text, preprocessing same to complete format conversion from the original input sequence or text into a large model input matrix, and caching a K matrix and a V matrix of an original input token in each attention layer on the basis of a KVcache method; and a decoding stage: unlike the conventional KVcache method, before each decoding, pre-determining whether to cache a K matrix and a V matrix of a current input token, and on the basis of the result, determining whether to cache the K matrix and the V matrix of the current input token during the current decoding.

The invention belongs to the technical field of large model security, and particularly relates to an efficient large model full-process encryption method. Comprising the steps of traversing a large model and performing operation deployment; the GPU calculates an attention score matrix and positions important parameters in a Q matrix and a K matrix; partitioning the Q matrix and the K matrix; the GPU extracts important parameters from each block and constructs a block parameter packet; after the TEE verifies the block parameter packet, OTP encryption is carried out on the important parameters, and encrypted blocks are generated; the GPU restores the encryption parameter to a parameter position corresponding to the original block to form a semi-encryption block; executing parallel linear ciphertext operation on the semi-encrypted blocks to generate ciphertext operation result blocks; the TEE performs block decryption on the ciphertext operation result, then performs nonlinear operation, and performs secondary OTP encryption on the nonlinear operation result to generate a secondary encryption result; and the GPU directly uses the secondary encryption result to carry out linear calculation, and then transmits the result back to the TEE to decrypt until a reasoning result is obtained, so that the security of sensitive parameter encryption is remarkably improved, and the efficiency is remarkably improved.

The invention relates to a method for estimating delay and Doppler shift of global navigation satellite system signals. A method of estimating delays and Doppler shifts of a GNSS signal includes obtaining respective output sequences from N correlators for snapshots of a received GNSS signal, where each sequence has K values, the N correlators corresponding to N different delays of a pseudorandom noise (PRN) code sequence, the K values being correlation values at K sampling time points in each sequence, n and K are positive integers. The method further comprises: for each output sequence, obtaining a spectrum by calculating a K-point discrete Fourier transform (DFT) of the output sequence; obtaining an N * K matrix by using the N frequency spectrums; determining at least one peak value in the N * K matrix; and estimating the delay and Doppler shift of the received GNSS signal using an index of the at least one peak in the N * K matrix.

Embodiments of the present application relate to a slot value extraction method and device based on slot meaning of a large model, related information of an mth slot at a current time is obtained; the related information is spliced from a to-be-filled sentence, an original input text and a fill-in-the-blank instruction; the to-be-filled sentence is composed of a slot name, a meaning explanation and a corresponding slot value of a first slot, a slot name, a meaning explanation and a corresponding slot value of a second slot, and a slot name, a meaning explanation and a corresponding slot value of a third slot; based on the related information of the mth slot at the current time, a parameter matrix, and a hidden layer representation vector of a decoder at a previous time, an attention matrix of the mth slot is determined; the parameter matrix includes q matrix, k matrix and v matrix in the attention mechanism; based on the attention matrix of the mth slot, a word table probability distribution corresponding to the mth slot is determined; based on the word table probability distribution, a slot value corresponding to the mth slot is determined, and a corresponding blank symbol is replaced to fill in a completed sentence.

The present application provides a reasoning method, electronic device and corresponding device based on neural network. The method can be executed by a target chip. In the method, at least part of the first matrix is ​​stored in the on-chip memory of the target chip, and in the reasoning process, based on at least part of the first matrix stored in the on-chip memory and the available storage space of the on-chip memory, the Q matrix, K matrix and V matrix are operated. When performing reasoning, the scheme provided in the embodiment of the present application can read at least part of the first matrix stored therein from the on-chip memory, which can reduce the amount of data read from the off-chip memory, that is, reduce the number of global simulations, compared with the existing scheme in which the Q matrix, K matrix and V matrix are all stored in the off-chip memory. The speed of reading data from the on-chip memory is faster than the speed of reading data from the off-chip memory. Therefore, the scheme provided in the embodiment of the present application can reduce the time consumption in the reasoning process and improve the reasoning efficiency.