57 results about "Matrix multiplier" patented technology

The invention discloses a logic design system for controller hardware acceleration, which relates to the technical field of controllers and comprises a target column extractor, a long-bit-width parallel comparator, a matrix multiplier and a truth table lookup device. The target column extractor is used for extracting target column data information from a preset page and sending the target column data information to the long-bit-width parallel comparator; the target column extractor does not need to read the column data information one by one, but directly reads the target column data information, so that the reading speed can be increased. Further, the long-bit-width parallel comparator is used for copying the target column data information to the corresponding long-bit-width register according to the data type, obtaining a comparison result according to the comparison operation instruction code and the target column data information, and sending the comparison result to the matrix multiplier; as the long-bit-width parallel comparator can compare the target column data information in parallel, the throughput rate of database calculation can be effectively improved, and the delay rate is reduced.

Aspects of the disclosed technology include techniques and mechanisms for using a custom scratchpad memory for partial dot product reductions. The custom scratchpad memory may be a special purpose memory that is dedicated to receiving and storing partial dot products determined by matrix multiplier units. Each partial dot product may correspond to tiles of a resultant matrix, where the resultant matrix is the product of matrix multiplication that can use a first matrix representing a user query as a left-hand side operand and a second matrix representing a trained model containing data that may be used to respond to the user query as a right-hand side operand. The custom scratchpad memory may append the tiles determined by the matrix multiplication, where the appended tiles may create the resultant matrix. Custom scratchpad memory may write the resultant matrix to general purpose memory, where it may be used to respond to the user query.

A matrix multiplier includes an input vector scaler configured to generate a scaled input matrix based on a first input matrix and a plurality of scaling factors; a first data type converter configured to convert a data type of the scaled input matrix into a fixed point and generate a fixed point input matrix; a multiply-accumulate operator array configured to receive the fixed-point input matrix and the plurality of binary vectors, generate a fixed-point output matrix based on the fixed-point input matrix and the plurality of binary vectors, generate a first input matrix and a second input matrix, and generate a first output matrix based on the first input matrix and the second input matrix; and a second data type converter configured to convert a data type of the fixed point output matrix into a floating point and generate a second output matrix.

The invention discloses a matrix multiplier, a chip, electronic equipment, a data processing method, a computer readable storage medium and a computer program product, and belongs to the field of artificial intelligence. The matrix multiplier comprises a multiplier array, a data preprocessing module and an accumulator; the multiplier array is configured to perform multiplication operation on an input first matrix and a second matrix; the data preprocessing module is configured to preprocess the third matrix to obtain a mask matrix; the accumulator is configured to add the multiplication result of the first matrix and the second matrix and the mask matrix; wherein the first matrix is an M * K matrix, the second matrix is a K * N matrix, the third matrix is an M * N matrix, and M, N and K are all integers greater than or equal to 1. According to the method, the mask process is fused into the matrix multiplication operation, so that the matrix multiplication operation and the mask adding operation can be synchronously carried out, and the calculation rate of the Attention (attention) operation is effectively improved.

To reduce the area requirements for sensing circuits of 3D NAND-based vector-matrix multiplication circuitry where weight values for a neural network are stored differentially as current levels on pairs of memory cells, techniques are presented for reducing the common mode current levels during sensing operations. When discharging a first capacitor through a first of a memory cell of a pair of memory cells storing a weight value by a first bit line and discharging second capacitor through a second memory cell of the pair by a second bit line, a reference current is applied to the bit lines. The product of a weight value with an input vector values is then determined by comparing the voltage levels on the two capacitors. The use of the reference current reduces the amount of voltage swing in the two capacitors, reducing the size requirements for the capacitors.

Systems and methods for a new field programmable gate array (FPGA) architecture that is optimized for machine learning (ML) applications are provided. Such ML applications can specifically include, for example, artificial neural networks and deep neural networks. Various embodiments enable the design of faster and more power efficient hardware accelerators for machine learning algorithms, compared to existing FPGAs in the market. This is made possible by hard systolic matrix multiplier blocks, hard activation blocks and soft ML-centric configurable logic blocks. The matrix multiplier blocks are connected to field programmable interconnect resources to enable creation of larger matrix multipliers. The hard matrix multipliers and the hard activation blocks have programmable interconnects between them and neighboring memory or compute blocks on the device.

A matrix multiplier includes an input vector scaler generating a first quantization scaled input vector based on a first input vector, a plurality of common scale coefficients, and first-to-Rth multiplication scale coefficients, a first data type converter generating a first fixed point quantization scaled input vector based on the first quantization scaled input vector, an element array comprising a first processing element generating a first fixed point output element based on the first fixed point quantization scaled input vector and first plurality of quantization sign bits, and a second processing element generating a second fixed point output element based on the first fixed point quantization scaled input vector and second plurality of quantization sign bits, and a second data type converter generating and outputting first and second output elements by converting data types of the first and second fixed point output elements.

A matrix multiplier includes an input vector scaler generating a first quantization scaled input vector based on a first input vector, a plurality of common scale coefficients, and first-to-Rth multiplication scale coefficients, a first data type converter generating a first fixed point quantization scaled input vector based on the first quantization scaled input vector, an element array comprising a first processing element generating a first fixed point output element based on the first fixed point quantization scaled input vector and first plurality of quantization sign bits, and a second processing element generating a second fixed point output element based on the first fixed point quantization scaled input vector and second plurality of quantization sign bits, and a second data type converter generating and outputting first and second output elements by converting data types of the first and second fixed point output elements.

A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.

Systems, apparatus, articles of manufacture, and methods are disclosed. An example apparatus includes a Vector Processor Unit (VPU) comprising: first vector lane circuitry including first matrix multiplier circuitry; second vector lane circuitry including second matrix multiplier circuitry; and interconnect circuitry to connect the first vector lane circuitry and the second vector lane circuitry in a ring structure.

The invention discloses a structured sparse matrix multiplier realized based on FPGA primitives, and belongs to the technical field of FPGA hardware acceleration and deep learning computing architecture. The multiplier is composed of a register area and a multiplication and addition area, and the core design thought is that a circuit is built in a customized mode by directly calling bottom layer physical resources based on FPGA primitives; the register area stores a dense matrix B and supports parallel reading of elements by utilizing the characteristic that LUT in an SLICEM can be configured to be double SRL16E; the multiplication and addition area refers to a partial product generation unit based on the 4-Booth algorithm and an improved GPC (4: 2) compressor structure, and efficient generation and rapid accumulation of partial products are achieved. According to the method, redundant loss caused by high-level HDL logic synthesis is avoided through precise physical resource binding of primitives, fine wiring constraint and structured sparse data characteristic adaptation.

A matrix multiplier includes an input vector scaler configured to generate a first scaled input vector based on a first input vector and a plurality of quantization scale coefficients, a first data type converter configured to generate a first fixed-point scaled input vector based on the first scaled input vector, a processing element array including a first processing element configured to generate a first fixed-point output element based on the first fixed-point scaled input vector and first plurality of quantization sign values and a second processing element configured to generate a second fixed-point output element based on the first fixed-point scaled input vector and second plurality of quantization sign values, and a second data type converter configured to generate first and second output elements by converting data type of the first and second fixed-point output elements, and to output a first output vector including the first and second output elements.

The embodiment of the invention provides an optimization method and device for a matrix multiplier, a storage medium and a program product, and relates to the technical field of artificial intelligence, and the method comprises the steps: determining target layout modes of a plurality of input matrixes of the matrix multiplier in a memory and a target multiplication mode of the matrix multiplier before the matrix multiplier is executed, therefore, the target layout mode and the target multiplication mode support that a plurality of input matrixes are read from a memory in a continuous memory access mode in the execution process of the matrix multiplier. In this way, in the execution process of the matrix multiplier, the artificial intelligence chip can read a plurality of input matrixes in a continuous memory access mode (namely, read data continuously stored in a memory) to carry out matrix multiplication calculation; compared with the reading of non-continuous storage data, the reading performance of the artificial intelligence chip on the continuous storage data is higher, so that the data reading efficiency in the execution process of the matrix multiplier is effectively improved, and the calculation efficiency of the matrix multiplier is improved.

To improve the speed of the 3D NAND-based vector-matrix multiplication circuitry, the circuit is designed such that the charge is accumulated on a slave capacitor which is not directly connected to the array. The master capacitor is reset in each cycle, reducing the maximum swing on the bit lines and, hence, reduce the capacitance size. The reduction of the capacitance size will allow the vector-matrix multiplication to run much faster because of smaller interconnect parasitics. A second set of aspects is based on modification of timings of each operation phase. Rather than equal time slots dedicated to different operation phases (e.g., the integration and scaling), the circuit is modified such the masking and scaling phases would be executed much faster as they typically have a much faster time constant.

The invention discloses an extensible 2 * 2 processing unit matrix multiplier and a Transform acceleration method, when a control module executes an OS data stream mode, each processing unit performs multiplication and addition processing according to a received activation value and a weight value to obtain a first multiplication and addition result; each processing unit performs multiplication of an activation value and a weight value according to the processing unit interconnected with the processing unit to obtain a first operation result, and the first operation result is accumulated with the first multiply-add result to obtain a product result of two matrixes; when the control module executes the WS data stream mode, each processing unit performs multiplication and addition processing according to the received activation value and weight value to obtain a second multiplication and addition result; and each processing unit locks the weight value to be unchanged, performs multiplication operation with the activation value according to the processing unit interconnected with the processing unit to obtain a second operation result, and accumulates the second operation result with the second multiply-add result to obtain a matrix multiplication result based on the fixed weight, so that the efficiency ratio and the flexibility are improved.

This invention discloses a method, apparatus, and medium for implementing a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in an FPGA. The method includes: designing matrix partitioning strategies at different levels based on the parallelism of the AI ​​engine array resources on the Versal ACAP platform; data scheduling and multiplexing based on the data packet stream and data packet exchange of the AI ​​engine and AXI stream, accelerating the kernel through matrix partitioned multiplication, and designing data scheduling and multiplexing strategies; and implementing a high-throughput vectorized matrix multiplication pipeline on the AI ​​engine vector processor. This invention achieves multi-level partitioning of matrix multiplication based on the AXI stream transport protocol and AI engine array on the Versal ACAP platform, enabling efficient utilization of hardware resources, effectively improving data reuse rate, and achieving high parallelism, while achieving high computational speed under the high-speed clock of the AI ​​engine. This invention can be widely applied in the field of high-performance computing.

The application discloses an extensible 2*2 processing unit matrix multiplier and a Transformer acceleration method, when a control module executes an OS data flow mode, each processing unit respectively performs multiplication and addition processing according to received activation values and weight values, and obtains a first multiplication and addition result; each processing unit respectively performs multiplication operation of activation values and weight values according to processing units interconnected with the processing unit, and obtains a first operation result, and the first operation result is accumulated with the first multiplication and addition result to obtain a two-matrix product result; when the control module executes a WS data flow mode, each processing unit respectively performs multiplication and addition processing according to received activation values and weight values, and obtains a second multiplication and addition result; each processing unit locks the weight values unchanged, and respectively performs multiplication operation of the activation values according to the processing units interconnected with the processing unit, and obtains a second operation result, and the second operation result is accumulated with the second multiplication and addition result to obtain a matrix multiplication result based on fixed weight, and the efficiency ratio and flexibility are improved.

Systems and methods for optical matrix multiplication are disclosed. An optical matrix multiplier includes an optical source configured to produce a sequence of optical pulses based on an input vector. The multiplier further includes a fanout module configured to receive a sequence of optical pulses and produce multiple optical pulse sequences. The multiplier further includes multiple delay lines, each delay line configured to apply a delay to an associated one of the optical pulse sequences. The multiplier further includes multiple modulators, each modulator configured to modulate one of the delayed optical pulse sequences. The multiplier further includes at least one accumulator configured to sum at least one modulated optical pulse sequence.

The invention relates to a quantification method, system and device based on neural network hardware and a storage medium, and relates to the field of neural networks. The method comprises the following steps: receiving initial data of which the data type is BF16; performing layer normalization on the initial data to obtain active data of which the data type is BF16; designing a matrix multiplier according to neural network hardware; processing and calculating the activation data according to a matrix multiplier to obtain first data; performing inverse quantization on the first data to obtain inverse quantization activity data; and outputting the inverse quantization activity data. The method has the technical effects that the operation speed of the large model is improved while the performance of the large model is kept.

The invention discloses an optical matrix multiplier, which comprises a laser light source, a grating coupler, a BTO electro-optical modulation array, an MZI waveguide array, a photoelectric detector array and a control unit, and is characterized in that the laser light source generates coherent light with stable wavelength, and the coherent light enters a BTO waveguide after being coupled by the grating coupler and then enters the MZI waveguide array through multi-stage beam splitting; the excellent electro-optical effect of a BTO material is utilized, phase modulation is achieved under electrode driving so as to load matrix element information, and then linear operation of matrix multiplication is completed through interference superposition of an MZI waveguide array. And the output optical signal is converted into an electric signal by the photoelectric detector array and is processed by the control unit to obtain an operation result. The system is compact in structure, high in modulation linearity, high in response speed, low in power consumption and capable of achieving multi-channel parallel optical calculation. Compared with a traditional silicon-based or lithium niobate modulator, higher integration density and stability are achieved, and the calculation efficiency of matrix multiplication in the fields of artificial intelligence, scientific calculation, signal processing and the like is remarkably improved.

The application discloses a general parallel matrix multiplier based on reconstruction calculation. The multiplier comprises a matrix reconstruction module, a multiplication module, a compression module, a shift module and an accumulation module. After matrix data is reconstructed into multiple groups of single-bit data according to bit positions by the matrix reconstruction module, the single-bit data is multiplied in the multiplication module; the multiplication result is compressed by the compression module. The compressor in the compression module can compress six data simultaneously. Finally, the compressed result is shifted by the shift module and accumulated by the accumulation module to obtain the multiplication result of the matrix data. According to the matrix multiplier provided by the application, the compression efficiency can be improved by compressing data by the compression module composed of the compressor; meanwhile, the compressor can also perform partial shift calculation and undertake part of the responsibility of the subsequent shift module, thereby reducing the area cost of the matrix multiplier; by decomposing the matrix data according to bit positions, the calculation of signed numbers and unsigned numbers can be realized.

Embodiments of the present innovation relate to a Fourier dot product analog matrix multiplier device. In one arrangement, the analog matrix multiplier device is configured as a set of analog transistor circuits and can include a set of signal generators and a Fourier dot product circuit. The components of the analog matrix multiplier device are configured to perform a Fourier dot product multiplication process relative to coefficients received by the analog matrix multiplier device. The Fourier dot product analog matrix multiplier device leverages the low power capabilities of analog circuits and a mathematical algorithm to reduce the number of components and power required to perform matrix multiplication.

The invention relates to the technical field of image processing, in particular to a binary full-order polynomial image interpolator and a binary full-order polynomial image interpolator method, the binary full-order polynomial image interpolator is composed of an ROM chip, an RAM chip and a matrix multiplier of a storage matrix, and the input end of an interpolator module is connected with the output end of a low-resolution image sensor; the output end of the interpolated high-resolution image signal is connected with an image display screen or a memory; according to the method, a binary total-order polynomial is used as an image interpolation function, and the term number of the binary total-order polynomial must be the square of a positive integer; according to strict proving, a powerful mathematical theory is used as a support, a reasonable image interpolation method is designed by adopting a binary full-order polynomial, low-resolution images and videos can be converted into high-resolution images and videos to be stored, the images obtained after interpolation are best in effect, clear in image and high in resolution, and the method is suitable for large-scale popularization and application. Distortion cannot be caused, and the requirement for high-quality image processing is met.

In the field of programmable in-memory computing devices for performing vector-matrix multiplication, an accumulator circuit for a charge-redistribution based vector-matrix multiplier is provided. The accumulator circuit provides column summation output voltages, which are linearized as a function of the capacitances of the matrix memory components associated with the summation charge storage component of a given column, by eliminating a non-linearity caused by the parallel connection of capacitances of the summation charge storage component and its associated matrix memory components in the column.

Integrated circuit in the form of an analog convolution plane of a neural network or an analog matrix multiplier, with at least one resistor network (100) having at least two memory cells (200a, 200d) for storing a resistance characteristic value, each having a first contact pair (104a / 106a, 104a / 106b) connected to a source electrode and a drain electrode of a transistor (201) designed as a field-effect transistor, in which a ferroelectric capacitance (203) is electrically connected in series between the front-gate electrode of the transistor (201) and the second contact (109) of the second contact pair, or of a transistor (201) designed as a ferroelectric field-effect transistor or of a transistor (201) designed as a charge-trap transistor, which is part of one of the memory cells (200a, 200d), and is configured toin at least one operating mode, providing an electrical resistance corresponding to the stored resistance characteristic value, wherein first contacts (104a) of the first contact pair of each of the two memory cells (200a, 200d) are directly connected to each other, and second contacts (106a, 106b) of the first contact pair (104a / 106a, 104a / 106b) of each of the two memory cells (200a, 200d) are electrically independent of each other, wherein the memory cells (200a, 200d) each have a second contact pair (108a / 109a, 108a / 109b) which is connected to a front-gate electrode of the transistor (201) and a back-gate electrode of the transistor (201), is electrically independent of the first contact pair (104a / 106a, 104a / 106b), and which is arranged such that by means of suitable electrical Signals via this second contact pair (108a / 109a, 108a / 109b) allow the stored electrical resistance characteristic of the respective memory cell (200a, 200d) to be reversibly changed,at least the two memory cells (200) are each configured to be able to be switched between at least three different memory states by means of suitable electrical signals via the second contact pair (108 / 109).

An image processing apparatus comprising a plurality of sparse transformer units, wherein the sparse transformer units each includes: an extraction unit that: uses a matrix formed such that a plurality of first feature vectors and a matrix formed such that a plurality of second feature vectors to calculate, the difference between the first feature vector and the second feature vector; and, based on the difference, extracts a feature vector that is a computation target; and a transformer processing unit that includes a plurality of matrix multipliers, wherein each of the matrix multipliers: executes matrix multiplication computation for the feature vector that is a computation target; and does not execute matrix multiplication computation and uses a result of the matrix multiplication computation at the second time point for a feature vector that is not a computation target among the first feature vectors.

Aspects of the disclosed technology include techniques and mechanisms for partial dot product reduction using customized temporary memory. The custom temporary memory may be a dedicated memory that is dedicated to receive and store the partial dot product determined by the matrix multiplier unit. Each partial dot product may correspond to a tile of a resulting matrix, where the resulting matrix is a product of matrix multiplications that may use a first matrix representing a user query as a left operand, and a second matrix representing a user query as a right operand. And using a second matrix representing a trained model containing data available to respond to the user query as a right side operand. The custom register memory may append tiles determined by matrix multiplication, where the appended tiles may create a resulting matrix. Customized temporary memory may write the resulting matrix to general purpose memory, where the resulting matrix may be used to respond to a user query.

The invention provides an execution method and device of a matrix multiplier, equipment and a storage medium, relates to the technical field of artificial intelligence, and is suitable for executing the matrix multiplier in parallel through a plurality of calculation thread bundle groups, and the method comprises the following steps: through the same carrying thread bundle group, matrix data required by the plurality of calculation thread bundle groups to execute the matrix multiplier is converted into matrix data required by the plurality of calculation thread bundle groups to execute the matrix multiplier; carrying to an on-chip cache from a video memory according to a loading sequence; wherein the loading sequence is that the preorder data required by the plurality of calculation thread beam groups are loaded in series, and the subsequent data required by each calculation thread beam group are continuously loaded; and respectively scheduling the calculation units through the plurality of calculation thread bundle groups, and carrying out matrix multiplication calculation on the respective required matrix data acquired from the on-chip cache. By continuously loading the subsequent data required by the same calculation thread bundle group, one calculation thread bundle group starts to calculate the subsequent steps by using the calculation unit more quickly, and the result of the calculation thread bundle group is more obviously staggered from the result of the other calculation thread bundle group, so that the calculation unit is prevented from being scrambled.

To provide a matrix multiplier and a matrix multiplication device configured to perform matrix multiplication with a faster speed and with a smaller computation amount.SOLUTION: A matrix multiplier includes: an input vector scaler for generating a scaled input vector based on an input vector and a plurality of quantization scale coefficients; a first material type converter for generating a fixed point scaled input vector based on the scaled input vector; a processing element array comprising a processing element for generating first and second fixed point output elements based on the fixed point scaled input vector and a plurality of quantization sign values; and a second material type converter configured to convert material types of the first and second fixed point output elements to generate first and second output elements and output an output vector including the first and second output elements.SELECTED DRAWING: Figure 7