88 results about "Systolic array" patented technology

The invention belongs to the field of machine learning, discloses an expert parallelism processing method and system for a large language model based on MoE, and realizes efficient parallel processing of the MoE model by dynamically distributing expert quantization bit width and sparse mode, predicting and prefetching to-be-activated expert parameters, grouping tokens to generate task queues and dynamically configuring hardware accelerators. Firstly, an importance score is calculated based on expert historical activation frequency, weight distribution and a model structure, so that quantization precision and a sparse proportion are adaptively allocated, and resource waste and precision loss of a unified strategy are avoided; secondly, predicting an expert to be activated by using a current layer hidden state and a historical activation sequence, loading parameters to a special cache in advance, reducing high-bandwidth memory access, and relieving bandwidth peak scrambling; moreover, tokens are grouped through a token-expert mapping table, a task queue is created, and dynamic configuration of a systolic array is combined, so that the problem of expert heterogeneity after compression is solved, and efficient and parallel hybrid precision matrix operation is ensured.

The invention discloses a self-attention mechanism calculation method, a calculation array, a calculation device, a calculation system and a computer readable storage medium. The self-attention mechanism calculation method is applied to a self-attention mechanism calculation array, the self-attention mechanism calculation array comprises a plurality of product accumulators, the plurality of product accumulators form a systolic array, and the self-attention mechanism calculation method comprises the following steps: performing calculation through the plurality of product accumulators according to a feature tensor and a weight tensor to obtain an intermediate tensor; and a result tensor is calculated according to the intermediate tensor through a plurality of product accumulators. Therefore, frequent memory access of the intermediate tensor between the self-attention mechanism calculation array and the external memory is avoided, the calculation efficiency is improved, and the problems of intermediate tensor multiplexing and storage optimization are solved.

The invention relates to the technical field of hardware acceleration, and provides a convolutional neural network hardware acceleration method and system, and the method comprises the steps: obtaining a convolutional neural network instruction which comprises an img2col instruction, a systolic array multiplication instruction and a col2img instruction; analyzing an operation instruction code and a function code field of each instruction, determining a target register, a source register and calculation parameters, determining an instruction function and generating a control signal; data to be operated are taken out from the source register and transmitted to the corresponding operation module together with the control signal and the calculation parameter to be calculated, and then the data are written back to the target register; through hardware optimization strategies such as multi-level address superposition, systolic array data multiplexing, double-buffer window construction and a parallel comparison tree, extra power consumption caused by multiple times of data loading and storage and intermediate result calculation is reduced, and data handling overhead, calculation delay and control complexity are greatly reduced.

The present disclosure is directed to an apparatus that includes a systolic array having an initial systolic stage that is clocked at a first edge of a clock signal. The apparatus further includes control logic configured to clock a first subset of a plurality of additional systolic stages of the systolic array at the first edge of the clock signal. The control logic is further configured to clock a second subset of the plurality of systolic stages at a second edge of the clock signal.

The present application relates to neural network inference hardware technical field, particularly to a kind of lightweight neural network processor storage architecture collaborative optimization method, comprising: step 1: the difference of on-chip data on bandwidth demand and access mode is analyzed, and the differential single-port storage setting is carried out to the buffer storage of each data in neural network processor;Step 2: the GEMM module of neural network processor is based on semi-pulsating array setting, and ALU operation fusion processing module is used, to reduce the storage bandwidth and bit width of data;Step 3: the storage structure is optimized by introducing the dynamic adjustment strategy based on timing analysis, to output the optimized neural network processor.The present application guarantees the computing performance, significantly improves the area efficiency and access efficiency of storage system, realizes the collaborative optimization of bandwidth, area and power consumption.

The application relates to a system and method for architecture-level fault injection based on systolic array accelerator reliability analysis, relates to the technical field of CNN accelerator analysis, and solves the problems that, in the existing CNN reliability analysis method, the reliability analysis of the software level and the architecture level produces contradictory conclusions, and the analysis from the software level alone cannot meet the actual reliability characteristics of the underlying hardware application, and the like, the system comprises an execution simulator, a fault injection module and an analysis module; the execution simulator is responsible for calculation simulation, the fault injection module customizes and simulates fault generation, and the analysis module performs data analysis and result recording; by adopting the method, period-accurate fault injection can be performed on three types of registers under different systolic array sizes, a plurality of different fault types are supported, and the analysis framework can generate detailed intermediate process data information and result record information; the application is simple to implement, cross-platform, has good expansibility, and meets the application requirements.

The application belongs to the field of information technology and provides a column reconfigurable systolic array for a Transformer model. The main idea is to realize that each column computing unit of the array can work together for a single operator or can be split to work individually for multiple operators. The main scheme includes a reconfigurable computing unit of a two-dimensional network, which is composed of row and column distributed computing units, data is transferred from the previous row to the next row and from the previous column to the next column; a register unit of a two-dimensional network, which is composed of row and column distributed register units, data is transferred from the previous row to the next row and from the previous column to the next column, and the data transfer direction is opposite to that of the reconfigurable computing unit of the two-dimensional network; and the register and the computing unit of the same row and column are connected through a data path. A mixed parallel line is supported to improve the hardware efficiency of the Transformer-based model.

A specialized integrated circuit for artificial neural networks is integrated with high bandwidth memory. The neural network includes a systolic array of interconnected processing elements, including upstream processing elements and downstream processing elements. Each processing element includes a pair of input / output ports for concurrent forward and backward propagation. The processing elements can be used for convolution, in which case the pair of input / output ports can support fast and efficient scanning of a kernel over activations.

The invention relates to an improved systolic array computing device capable of improving pipeline efficiency, the device comprises an input characteristic value cache array, a multiply-accumulate array and an output characteristic value cache array, and the multiply-accumulate array is composed of N * N multiply-accumulate units. In order to improve the utilization rate of the array, data paths connected end to end are added in the horizontal direction and the vertical direction respectively, so that the array forms a Torus annular interconnection structure; input characteristic values are injected through diagonal positions, partial sum results are annularly accumulated in the vertical direction, so that continuous flow of input and partial sum is achieved, the structure remarkably reduces assembly line filling and emptying delay, hardware redundancy caused by input and output FIFO arrays in a traditional systolic array is avoided, calculation of a 512 * 512 matrix through a 256 * 256 array is taken as an example, and the structure has the advantages of being simple in structure, convenient to operate and low in cost. The calculation delay is reduced from 2560 clock cycles to 2176 clock cycles, and the array utilization rate is improved to 94%. The method has the advantages of being high in efficiency, low in cost, good in expansibility and suitable for neural network and matrix calculation acceleration.

The invention discloses a hardware-friendly Transform column balance pruning model compression and efficient deployment method. A model compression algorithm, a lightweight parameter storage format, an operation data buffer, a systolic array operation block, a vector operation unit, a nonlinear operator unit, a data flow controller and a DMA unit are included. A model compression algorithm and an efficient deployment architecture are explored according to Transform network software and hardware collaborative reasoning requirements: in a software level, the scale calculation complexity of model parameter quantities is reduced through a fine-grained column balance structured pruning strategy, and parameters are stored in a single-instruction multi-data-stream format and the parameter storage efficiency is optimized through mask code storage; according to the hardware level, an edge computing-oriented Transform special accelerator architecture is designed, so that the architecture can support column balance structured pruning characteristics and a lightweight parameter storage scheme in an original manner. According to the Transform model compression and efficient deployment method, the parameter sparsity after structured pruning is fully utilized, so that the parameter storage pressure of a hardware architecture is reduced, the complex balance of an arithmetic unit is ensured, the operation efficiency of an accelerator is improved, and load balance and efficient reasoning during software and hardware collaborative optimization are realized; the method is widely applicable to efficient deployment scenes of Transform models for edge calculation.

The invention provides a dynamic folding and unfolding method based on a Systolic array, belongs to the technical field of integrated circuit design and parallel computing, and aims at solving the problems of resource idleness, bandwidth bottleneck and other pain points of a traditional Systolic array matrix multiplication, and provides a 4 * 4 basic array dynamic folding / unfolding framework from four aspects of an array structure, an operation mode and the like. A storage / pulse / half-flow mode is intelligently switched, a three-dimensional resource collaborative scheduling mechanism is established, a QC write-back channel is reserved, resources are utilized to the maximum extent, and the core problem of a traditional scheme is solved.

The invention discloses a fluid pulsation accelerator architecture for reinforcement learning of a large language model, and belongs to the technical field of artificial intelligence hardware acceleration. The system aims at solving the problems that when a current universal processor executes an RLHF working load, an instruction driving normal form is not matched with data flow calculation, fixed parallel granularity cannot adapt to a dynamic load, and the efficiency is low due to the fact that a framework does not sense data statistical characteristics. The core of the system is a liquid systolic array calculation fabric capable of being dynamically reconstructed, execution is triggered through data flow, and kernel boundaries and instruction overhead are eliminated. According to the system, a global lookup table subsystem is fused, and nonlinear calculation is optimized through merging query and parallel lookup by using data distribution prior; and a dynamic scheduling unit is configured, and elastic parallelism is realized by adopting a resource allocation algorithm supporting work stealing. According to the method, the throughput and the energy efficiency of RLHF training and a large language model reasoning stage can be remarkably improved, and a new design normal form is provided for a next-generation AI special computing architecture.

A method and apparatus for deep learning are disclosed. The apparatus for deep learning includes a processor configured to support a plurality of different operation modes, the processor including a systolic array having a plurality of multiplier accumulator (MAC) units, and a control circuit configured to control a selection operation of the plurality of MAC units and a data movement between the plurality of MAC units, respectively, for each of the plurality of different operation modes.

The invention requests to protect a lightweight convolutional neural network bearing fault diagnosis method based on knowledge distillation. The method comprises the following steps of: firstly, converting a bearing fault signal from a time domain to a frequency domain complex signal through fast Fourier transform, and reserving an effective frequency spectrum of a rear half part so as to avoid excessive calculation overhead caused by a redundant signal; in order to ensure the accuracy of the teacher network in the training process, one-dimensional ResNet18 is used as a normalized knowledge distillation teacher network; an ultra-lightweight convolutional neural network composed of a convolutional layer, a maximum pooling layer and a full connection layer is designed, the total 1.95 K parameters are used for bearing signal fault diagnosis and normalized knowledge distillation training, and Gaussian white noise with different signal-to-noise ratios is used for verifying the effect of the designed model; then, carrying out board-level FPGA model deployment operation, taking a one-dimensional systolic array as a basic calculation unit, constructing a single-channel calculation core through a plurality of rows of systolic arrays, and further constructing a plurality of levels of parallel calculation blocks to accelerate reasoning of the model;

The invention discloses a general extensible digital beam forming chip design architecture and a method. The architecture comprises a chip control architecture which adopts a dual-bus master-slave control architecture of an AXI bus and an APB bus, a data source of a chip data stream architecture is external AD data or cascade beam forming dedicated chip sending end data, and the data is converted and sent in a stream form; according to the systolic array digital beam forming network architecture, a synthesis bandwidth, an input channel and an output channel are split to a plurality of beam forming chips, and the chips are connected in an interconnection topology form. The method is specifically a working method of the chip data stream architecture. According to the invention, the parameter reconfiguration of the chip is realized, the use expandability is realized, and the universality of the chip is ensured.

Provided are techniques for accelerating a Fully Homomorphic Encryption (FHE) operation with an on-chip systolic array. A computer processing chip comprises an Artificial Intelligence (AI) accelerator comprising a direct memory access and a systolic array, a Level 3 (L3) cache connected to the AI accelerator, and a core connected to the AI accelerator and the L3 cache. The AI accelerator receives AI accelerator code from the core, where the AI accelerator code comprises new instructions, where the systolic array executes the new instructions using first data by executing a BMUL instruction to perform multiplication and generate first results, a BSUB instruction to perform subtraction using the first results to generate second results, and a BADDSUB instruction to perform modular correction on the second results to generate final results, and where the direct memory access prefetches second data for the systolic array.

The present disclosure relates to the technical field of computer system, and provides a fine-grained quantization matrix multiplication device and method based on systolic array, which comprises: an input scheduling module, configured to receive input matrix data and corresponding quantization factor data, and to attach type identifiers to the input matrix data and the quantization factor data; a systolic array, connected by multiple processing units in a two-dimensional mesh structure, and each processing unit is configured to select to perform a multiply-accumulate operation or a dequantization operation according to the data received from an upstream and the type identifier of the data; wherein the input scheduling module is configured to schedule the data with the attached type identifiers to the systolic array by rows and by columns; and an output module, configured to receive and output the calculation results calculated by the systolic array. The problems of low efficiency of quantization factor insertion, fragmentation of dequantization and calculation pipeline, and large task switching overhead are solved, and the data multiplexing rate, calculation efficiency and overall energy efficiency ratio of the systolic array in the fine-grained quantization scenario are significantly improved.

This provides a systolic array that performs depth-direction convolution operations on multiple channels. [Solution] Depth convolution, which includes parallel processing of input and output, is performed by a plurality of multiplication-accumulation (MAC) units of the systolic array, each MAC unit having a weight register for storing weight values, an activation register for storing activation values, a multiplexer that transmits the activation value received from one of the activation registers and input lines, a multiplier 336 that multiplies the weight values ​​from the weight register and the activation value from the multiplexer, a memory that communicates with the plurality of MAC units, and a controller that transmits the weight values ​​of each MAC unit from the memory to the weight register of the MAC unit and transmits the activation value from the memory to the upstream MAC unit among the plurality of MAC units.

Systems and methods are provided for designing approximate, low-power deep learning accelerator chips that have little to no accuracy loss when executing a deep learning model. A set of approximate systolic arrays may be generated. The performance of each approximate systolic array in the set of approximate systolic arrays processing a deep neural network (DNN) may be determined. Each layer in the DNN may be mapped to an approximate systolic array in the set of approximate systolic arrays. A subset of the set of approximate systolic arrays may be selected for inclusion in the inference chip design based on the mapping and the performance of each approximate systolic array in the set of approximate systolic arrays.

The application relates to a systolic array structure for twice multiply-accumulate operation, relates to a deep learning technology, and comprises matrix-arranged systolic array units, each of which comprises two input registers, two weight registers, a partial sum register, two multipliers, an adder and an accumulator; the output end of the first input register is connected with the first input end of the first multiplier, the output end of the second input register is connected with the first input end of the second multiplier, the output end of the first weight register is connected with the second input end of the first multiplier, the output end of the second weight register is connected with the second input end of the second multiplier, the output end of the first multiplier and the output end of the second multiplier are respectively connected with the two input ends of the adder, the output end of the adder is connected with the first input end of the accumulator, and the output end of the partial sum register is connected with the second input end of the accumulator. The application reduces the operation time and reduces the hardware cost.

The invention relates to a dynamic heterogeneous graph neural network reasoning device and method based on an FPGA. The device comprises a hierarchical data loading module and a collaborative feature aggregation module. The hierarchical data loading module senses and captures changes of a vertex topological structure between continuous snapshots in the dynamic heterogeneous graph in real time; meanwhile, neighbors of different target vertexes under each timestamp are combined into neighbor set information, and corresponding feature vectors are sequentially loaded according to the occurrence frequency from high to low; the collaborative feature aggregation module receives neighbor set information, reads feature vectors corresponding to the neighbor set information, decouples an incremental calculation process into two parts of matrix vector multiplication and element calculation of corresponding positions, and completes an operation process of feature vector multiplication between a target vertex and a neighbor based on a local aggregation unit formed by a systolic array; and organizing the calculation units into a single instruction stream and multiple data streams based on the feature collection unit so as to complete normalization of calculation results of the same vertex.

The present disclosure provides a processing unit of a systolic array, the systolic array and a data processing method, and an electronic device. The processing unit includes a first set of data transmission paths for transmitting data in a first dimension and a second set of data transmission paths for transmitting data in a second dimension. The first set of data transmission paths are configured to transmit first data. The second set of data transmission paths is configured to transmit second data. The first set of data transmission paths includes a first data transmission path for transmitting data in a first direction and a second data transmission path for transmitting data in a second direction. The second set of data transmission paths includes a third data transmission path for transmitting data in a third direction and a fourth data transmission path for transmitting data in a fourth direction.

An AI-accelerating processor system that includes memory configured to store weights of a machine learning model. The processor system also includes a systolic array that has 1000x1000 or more computation tiles arranged in a grid for performing matrix multiplication involving the weights. In some embodiments, the systolic array may perform first multiplications of data in a first set of columns of the computation tiles. The systolic array may accumulate first multiplication results in the first set of columns. The systolic array may transmit accumulated results from the first set of columns to a second set of columns of the computation tiles. The systolic array may perform second multiplications of the accumulated results in a second set of columns. The systolic array may accumulate second multiplication results in the second set of columns.

The invention discloses an efficient and parallel autoregressive diffusion model hardware acceleration system based on an FPGA (Field Programmable Gate Array), and relates to the technical field of FPGA and machine learning. The system comprises a host end and an FPGA hardware end, the host end is responsible for preprocessing and post-processing of data, and the FPGA hardware end comprises a system control scheduling layer, a three-level storage structure layer and an autoregressive diffusion parallel architecture. According to the method, a collaborative pipeline design is adopted, deep parallelism of an autoregression Token generation process and a diffusion picture generation process is realized through a Token grouping adding mechanism, and a diffusion denoising process can be started without waiting for the completion of complete sequence generation. Besides, a universal attention processing hardware module is designed in the system, QKV projection and matrix multiplication are realized by using a systolic array, and calculation resources are optimized in cooperation with a pipelined Softmax calculation and delay normalization strategy. Through collaborative design of software and hardware, the problems of high reasoning delay and large resource overhead of the autoregression diffusion model in an edge computing scene are effectively solved, and the throughput and the energy efficiency ratio are remarkably improved while the generation quality is ensured.

The invention discloses an accelerator of a linear time sequence model, and belongs to the technical field of hardware accelerators, the accelerator comprises a global control unit used for overall planning of instructions, a DMA module, a global memory, a state updating array customized for state updating operation, and a systolic array matrix calculation array used for matrix operation, the vector unit is used for performing a general operator; the unified buffer is realized by an external global memory and an SRAM (Static Random Access Memory) on the memory level; and the DMA module is used for carrying block data between the external global memory and the buffer or between the buffers. The accelerator performs matrix operation, state updating operation and vector operation in a tile interlacing manner to form a highly overlapped software pipeline, so that hardware units can be executed in parallel to the maximum extent. The problems of on-chip redundant operation, low on-chip resource utilization rate and the like when the existing GPU and the special accelerator execute the linear time sequence model can be solved.

The application provides a non-privacy and privacy reasoning scheduling method based on a reconfigurable chip, which comprises the following steps: determining whether the privacy requirement of a neural network reasoning task is privacy reasoning; if yes, performing a first scheduling step; otherwise, performing a second scheduling step; in the first scheduling step, analyzing an arithmetic logic unit required by the privacy reasoning, combining a reconfigurable multiplier and an adder by a data layout converter to form the arithmetic logic unit, and reconfiguring an interconnection network of a processing unit array into a butterfly network or a SIMD data path to obtain a reconfigurable chip and perform a reasoning step; in the second scheduling step, setting a slice as a tensor mode by the data layout converter, independently and parallelly working all the slices, performing a multiply-accumulate operation by each slice, transmitting data between processing units PEs through horizontal and vertical links to form a pulsating data stream, forming a pulsating array by the processing unit array, obtaining the reconfigurable chip, and performing the reasoning step; and in the reasoning step, performing the neural network reasoning task by the reconfigurable chip to obtain a reasoning result.

The invention relates to the technical field of computer systems, and provides a fine-grained quantization matrix multiplication device and method based on systolic arrays, and the device comprises a first systolic array which is used for receiving element streams of a first quantization factor matrix and a second quantization factor matrix, and carrying out the fusion calculation of quantization factors according to the element streams of the quantization factors, generating a quantization factor product; the second systolic array is used for receiving element streams of the first data matrix and the second data matrix, and performing multiplication and accumulation calculation according to the element streams of the matrixes; each first processing unit in the first systolic array is configured to send a quantization factor product obtained by calculation to a corresponding anchor processing unit in the second systolic array according to a mapping proportion; and the anchor processing unit is configured to receive the quantization factor product and serve as a propagation source point, the quantization factor product is propagated in the second systolic array in the row direction and the column direction within the quantization block size range corresponding to the quantization factor product, and the calculation efficiency is improved.

The invention relates to the technical field of computer systems, and provides a fine-grained quantization matrix multiplication device and method based on a systolic array, and the device comprises an input scheduling module which is used for receiving input matrix data and corresponding quantization factor data, and adding type identifiers to the input matrix data and the quantization factor data; the systolic array is formed by connecting a plurality of processing units in a two-dimensional net structure, and each processing unit is used for selectively executing multiply-accumulate operation or inverse quantization operation according to the data received at the upstream and the type identifier of the data; wherein the input scheduling module is used for scheduling data added with a type identifier to a systolic array according to rows and columns; and the output module is used for receiving and outputting a calculation result calculated by the systolic array. The problems that quantization factor insertion is low in efficiency, inverse quantization is separated from calculation flow and task switching overhead is large are solved, and the data reuse rate, the calculation efficiency and the overall energy efficiency ratio of the systolic array in a fine-grained quantization scene are remarkably improved.

The present disclosure is directed to an apparatus that includes a systolic array having an initial systolic stage that is clocked at a first edge of a clock signal. The apparatus further includes control logic configured to clock a first subset of a plurality of additional systolic stages of the systolic array at the first edge of the clock signal. The control logic is further configured to clock a second subset of the plurality of systolic stages at a second edge of the clock signal.

The invention provides a non-privacy and privacy reasoning scheduling method based on a reconfigurable chip, which comprises the following steps of: judging whether a privacy demand of a neural network reasoning task is privacy reasoning, if so, executing a first scheduling step, otherwise, executing a second scheduling step; a first scheduling step: analyzing an arithmetic logic unit required by privacy reasoning, combining and reconstructing a multiplier and an adder by a data layout converter to form the arithmetic logic unit, reconstructing an interconnection network of a processing unit array into a butterfly network or an SIMD (Single Instruction Multiple Data) data path to obtain a reconstructed chip, and executing a reasoning step; in the second scheduling step, the data layout converter sets the slices into a tensor mode, all the slices independently work in parallel, each slice executes multiplication and addition accumulation operation, data are transmitted between processing units PE through horizontal and vertical links, a pulsation data stream is formed, a pulsation array is formed by a processing unit array, a reconstructed chip is obtained, and the reasoning step is executed; and a reasoning step: executing the neural network reasoning task by the reconstruction chip to obtain a reasoning result.