148 results about "Vector operations" patented technology

The embodiment of the invention provides a matrix and vector operation device based on an RISC-V extension instruction, the device comprises a coprocessor and a main processor which are connected with each other, an RISC-V instruction set and a coprocessor extension instruction set are configured in the main processor, the RISC-V instruction set comprises a vector extension instruction set related to a vector operation task, and the coprocessor extension instruction set comprises a coprocessor extension instruction set related to the vector operation task. The coprocessor extension instruction set comprises a matrix processing instruction set related to a matrix operation task, and the main processor is configured to send the matrix processing instruction set related to the matrix operation task or a vector extension instruction set related to a vector operation task to the coprocessor when a calculation task is executed, the computing task is executed in parallel with the coprocessor; the coprocessor is configured to execute a matrix operation task unloaded by the main processor according to the matrix extension instruction set to obtain an execution result of the matrix operation task; or executing the vector operation task unloaded by the main processor according to the vector extension instruction set to obtain an execution result of the vector operation task.

The invention discloses an optimization method for a multi-head attention mechanism operator oriented to a domestic heterogeneous processor, which improves the efficiency of the multi-head attention mechanism operator from four aspects, and comprises the following steps of: firstly, combining the general computing power of a CPU (Central Processing Unit) and the advantages of a DSP (Digital Signal Processor) in the aspect of parallel computing; a high-performance multi-head attention mechanism operator operation method on a CPU + DSP heterogeneous processor is constructed; secondly, a previous softmax optimization algorithm is used for fusing the two batch processing matrix multiplication and a softmax operator, so that the memory access overhead of DSP equipment is reduced; then, a vector operation instruction of a DSP calculation core is used in the calculation process of a multi-head attention operator, and SIMD calculation characteristics of DSP equipment are fully utilized; and finally, during data transmission and calculation, a double-flow water mechanism is adopted, the data transmission time is covered, and the multi-head attention mechanism operator calculation efficiency is improved.

The invention provides a universal 3D printing global continuous path planning method and system, and relates to the technical field of 3D printing data processing or control, the method comprises the following steps: inputting an STL model and slicing to obtain each layer of section contour; calculating offset contours of the inner and outer contours by adopting vector operation; interference is eliminated when self-intersection or intersection occurs in the offset contour; constructing a path point set; establishing a mathematical model for path planning; solving the mathematical model by adopting a pointer network based on reinforcement learning to obtain a continuous filling path traversing all the path point sets; curvature constraint smoothing is carried out on the path through an n-order Bezier curve; and generating a 3D printing global continuous path by adopting a greedy search algorithm. The method is suitable for a complex porous structure, path intersection and interruption are reduced, the path length and the bending angle are minimized, and therefore the printing precision and the mechanical property are improved.

The invention provides a high-speed NTT method and system based on GPU matrix-thread collaborative optimization. The method comprises the steps that an input twiddle factor is converted into a matrix form; transmitting the updated twiddle factor matrix and the input array to a graphics processor; according to the input array and the size of the twiddle factor matrix, the number of blocks in the grid and the number of threads included in each block are set, so that each block is responsible for the operation of multiplying one row by one column, and each thread in the block completes the operation of multiplying one pair of data in the corresponding row-by-column process to obtain a group of intermediate results; and obtaining output data corresponding to the block according to an intermediate result in the block, and sending the output data to a corresponding position of an output matrix to serve as a multiplication result. According to the method, NTT calculation is reconstructed into parallel matrix-vector operation, the maximum utilization of GPU calculation resources is realized through the combination of an efficient thread allocation strategy and an optimized memory access mode, and the calculation efficiency is remarkably improved.

Processors, systems and methods are provided for thread level parallel processing. A processor may include a sequencer and a plurality of columns of vector processing units coupled to the sequencer. The sequencer may include a scalar instruction decoder, a vector instruction decoder, and a plurality of scalar processors configured to concurrently execute one or more scalar instructions decoded by the scalar instruction decoder to generate one or more vectors of parameters. The vector instruction decoder may be configured to decode one or more vector instructions to generate a set of configurations with the one or more vectors of parameters embedded as one or more vectors of immediate values and send the configurations to a target column. Each column of vector processing units may be configured to repeatedly execute the vector operations in the configurations using one or more respective elements of one or more vectors of immediate values per repetition.

The embodiment of the invention belongs to the technical field of computing, and particularly relates to a method for executing FFT, a processor and computing equipment. The method includes a processor decomposing a fast Fourier transform (FFT) calculation into a plurality of calculations in response to an execution request of the FFT calculation of an application program. The processor sequentially executes a plurality of calculation stages, and when the target calculation stage is executed, twiddle factor calculation is executed based on the vector operation unit, and DFT calculation is executed based on the matrix operation unit. And after the execution of the plurality of calculation stages is completed, based on the execution result of the last calculation stage, determining the execution result of the FFT calculation, and returning the execution result to the application program. According to the invention, the processor can achieve the calculation of the FFT based on the vector operation unit and the matrix operation unit, and can improve the FFT calculation execution efficiency of the processor.

The invention provides a method for executing convolution operation in a vector operation unit, computing equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: loading a first matrix of convolution operation to a thread local register; loading the second matrix of the convolution operation to a thread bundle scalar register of the vector operation unit through the thread local register; multiplying and adding two elements of one sub-matrix of the first matrix read from the thread local register and two corresponding elements of the second matrix read from the thread bundle scalar register through a multiply-add instruction to obtain a result of the multiply-add instruction, wherein the multiply-add instruction is used for multiplying and adding two elements of one sub-matrix of the first matrix read from the thread local register and two corresponding elements of the second matrix read from the thread bundle scalar register; and adding a result of the multiply-add instruction and a product of a third element of the sub-matrix read from the thread local register and a third corresponding element of the second matrix read from the thread bundle scalar register to obtain one element of a result matrix of the convolution operation.

The invention discloses an automatic code vectorization optimization method and device and a medium, belongs to the technical field of high-performance computing, and aims at solving the problems that existing automatic code vectorization is inconvenient, the manual operation proportion is large, vectorization of irregular codes or codes with a load dependency relationship is difficult to support, and the vectorization efficiency is high. And improvement of calculation performance and reduction of energy consumption are not facilitated. The method comprises the steps of obtaining a scalar instruction diagram of a calculation task according to a calculation kernel of parallel calculation; performing protocol conversion processing on the scalar instruction diagram to obtain a protocol instruction diagram; according to the high-level organization structure of the instruction, grouping processing related to mapping vectors is carried out on the protocol instruction graph, and a grouping graph is obtained; based on the limited vector length of hardware, performing splitting matching processing on instruction vectors in the grouping graph to obtain a vector matching grouping graph; and executing configuration on each group of elements in the vector matching grouping graph to obtain a vector operation graph.

Embodiments of the invention provide a data processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a unit vector operation width supported by hardware; obtaining a source address to be aligned and the number of object data to be operated; in response to the fact that the width of the object data is larger than or equal to a first threshold value, the remainder obtained after the to-be-aligned source address is aligned with the unit vector operation width is calculated so as to obtain the data width of which the addresses are not aligned in the object data, and the width of the object data is equal to the product of the unit data width of the object data and the number of the object data; the width of the address misalignment data is equal to the difference value between the unit vector operation width and the remainder; determining the number of the address misalignment data according to the unit data width of the object data and the width of the address misalignment data, and determining first mask information based on the number of the address misalignment data and the unit vector operation width; and according to the first mask information, loading and processing the data of which the addresses are not aligned, and the method improves the data processing performance.

A data processing apparatus, a processor, a board card, and a data processing method. The data processing apparatus may be comprised by a combined processing apparatus (20), the combined processing apparatus (20) comprising a computing apparatus (201), an interface apparatus (202), a processing apparatus (203) and a storage apparatus (204). The computing apparatus (201) is configured to execute an operation specified by a user, so as to execute deep learning or machine learning calculation. The computing apparatus (201) may interact with the processing apparatus (203) by means of the interface apparatus (202), so as to jointly complete the operation specified by the user. The interface apparatus (202) is used to transmit data and a control instruction between the computing apparatus (201) and the processing apparatus (203). The storage apparatus (204) is used to store data of the computing apparatus (201) and the processing apparatus (203). The data processing apparatus provides a vector computation scheme integrating different types of fine-grained quantization, so that processing can be simplified, and the advantages of a low bit width operation and small memory occupation of a fine-grained quantization format are fully utilized.

Techniques are described herein that are capable of compressing and transforming vector operations in an AI model. First output multi-bit elements (MBEs) are generated by combining input single-bit components (SBCs) representing an input token in an AI prompt and first SBCs representing a first layer of the AI model using an exclusive-or operation. The first output MBEs are transformed into first output single-bit elements (SBEs) using a random probability distribution. Second output MBEs are generated by combining intermediate SBEs corresponding to intermediate MBEs derived from the first output SBEs and second SBCs representing a second layer of the AI model using the exclusive-or operation. A response to the AI prompt is generated to include an output token corresponding to a combination of a norm of the intermediate MBEs, a norm of second multi-bit components from which the second SBCs are derived, and a representation of the second output MBEs.

Methods, systems, and computer program products are provided that implement vector-related requests using query operators. For example, a system includes a database, a parser, a converter, an optimizer, and an execution engine. The database is associated with query operators that are non-vector-specific. The database stores a table with vector embeddings. The parser is configured to parse a request indicating a vector operation associated with the vector embeddings. The converter is configured to convert the vector operation into a logical operator tree comprising a representation of the request as a logical flow of the query operators, enabling the vector operation without vector-specific executable code or operators. The optimizer is configured to convert the logical operator tree into an executable plan. The execution engine is configured to execute the executable plan against the table with vector embeddings.

According to an embodiment of the present disclosure, a processor and a method, device and storage medium for data processing are provided. The processor includes an instruction decoder configured to decode a target instruction for a vector operation. The target instruction involves a target opcode, a source operand and a target operand. The target opcode indicates the vector operation specified by the target instruction. The source operand specifies a source storage location in the memory for reading data to be processed. The target operand specifies a target storage location in the memory for writing a processing result. The processor also includes an arithmetic logic unit coupled to the instruction decoder and the memory. The arithmetic logic unit is configured to: read data to be processed from a source storage location in the memory; perform an arithmetic logic operation associated with the vector operation specified by the target instruction on the data to be processed; and write the processing result to a target storage location in the memory. In this way, the efficiency of vector calculations can be improved.

The invention provides vector computing equipment, which comprises an instruction scheduling unit, an instruction decoding unit connected with the instruction scheduling unit, and a plurality of vector kernel execution modules connected with the instruction decoding unit, wherein the plurality of vector core execution modules are connected in series, only one first vector core execution module is connected with the instruction decoding unit, and the instruction scheduling unit is configured to schedule a single-instruction multi-thread instruction and send the SIMT instruction to the instruction decoding unit. And the instruction decoding unit is configured to decode the SIMT instruction to determine an operand address of an operand of the SIMT instruction, and instruct the plurality of vector core execution modules to execute vector operations of the SIMT instruction on the operand in a plurality of clock cycles, respectively.

A processor and a method, device and storage medium for data processing are provided. The processor includes an instruction decoder configured to decode a target instruction for a vector operation. The target instruction involves a target opcode, a source operand, and a target operand. The target opcode indicates a vector operation specified by the target instruction. The source operand specifies a source storage location in the memory for reading to-be-processed data. The target operand specifies a target storage location in the memory for writing a processed result. The processor further includes an arithmetic logic unit configured to: read the to-be-processed data from the source storage location of the memory; perform, on the to-be-processed data, an arithmetic logic operation associated with the vector operation specified by the target instruction; and write the processed result to the target storage location of the memory.

The invention relates to the field of heterogeneous parallel computing, in particular to a matrix memory reading method applied to mercuric chloride NPU, which comprises the following steps of: dividing a matrix into a regular matrix and an irregular matrix according to feature information of the matrix; arranging memory data of the regular left matrix and the regular right matrix into a format required by the L1 cache for matrix multiplication; according to the equivalent bandwidth quantization preprocessing income, judging whether to use a vector operation unit to preprocess memory data arrangement on the irregular matrix; when the equivalent bandwidth after preprocessing is larger than the original bandwidth, preprocessing of memory data arrangement is carried out on the irregular matrix, and otherwise, preprocessing of memory data arrangement is not carried out on the irregular matrix; and reading the preprocessed matrix to a cache by using an efficient memory reading interface, and carrying out matrix multiplication. According to the method, the problem that the memory bandwidth cannot be efficiently utilized in a memory access bottleneck scene can be optimized, high-bandwidth memory reading can be realized during matrix multiplication, and the overall calculation efficiency of a system is improved.

The present disclosure provides a data processing method and processor instructions that utilize scatter operations to efficiently merge vector and matrix indices (as compared to standard matrix and vector operations) and merge other arithmetic results, quantity lists, etc.

The invention relates to the technical field of RISC-V instruction set processors, in particular to an RISC-V-based AI processor design method for small convolution kernel acceleration, which comprises the following steps: determining the acceleration demand of a 3 * 3 convolution kernel in a convolutional neural network (CNN), selecting an open architecture RV32I of RISC-V as a basis, defining a core architecture of an AI processor, comprising a calculation unit, a register file and a control unit, the processing of the small convolution kernel is optimized; designing and realizing a vector arithmetic logic unit (VALU) and a convolution operation unit (CUU); the VALU has parallel computing capability and can execute addition, subtraction and multiplication vector operations to support instruction set extension; a convolution arithmetic unit CUU processes 3 * 3 convolution operations, including dedicated hardware logic to perform element-by-element multiplication and accumulation operations, thereby accelerating the convolution calculation process.

The invention provides an instruction processing method and device, a chip product, computer equipment and a storage medium. The method comprises the following steps: in response to a received to-be-processed vector instruction, splitting the vector instruction into a plurality of microoperations respectively corresponding to a plurality of destination registers; the micro-operation is used for acquiring data access information required by the destination register corresponding to the micro-operation; executing each item of microoperation in parallel; triggering an execution unit to execute a calculation operation under the condition of determining that all the micro-operations are executed; wherein the execution unit, in response to the calculation operation, performs a vector operation on the plurality of destination registers based on the data access information acquired by all the micro-operations. The number of micro-operations can be greatly reduced, and the problem of pipeline blockage caused by the fact that a transmitting queue is filled is avoided.

Embodiments disclosed in this application pertain to the field of computer technologies, and in particular, to a method for performing FFT, a processor, and a computing device. The method includes: A processor responds to an execution request of fast Fourier transform FFT calculation of an application, and decomposes the FFT calculation into a plurality of calculation stages. The processor sequentially executes the plurality of calculation stages, where when a target calculation stage is executed, a vector operation circuit performs rotation factor calculation, and a matrix operation circuit performs DFT calculation. After the execution of the plurality of calculation stages is completed, the processor determines an execution result of the FFT calculation based on an execution result of a last calculation stage, and returns the execution result to the application. vector operation circuit matrix operation circuit

The application relates to a synthesis method of a rotating phasor, which is suitable for the technical field of electrical rotating phasor conversion and is particularly directed to the decomposition and synthesis of an electrical rotating phasor. The method adjusts the initial phase of two amplitude-constant sub-rotating phasors, and then synthesizes a target rotating phasor with adjustable amplitude and initial phase, and has the characteristics of easy generation of the sub-rotating phasor, good flexibility and high reliability. The application mainly utilizes the parallelogram rule of vector operation, reduces the generation difficulty of the sub-rotating phasor by reducing the generation problem of the target rotating phasor with adjustable amplitude and initial phase to the synthesis problem of the two sub-rotating phasors with constant amplitude and adjustable phase, and solves the problem of synthesizing a complex rotating phasor from a simple rotating phasor. The technical scheme provided by the application forms a simple sub-rotating phasor through electromagnetic conversion, and provides a new idea for the synthesis of a complex rotating phasor.

A neural processing unit for performing inference operations of a large-scale language model based on an artificial neural network is disclosed. The neural processing unit according to the present disclosure includes a processing element core configured to perform an attention mechanism-based operation based on input data in vector format to output an operation result, a special function unit comprising a plurality of arithmetic circuits including at least one vector-dedicated arithmetic circuit that exclusively performs vector operations and at least one mixed arithmetic circuit capable of performing both vector and scalar operations, and configured to perform a special function operation on the operation result, and a controller configured to, upon receiving an RMS normalization operation execution command, activate at least one of the plurality of arithmetic circuits to control the special function unit to perform an operation of converting at least one of the operation result or the input data into a normalized vector whose magnitude is adjusted based on a root mean square (RMS), wherein the operation result may include an attention score for the input data.

Systems, apparatuses, and methods are disclosed for improved matrix–vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values in floating point formats. The CIM macro has a functional block configured to align mantissa bits of primitive products between the activation values and the weight values by shifting the mantissa bits and an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

The invention discloses a vector calculation-based water meter water quantity and water rate calculation method, and aims to solve the technical problems of I / O bottleneck, high calculation delay and poor expansibility in existing large-scale water meter charging. The method comprises the following steps: constructing a water meter data column vector, calculating an original difference vector based on a current-period reading vector and a previous-period reading vector, and performing Boolean mask processing to obtain a meter reading water volume vector; actual water consumption accounting, total and score table allocation and allocated water amount calculation are achieved through vector operation; a billing water volume vector is obtained in combination with a preferential strategy, and an actual water fee is finally obtained through mixed water splitting, stepped billing matrix operation and water fee preferential adjustment. According to the method, vectorization and matrix operation are adopted to replace traditional line-by-line calculation, so that the operation efficiency can be improved through parallel calculation, the vector algorithm is designed to adapt to complex water affair billing logic, the processing performance is effectively improved, efficient and large-scale billing is achieved, the intelligent water affair development requirement is met, and practicability and popularization value are achieved.

The invention discloses an AI calculation acceleration method and system based on cache mode dynamic switching, and the method comprises the steps: determining a to-be-switched target cache block in a last-stage cache after receiving an AI calculation acceleration request; based on the physical address, the data is quickly refreshed to guarantee the consistency; marking the AI calculation mode as an AI calculation mode and shielding the AI calculation mode in an allocation strategy; executing in-memory calculation by utilizing a storage array and a vector operation unit which are integrated; and after the task is completed, clearing the mark and recovering the available state. The system comprises a cache switching controller, a matrix multiplication scheduler, a data preprocessing module, a cache isolation module and an instruction interface and software scheduling unit. By multiplexing the last level of cache and dynamically switching the working mode of the last level of cache, the AI calculation is efficiently completed while the system stability is ensured, and the computing power per unit area of a chip is remarkably improved.

A The present invention relates to a technique for controlling input / output operation of a vector processor, which is designed to optimize vector operation in a mixed precision environment, and to a technique for maximizing data processing performance while minimizing waste of operation resources by improving the data conversion process between the memory and the vector processor.

The present invention provides a vector operation device, method, product and storage medium, which relate to the field of vector operation technology. The device includes an input operation component, a data operation component and an output component; the input operation component includes an input interleaving unit and a complex multiplication and addition unit, the input interleaving unit is used to interleave the input data to obtain first interleaved data; the complex multiplication and addition unit is used to perform multiplication and addition operations on the first interleaved data to obtain an initial operation result; the data operation component includes an operation interleaving unit and a complex addition unit, the operation interleaving unit is used to interleave the initial operation result to obtain second interleaved data; the complex addition unit is used to perform addition operations on the second interleaved data to obtain an intermediate operation result; the output component includes an output interleaving unit, and the output interleaving unit is used to interleave the intermediate operation result to obtain a vector operation result. The present invention can support special data arrangement requirements and complete operations, thereby improving operation efficiency.

The present invention discloses a cloud computing task scheduling method based on privacy protection, which relates to the field of task processing technology and includes the following steps: S1, obtaining tasks to be scheduled and performing privacy processing on the tasks to be scheduled; S2, after the privacy processing, generating an influence matrix between the tasks to be scheduled and each container based on a number of containers of the cloud platform, and determining the scheduling unit of the tasks to be scheduled according to the influence matrix; S3, completing the scheduling processing according to the scheduling unit of the tasks to be scheduled. The present invention converts the security attributes of the container nodes into vector form through a hash vectorizer, determines the influence matrix in combination with the task priority; then captures the synergistic effect of risk and priority through vector operations, avoiding the one-sidedness of single indicator decision-making; and finally enables the tasks to be scheduled to run in units of containers, reducing interference between tasks. At the same time, the present invention enhances the robustness against unknown attacks and overcomes the defense shortcomings of traditional scheduling methods.

Systems, apparatuses, and methods are disclosed for improved matrix-vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values in floating point formats. The CIM macro has a functional block configured to align mantissa bits of primitive products between the activation values and the weight values by shifting the mantissa bits and an adder tree configured to output an accumulation value of the primitive products in an integer format by adding the shifted mantissa bits.

A processing-in-memory (PIM) device implements block quantization techniques for matrix-vector operations. The PIM device performs matrix-vector operations between portions of a weight matrix and an input vector, and copies results to a register. A read operation retrieves the copied results while additional matrix-vector operations are performed in parallel. The device may apply scaling factors to the results using multipliers within the PIM device. In some implementations, the weight matrix includes data columns and scaling factor columns interspersed at regular intervals. The scaling factors may be applied to accumulated results using parallel multiplication operations. Disclosed techniques enable efficient implementation of block quantization for applications such as Large Language Models while managing computational resources within the PIM architecture.