Dot product operation method and apparatus for a processor

Abstract

The application provides a dot product operation method and device of a processor. The dot product operation method of the processor comprises the following steps: according to an accumulation order, a first vector is divided into a plurality of first data blocks, and a second vector is divided into a plurality of second data blocks; the first data blocks and the second data blocks are loaded into a first register of the processor; elements in the first data blocks and elements in the second data blocks are rearranged in the first register, and the elements in each rearranged data block are arranged continuously according to the accumulation order; parallel multiplication and addition operations are performed on the elements in a plurality of corresponding positions of the rearranged first data blocks and the rearranged second data blocks; and a dot product operation result of the first vector and the second vector is obtained based on a result of the parallel multiplication and addition operations. According to the application, the speed of the dot product operation of the processor is improved on the basis of maintaining the calculation precision.

Description

TECHNICAL FIELD

[0001] The present application relates to floating point calculation technology, and in particular to a dot product operation method and device of a processor. BACKGROUND

[0002] Since a floating point number can represent a wide range of data range, a large number of dot product operations of floating points are performed in the process of data processing of a processor.

[0003] In the related art, the dot product operation is completed by element-by-element multiplication and accumulation, and the calculation speed is slow. SUMMARY

[0004] Embodiments of the present application provide a dot product operation method and device of a processor, a chip, an electronic device, a computer readable storage medium and a computer program product, which can complete the dot product operation by parallel multiplication, thereby improving the speed of the dot product operation.

[0005] The technical solutions of the embodiments of the present application are implemented as follows:

[0006] The embodiments of the present application provide a dot product operation method of a processor, which comprises:

[0007] The first vector is divided into a plurality of first data blocks according to an accumulation order, and the second vector is divided into a plurality of second data blocks, and the first data blocks and the second data blocks are loaded into a first register of the processor, wherein the accumulation order comprises an order and a number of data layers in which the product of the elements at the corresponding positions in the first vector and the second vector is accumulated in the case of single-thread dot product calculation of the first vector and the second vector, and the first data blocks and the second data blocks are arranged one by one in the first register;

[0008] The elements of the first data blocks are rearranged in the first register, and the elements of the second data blocks are rearranged, to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks, wherein the elements in each rearranged first data block are arranged continuously according to the accumulation order, and the elements in each rearranged second data block are arranged continuously according to the accumulation order;

[0009] Parallel multiplication and addition operations are performed on the elements at a plurality of corresponding positions in the rearranged first data blocks and the rearranged second data blocks by a plurality of threads of the processor;

[0010] The dot product operation result of the first vector and the second vector is obtained based on the result of the parallel multiplication and addition operations.

[0011] The embodiments of the present application provide a dot product operation method and device, which comprise:

[0012] a split loading module configured to split the first vector into a plurality of first data blocks and split the second vector into a plurality of second data blocks according to an accumulation order, and load the first data blocks and the second data blocks into a first register of a processor, wherein the accumulation order comprises an order and a number of data ranks in which products of elements at corresponding positions in the first vector and the second vector are accumulated in a case that a single-threaded dot product calculation is performed on the first vector and the second vector, and the first data blocks and the second data blocks are arranged in the first register in a one-to-one correspondence;

[0013] an element rearranging module configured to rearrange elements in the first data blocks and rearrange elements in the second data blocks in the first register, to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks, wherein elements in each of the rearranged first data blocks are arranged in a continuous manner according to the accumulation order, and elements in each of the rearranged second data blocks are arranged in a continuous manner according to the accumulation order;

[0014] a parallel calculation module configured to perform parallel multiplication and addition operations on elements at a plurality of corresponding positions in the rearranged first data blocks and the rearranged second data blocks by a plurality of threads of the processor;

[0015] a result processing module configured to obtain a dot product operation result of the first vector and the second vector based on a result of the parallel multiplication and addition operations.

[0016] In the above scheme, the element rearranging module is configured to rearrange elements in the first data blocks and rearrange elements in the second data blocks in the first register, to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks, by the following manner:

[0017] rearranging elements in the first data blocks by a single instruction multiple data hardware instruction set to replace one or more rows of elements in each of the first data blocks, to obtain a plurality of rearranged first data blocks, wherein the replacement comprises replacement within the data blocks or replacement between two data blocks;

[0018] rearranging elements in the second data blocks by a single instruction multiple data hardware instruction set to replace one or more rows of elements in each of the second data blocks, to obtain a plurality of rearranged second data blocks.

[0019] In the above scheme, the result processing module is further configured to obtain the dot product operation result of the first vector and the second vector based on the result of the parallel multiplication and addition operations, by the following manner:

[0020] Adjust the arrangement order of the elements in the result of the parallel multiplication and addition operation, so that the arrangement order of the elements in the result of the parallel multiplication and addition operation conforms to the reduction summation order, to obtain a third data block; perform parallel accumulation calculation on the elements in the third data block to obtain the dot product operation result of the first vector and the second vector.

[0021] In the above scheme, the number of elements of the first vector and the second vector is an integer multiple of the number of elements that the first register can store;

[0022] The split loading module is further configured to enable the number of elements of the first vector and the second vector to be an integer multiple of the number of elements that the first register can store by performing the following processing:

[0023] write the first vector to a first buffer area of a memory, and fill in null value elements in the first buffer area to enable the first buffer area to store an integer multiple of the number of elements that the first register can store;

[0024] write the second vector to a second buffer area of the memory, and fill in null value elements in the second buffer area to enable the second buffer area to store an integer multiple of the number of elements that the first register can store.

[0025] In the above scheme, the split loading module is further configured to split the first vector into a plurality of first data blocks and split the second vector into a plurality of second data blocks according to the accumulation order by the following manner:

[0026] split the first vector into a plurality of first data blocks with a size of a processor processing granularity according to the accumulation order, and split the second vector into a plurality of second data blocks with a size of a processor processing granularity according to the accumulation order.

[0027] In the above scheme, the split loading module is further configured to load the first data blocks and the second data blocks into the first registers of the processor by the following manner:

[0028] load a plurality of the first data blocks and the plurality of second data blocks into a plurality of the first registers in parallel by a plurality of threads of the processor.

[0029] In the above scheme, the split loading module is further configured to, in response to detecting that the first data blocks and the second data blocks that have not been loaded into the first registers, load the first data blocks and the second data blocks that have not been loaded into the first registers into the first registers, calculate the dot product calculation result of the loaded data blocks, and repeatedly detect the first data blocks and the second data blocks that have not been loaded into the first registers until the first data blocks and the second data blocks are all loaded into the first registers.

[0030] The parallel computing module is further configured to add the dot product calculation result and the unloaded dot product calculation result to obtain a new dot product calculation result.

[0031] In the foregoing solution, the result of the parallel multiplication and addition operation is stored in the second register.

[0032] The loading, in response to detecting the first data block and the second data block not loaded into the first register, of the first data block and the second data block not loaded into the first register into the first register includes:

[0033] The loading, in response to detecting the first data block and the second data block not loaded into the first register and completing the parallel multiplication and addition operation, of the first data block and the second data block not loaded into the first register into the first register.

[0034] In the foregoing solution, the split loading module is further configured to, before splitting the first vector into a plurality of first data blocks and splitting the second vector into a plurality of second data blocks according to the accumulation order, perform:

[0035] The first vector and the second vector are compressed into a compressed first vector and a compressed second vector, where the compressed first vector has a lower precision than the first vector, and the compressed second vector has a lower precision than the second vector.

[0036] Embodiments of the present application provide an electronic device, which comprises:

[0037] A memory configured to store computer executable instructions or computer programs.

[0038] A processor configured to implement the dot product operation method of the processor provided in the embodiments of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0039] Embodiments of the present application provide a processing chip configured to execute the dot product operation method of the processor described above.

[0040] Embodiments of the present application provide a computer readable storage medium storing computer programs or computer executable instructions, configured to implement the dot product operation method of the processor provided in the embodiments of the present application when executed by a processor.

[0041] Embodiments of the present application provide a computer program product comprising computer programs or computer executable instructions, configured to implement the dot product operation method of the processor provided in the embodiments of the present application when executed by a processor.

[0042] The embodiments of the present application have the following beneficial effects:

[0043] The embodiments of the present application rearrange the elements in the data block to be accumulated obtained by cutting the vector in the register, so that the elements in each rearranged data block are arranged according to the accumulation order. When the multiplication and addition operations required by the dot product algorithm are performed, the accumulation order of the elements in the data block is preserved by means of multi-thread parallel calculation, so that the dot product operation method of the processor can be executed in parallel on the basis of maintaining the calculation accuracy, and the speed of the dot product operation of the processor is improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a structural schematic diagram of an electronic device provided by the embodiments of the present application;

[0045] Figure 2 is a structural schematic diagram of a processor architecture provided by the embodiments of the present application;

[0046] Figure 3 is a flow schematic diagram of a dot product operation method of a processor provided by the embodiments of the present application;

[0047] Figure 4 is a schematic diagram provided by the embodiments of the present application that cutting a vector into data blocks will cause the accumulation order to be destroyed;

[0048] Figure 5 is a schematic diagram provided by the embodiments of the present application before the first data block is rearranged;

[0049] Figure 6 is a schematic diagram provided by the embodiments of the present application after the first data block is rearranged;

[0050] Figure 7 is a flow schematic diagram of the embodiments of the present application in an actual application scenario;

[0051] Figure 8 is a principle schematic diagram provided by the embodiments of the present application for performing parallel multiplication and addition operations on the half-precision floating-point vector elements in the register;

[0052] Figure 9 is a principle schematic diagram provided by the embodiments of the present application for filling empty values into the vector when the number of vector elements is less than 64 to perform data block cutting.

[0053] It should be noted that the "first" and "second" above are only used to distinguish different schemes, and do not represent the degree of superiority or priority in the implementation process. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the present application clearer, the following further describes the present application in conjunction with the accompanying drawings, the described embodiments should not be regarded as limiting the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0055] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0056] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0057] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0058] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0059] Before the embodiments of the present application are further described in detail, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0060] 1) In response to: used to indicate the conditions or states on which the operations performed depend, when the dependent conditions or states are met, one or more operations performed can be real-time or have a set delay; in the absence of special instructions, there is no restriction on the execution order of multiple operations performed.

[0061] 2) Processor: The core component of a computer system, also known as the Central Processing Unit (CPU). It is primarily responsible for executing instructions in computer programs and processing and performing calculations on data.

[0062] 3) Registers: Registers are high-speed storage units inside the processor, used for temporary storage of data and instructions. When the processor loads a matrix or data block through a register, each element of the matrix or data block is placed in a specific location within the register. Common registers include general-purpose registers (used to store operands and intermediate results), the instruction register (stores the currently executing instruction), and the program counter (stores the address of the next instruction to be executed). Registers are much faster to read and write than main memory, so the processor tries to store as much data as possible in registers when executing instructions to improve processing speed.

[0063] 4) The Floating Point Unit (FPU) is the structure that performs floating-point operations. The FPU is primarily responsible for performing mathematical operations involving floating-point numbers, such as basic arithmetic operations like addition, subtraction, multiplication, and division. Compared to the integer arithmetic unit, the FPU directly supports numerical calculations including decimal points through hardware circuitry, significantly improving the accuracy and speed in scenarios such as scientific computing and graphics rendering. Furthermore, the FPU can also handle transcendental function operations (such as trigonometric functions and logarithmic functions), further expanding its application scope.

[0064] 5) Multiply Accumulate (MAC) is a special operation in digital signal processors or some microprocessors. The hardware circuit unit that implements this operation is called a "multiplier accumulator". This operation adds the product of the multiplication to the value of accumulator A and then stores it in the accumulator.

[0065] 6) Instruction Set: An instruction set is the collection of all instructions a processor can execute. Different processor architectures have different instruction sets. For example, x86 processors and ARM processors have different instruction sets. The richness and efficiency of the instruction set also affect processor performance. For example, Single Instruction, Multiple Data (SIMD) instructions are a type of processor instruction that processes multiple data sets simultaneously using the same instruction. In the x86 architecture, this may include, for example, SSE (128-bit), AVX / AVX2 (256-bit), and AVX-512 (512-bit) instruction sets.

[0066] 7) Floating-point number: A numerical format used in processors to represent real numbers. It allows computers to represent very large or very small values ​​with a finite number of bits by representing the number as a combination of a mantissa and an exponent.

[0067] It should be understood that floating-point numbers are mainly composed of three parts: the sign bit, the exponent (exp), and the mantissa, and their encoding format is shown in Table 1.

[0068] Table 1

[0069]

[0070] Taking single-precision floating-point numbers as an example, a single-precision floating-point number occupies 32 bits and is divided into the following three parts:

[0071] The sign bit (1 bit) is used to indicate whether the floating-point number is positive or negative; 0 indicates a positive number and 1 indicates a negative number.

[0072] The exponent bit, also known as the "exponent code" (8 bits), is used to represent the exponent part of a floating-point number; it uses offset representation, where the actual exponent value is the stored value minus 127 (the offset).

[0073] The mantissa (23 bits) represents the fractional part of the floating-point number.

[0074] In this application embodiment, floating-point numbers include multiple formats, namely: normal numbers, subnormal numbers, and special numbers. Special numbers can include positive and negative zero, positive and negative infinity, and NOT numbers. Specifically, the exponent and mantissa bits of positive and negative zero are all 0; the exponent bit of positive and negative infinity is all 1 and the mantissa bit is all 0; the exponent bit of NOT numbers is all 1 and the mantissa bit is not 0; the exponent bit of subnormal numbers is all 0 and the mantissa bit is not 0; and the rest are represented as normal numbers.

[0075] 8) Register Alignment Boundary: This refers to the address alignment rules that must be followed when storing data in memory. This rule ensures that the starting address of data in memory is a multiple of a certain value. The alignment boundary is usually related to the processor architecture and the size of the registers, with the aim of improving the efficiency and performance of memory access. In the embodiments of this application, the processor's alignment boundary is determined based on the size of the first register.

[0076] 9) Reduction Sum, is an algorithm commonly used in parallel computing to calculate the sum of a set of data. In reduction sum, the accumulation order refers to how to organize the addition operation of data to fully utilize the advantages of parallel computing and improve computing efficiency. Different accumulation orders will affect the performance and parallelism of the algorithm.

[0077] In related technologies, when the processor needs to perform dot product operation, it is usually necessary to multiply and accumulate one by one according to the order of elements in the vector. This processing method has a low utilization rate of the computing power of the processor, resulting in slow calculation speed. As the size of the vector to be calculated increases, the time required for dot product operation will increase significantly, which cannot meet the requirements of real-time scenarios.

[0078] In recent years, with the development of modern computer hardware, especially multi-core processors and special accelerators (such as GPUs), processors can process multiple thread logical operations. Scholars have proposed dividing two floating-point vectors for dot product calculation into multiple layers of data blocks, and performing parallel multiplication calculation on the elements in the multiple layers of data blocks, then accumulating the multiplication calculation results of the elements in the multiple layers of data blocks according to the division layers, and finally obtaining the accumulation result as the dot product operation result of the two floating-point vectors. However, due to the large range of values that floating-point numbers can represent, the results of parallel multiplication operations need to be accumulated to obtain the dot product operation result. When accumulating, the value of the floating-point number usually needs to be rounded, for example, when adding a large floating-point number such as 1x10 15 to a small floating-point number 1x10 -15 , the small floating-point number 1x10 -15 will be discarded. Therefore, in floating-point matrix dot product multiplication, the accumulation order in the dot product operation will affect the accuracy of the result. Therefore, the dot product operation method in related technologies, which directly divides the vectors to be calculated into multiple layers and performs parallel multiplication calculation on the elements in the multiple layers of data blocks, then accumulates the multiplication calculation results of the elements in the multiple layers of data blocks according to the layers, changes the accumulation order of the elements in the process of layering, resulting in a decrease in the accuracy of the dot product operation. The dot product operation method of related processors accumulates the multiplication operation results through a single thread of the processor in order to maintain the accumulation order of the multiplication operation results in the dot product, which cannot effectively utilize the parallel computing resources of modern processors and cannot fully utilize the performance advantages of hardware, limiting the computing efficiency.

[0079] In addition, the dot product operation method in related technologies also faces the following problems:

[0080] 1. After obtaining the multiplication operation result of the elements in the data block to be dot product decomposed, the multiplication operation result is accumulated, and the accumulated result is added according to layers, which is slow in operation speed.

[0081] 2. In the related art, two vectors to be dot product operated need to meet a set length to perform dot product operation in a manner that the vectors to be dot product operated can be decomposed into multiple layers, and multiplication and addition operation of multiplication operation result are performed layer by layer, which cannot calculate vectors of different sizes.

[0082] 3. The register used to perform dot product operation needs to multiply and accumulate elements one by one, and when large-scale operation is performed, the processor loads the elements to be accumulated in the next stage into the register only after completing the accumulation in a stage, which is low in calculation efficiency.

[0083] That is, although some schemes capable of floating point dot product operation exist in the related art, these schemes still have some defects, such as precision loss, low calculation efficiency, single-thread processing, thereby wasting the calculation performance of a multi-core processor, and failing to meet the requirement of real-time performance.

[0084] Therefore, an embodiment of the present application provides a dot product operation method and device of a processor, a chip, an electronic device, a computer readable storage medium, and a computer program product. For the processor, parallel multiplication can be realized while maintaining the accumulation order of dot product operation, and the parallel multiplication operation result is accumulated, thereby realizing parallel execution of dot product operation of the processor, improving the calculation speed of dot product operation, avoiding waste of hardware resources, and improving operation efficiency. The following describes an exemplary application of the dot product operation device provided by the embodiment of the present application. The device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart television, a vehicle-mounted terminal, and various types of terminals. The device can also be implemented as a server. The following describes an exemplary application when the device is implemented as a terminal or a server.

[0085] In some embodiments, the server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, and other basic cloud computing services.

[0086] Referring to Figure 1 , Figure 1 is a structural schematic diagram of an electronic device 100 provided by an embodiment of the present applicationFigure 1 , Figure 1 The illustrated electronic device 100 includes at least one processor 10, volatile memory 120, and non-volatile memory 130. Various components in the electronic device 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to implement communication between these components. In addition to a data bus, the bus system 140 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 1 The general labeled all buses as Bus System 140.

[0087] Processor 10 has signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0088] Volatile memory 120, an exemplary hardware device, includes random access memory (RAM). Volatile memory 120 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof. In this embodiment, the memory mentioned below refers to volatile memory 120.

[0089] Non-volatile memory 130 is used to store executable software.

[0090] In some embodiments, the matrix dot product apparatus provided in this application can be implemented in software. For example, the processor 10 executes... Figure 2 The dot product operation device 131 in the non-volatile memory 130 shown implements the dot product operation method of the processor described above. The dot product operation device 131 can be software in the form of programs and plug-ins, including the following software modules: a segmentation and loading module 1311, an element rearrangement module 1312, a parallel computing module 1313, and a result processing module 1314. These modules are logically related, and therefore can be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0091] See Figure 2 , Figure 2 This is a schematic diagram of the architecture of the processor 10 provided in an embodiment of this application, as shown below. Figure 3 The processor 10 shown may include a floating-point processing unit 111, multiple registers 112, and a cache 113, wherein:

[0092] The register 112, which can be collectively referred to as all registers involved in the embodiments of the present application, including the first register and the second register, is used to load the matrix, vector or data block to be calculated for dot product into a set position in the memory. The register 112 is also used to accept the control instruction of the floating point processing unit 111 and perform operations or position adjustment on the elements of the matrix, vector or data block stored in the set position.

[0093] In the embodiments of the present application, the floating point processing unit 111 can use multiple threads to perform parallel logical operations, such as multiplication, addition and position permutation, on multiple elements in the register 112 to obtain corresponding operation results.

[0094] By way of example, the entire floating point processing process can be divided into storing the vector to be calculated for dot product in the memory, which is stored in the disk storage, other memory area or input by an external device through an input device. The processor 10 loads the vector or data block in the memory through the cache 113 or the register 112, and stores the elements in the loaded vector or data block in the register 112 in a set position. The floating point processing unit 111 performs parallel logical operations on the elements stored in the register 112, and stores the logical operation results in the register 112. After obtaining the logical operation results, the register 112 can output the logical operation results to the external memory of the processor 10 through the cache 113.

[0095] In other embodiments, the apparatus provided by the embodiments of the present application can be implemented in hardware. By way of example, the apparatus provided by the embodiments of the present application can be a hardware decoding processor programmed to perform the dot product operation method of the processor provided by the embodiments of the present application.

[0096] In some embodiments, the electronic device can implement the dot product operation method of the processor provided by the embodiments of the present application by running various computer executable instructions or computer programs. By way of example, the computer executable instructions can be microprogram level commands, machine instructions or software instructions. The computer program can be a native program in the operating system or a software module; the computer executable instructions described above can be any form of instructions, and the computer program described above can be any form of application program, module or plug-in.

[0097] The dot product operation method of the processor provided by the embodiments of the present application will be described in conjunction with an exemplary application and implementation of the electronic device provided by the embodiments of the present application.

[0098] The dot product operation method of the processor provided by the present application is applied to the scene of high-performance floating point calculation.

[0099] Referring to Figure 3 , Figure 3is a flowchart of a dot product operation method of a processor provided in the embodiments of the present application. The embodiments of the present application will be described below with reference to the steps shown in the flowchart. Figure 3 The steps shown in the flowchart will be described below. Figure 4 The steps shown in the flowchart will be described below.

[0100] In step 101, the first vector is divided into a plurality of first data blocks and the second vector is divided into a plurality of second data blocks according to an accumulation order, and the first data blocks and the second data blocks are loaded into the first registers of the processor, wherein the accumulation order includes an order of accumulating products of elements at corresponding positions in the first vector and the second vector in the case of single-thread dot product calculation of the first vector and the second vector and a number of data layers, and the first data blocks and the second data blocks are arranged one by one in the first registers.

[0101] In some embodiments, an electronic device such as a computer or a mobile phone, when solving a problem such as image processing, signal processing, artificial intelligence model calculation, linear calculation scientific calculation and statistical analysis, usually converts the problem to be solved into a series of vector calculations, and stores the vectors to be calculated in the memory. For example, when the electronic device performs an image processing task such as style transfer, the image to be processed is usually converted into a floating-point matrix or vector, and the result of the logical operation is restored to the image to achieve processing of the original image. In the embodiments of the present application, the first vector and the second vector are two vectors that need to be subjected to dot product operation.

[0102] In some embodiments, the first vector and the second vector can be stored in the memory outside the processor in advance, and the processor can load the first vector and the second vector in the memory through the cache for reading by the register to complete the dot product operation. The processor includes at least one first register, for example, a vector register (ZMM Register). The ZMM register is a 512-bit wide vector register in the x86 architecture for the AVX-512 instruction set, which can be used to load and process 32-bit and 64-bit integers, floating-point numbers and double-precision floating-point numbers, and the ZMM register supports a variety of data rearrangement operations.

[0103] In some embodiments, the accumulation order refers to the order of adding each product item in the vector when performing the dot product operation of two vectors. For example, it can be the order of arranging the elements in the first vector and the second vector, the order of dividing the first vector and the second vector into multiple layers, the order of calculating the product of the elements in each layer, the order of accumulating the product calculation results, and the order of merging the accumulation results. Referring to FIG. 4, taking a 4x8 vector with indexes from 0 to 31 as an example, Figure 3 The numbers in FIG. 4 represent the indexes of the elements in the vector. As can be seen, before the vector is divided, the accumulation order is 0 to 31 in ascending order. After the vector is divided into two data blocks, the first data block will perform the accumulation of the multiplication results of the elements with indexes from 8 to 11 after completing the accumulation of the multiplication results of the elements with indexes from 0 to 3. That is, both data blocks have broken the original accumulation order of the vector, which may result in the final accumulation result being inconsistent with the result of accumulating the multiplication results from 0 to 31.

[0104] In some embodiments, Figure 3 As shown in step 101, the first vector can be divided into multiple first data blocks and the second vector can be divided into multiple second data blocks according to the accumulation order by performing the following step 1011:

[0105] The first vector is divided into multiple first data blocks with a size of a processor processing granularity according to the accumulation order, and the second vector is divided into multiple second data blocks with a size of a processor processing granularity according to the accumulation order.

[0106] In some embodiments, the processing granularity of the processor refers to the degree of refinement of the processor in decomposing a task when performing the task. The processing granularity of the processor describes the amount of data or the size of the task that the processor can process at a time when processing data or instructions. In the embodiments of the present disclosure, the processing granularity of the processor can be embodied as the width of the first register. Taking an example of a vector V of a vector length of 16 to be loaded, V = [v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15], if the width of the first register is 256 bits, and the size of each element in the vector is a 32-bit floating point number, then each first register can store 8 floating point numbers, and therefore the vector V needs to be divided into V0 = [v0, v1, v2, v3, v4, v5, v6, v7] and V1 = [v8, v9, v10, v11, v12, v13, v14, v15]. The elements in the first data block and the second data block obtained by dividing the first vector and the second vector can be loaded into the plurality of first registers in turn.

[0107] In some embodiments, the accuracy of the first vector and the second vector can be adjusted by performing step 105 before step 101. Figure 3

[0108] In step 105, the first vector and the second vector are compressed into a compressed first vector and a compressed second vector, wherein the accuracy of the compressed first vector is lower than that of the first vector, and the accuracy of the compressed second vector is lower than that of the second vector.

[0109] In some embodiments, the high-precision data (such as a 32-bit floating point number) can be converted into low-precision data (such as an 8-bit integer) by quantization techniques (such as scalar quantization) to reduce storage and calculation costs, or the floating point numbers in the vector can be compressed from double precision (64 bits) to single precision (32 bits) or half precision (16 bits). The compression method that can be implemented can also be an adaptive lossless floating point compression algorithm (Algorithmic Level Progression, ALP), a normal compression method, etc., which is not specifically limited in the present disclosure.

[0110] The floating point compression can be implemented in the processor by a floating point processing unit or a vector processing unit, or can be run in the memory. Taking an example of performing floating point compression in the memory, the floating point numbers can be normalized to a specific range by a software algorithm, and the normalized floating point numbers are stored in the memory as compressed floating point numbers.​

[0111] By converting the first vector and the second vector into low-precision vectors through step 105, the storage cost and the calculation cost are reduced, and on the basis of the parallel dot product operation method of the present disclosure which maintains the accumulation order, the calculation speed is further improved. It should be noted that the precision reduction caused by the execution of the vector compression is a predictable precision reduction, and compared with the destruction of the accumulation order, the precision reduction caused by the vector compression is controllable.

[0112] In some embodiments, before step 101, the number of elements of the first vector and the second vector can be made an integer multiple of the number of elements that the first register can store by performing step 106. Figure 3

[0113] In step 106, the first vector is written into a first buffer area of the memory, and null elements are filled in the first buffer area so that the number of elements stored in the first buffer area is an integer multiple of the number of elements that the first register can store.

[0114] The second vector is written into a second buffer area of the memory, and null elements are filled in the second buffer area so that the number of elements stored in the second buffer area is an integer multiple of the number of elements that the first register can store.

[0115] In some embodiments, the processor usually loads the vectors in the memory in blocks of a fixed size through the cache, that is, a fixed number of elements are loaded in the cache each time. The first vector and the second vector to be calculated may not be consistent with the multiple of the number of elements that the register can store for calculation, that is, the size of the data block is not aligned with the size that the cache in the processor can load. If the size is not aligned, the processor may need to access the memory multiple times and splice the data obtained by the access to obtain complete data, which increases the instruction overhead and reduces the speed of the processor loading the vectors in the memory, and there is a risk of triggering an exception prompt.

[0116] By filling the buffer areas where the first vector and the second vector are stored with null values in step 106, the number of elements of the first vector and the second vector is made an integer multiple of the number of elements that the register can store, so that the first vector and the second vector are stored in the memory in a manner aligned with the alignment boundary of the processor. The processor can load the entire vector in the memory into the register through one instruction, which reduces the number of times the processor accesses the memory, improves the utilization rate of the register loading performance, reduces the complexity and cost of hardware loading, and improves the stability and security of the program.

[0117] In some embodiments, Figure 5 ​In step 101, the first data block and the second data block can be loaded into the first register of the processor by performing the following step 1011:

[0118] In step 1011, the plurality of first data blocks and the plurality of second data blocks are loaded into the plurality of first registers in parallel by the plurality of threads of the processor.

[0119] In some embodiments, a thread of the processor is the smallest unit that can be scheduled by the operating system of the processor. It is smaller than a process and is an execution unit in a process. The thread is an important mechanism for implementing concurrency and parallel processing in modern multitasking operating systems.

[0120] In step 1011, the first vector and the second vector are divided according to the processing granularity of the processor, the multi-core processing capability of the processor is utilized, and the single instruction multiple data characteristic of the SIMD instruction set is utilized, so that parallel writing and parallel calculation are realized, and the calculation efficiency is improved. In addition, since the processor usually loads data in the memory through the cache, and then reads the elements from the cache through the register, and the speed of reading the elements from the cache through the register is faster than the speed of reading the elements from the memory through the cache, in the embodiment, the vectors to be calculated by the dot product are divided into a plurality of data blocks, so that the register can read the complete data block from the cache, and the memory access efficiency is improved.

[0121] In step 102, the elements of the first data block are rearranged in the first register, and the elements of the second data block are rearranged, to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks, wherein the elements in each rearranged first data block are arranged in continuous order according to the accumulation order, and the elements in each rearranged second data block are arranged in continuous order according to the accumulation order.

[0122] In some embodiments, the loading of the first register is considered to be completed when the positions in the first register are all loaded with floating-point numbers. Referring to Figure 6 and Figure 5 Taking the first vector with a size of 4x16 as an example, the effect of rearranging the elements of the first data block is described. Figure 6 is a schematic diagram of the first data block before rearrangement, Figure 5 is a schematic diagram of the first data block after rearrangement. It should be noted that in Figure 5 , the serial numbers represent the accumulation order of the elements rather than the indices of the stored elements. As Figure 6 indicates, after the data blocks are loaded into the first register, the accumulation order in the plurality of first data blocks is changed compared with the accumulation order in the first vector, so that the first data blocks are directly used for parallel multiplication and addition calculation, and the result may have insufficient precision. By rearranging the elements of the first data block, a result asFigure 6 The rearranged first data block is shown in Figure 3 In the above, the element corresponding to the sequence number represents the accumulation sequence rather than the index of the stored element. It can be seen that the elements in the rearranged first data block are arranged according to the accumulation sequence. Since the dot product operation of a matrix is equivalent to multiplying the elements at corresponding positions in the two matrices and accumulating the products in sequence, the rearranged first data block obtained by the method performs parallel multiplication and addition, and the calculation result of the matrix obtained by element-by-element multiplication and accumulation is the same. The position of the element at the corresponding position that needs to be multiplied when performing the dot product of the two matrices, for example, when calculating the dot product result of the first row and the first column, the elements in the first row of the first data block are multiplied with the elements in the first column of the second data block one by one. The elements in the first row of the first data block and the elements in the first column of the second data block are the elements at the corresponding positions.

[0123] In some embodiments, Figure 3 The step 102 shown can be implemented by the following step 1021 and step 1022, which are specifically described below.

[0124] In step 1021, rearrange the elements of the first data block by permuting one or more rows of elements of each first data block through a single instruction multiple data hardware instruction set, to obtain a plurality of rearranged first data blocks, wherein the permutation includes permutation within the data block or permutation between two data blocks.

[0125] In step 1022, rearrange the second data block by permuting one or more rows of elements of each second data block through a single instruction multiple data hardware instruction set, to obtain a plurality of rearranged second data blocks.

[0126] In some embodiments, the permutation of elements in the data block can be implemented by the SIMD instruction mm512_permutex2var_epi64.

[0127] By means of the steps 1021 and 1022, the elements in the plurality of first data blocks in the same row are replaced by the way of the single instruction multiple data hardware instruction set, the parallel rearrangement of the elements in the first register is fully utilized by the SIMD instruction, the rearrangement efficiency is improved, and the time required for the floating-point dot product operation is further reduced. It should be noted that, although the arrangement order of the elements in the rearranged data block in the first register is the same as the accumulation and summation order in which the two vectors are loaded into the register one by one according to the summation order, the way of loading the elements in the data block one by one needs to write the elements in the positions one by one, that is, as many times of loading as many elements, and in the scheme of the present application, the elements in the first data block and the second data block are loaded into the register in parallel by the strategy of replacing the rows one by one, and only parallel writing and replacing according to the number of rows are needed to obtain the rearranged data block, thereby improving the efficiency of loading the elements in the data block in the register.

[0128] In step 103, the elements in the plurality of corresponding positions in the rearranged first data block and the rearranged second data block are subjected to parallel multiplication and addition operations by the plurality of threads of the processor.

[0129] In some embodiments, the dot product operation of the matrix can be converted into multiplication and addition of the elements in the corresponding positions in the matrix, and in the case where the accumulation order is determined, the time of the calculation result of the product of the elements in the first data block and the second data block does not affect the calculation result, so that the plurality of threads in the processor can perform parallel multiplication on the elements in the plurality of positions, and perform addition calculation on the parallel multiplication and addition operations. The corresponding positions refer to the positions that need to be multiplied when the dot product of the two matrices is calculated, for example, the element in the first row and the second column in the first data block needs to be multiplied with the element in the second row and the first column in the second data block, and the two positions are corresponding positions.

[0130] In step 104, the dot product operation result of the first vector and the second vector is obtained based on the result of the parallel multiplication and addition operation.

[0131] In some embodiments, Figure 3 The step 104 shown can be implemented by the following step 1041, which is specifically described below.

[0132] In step 1041, the arrangement order of the elements in the result of the parallel multiplication and addition operation is adjusted so that the arrangement order of the elements in the result of the parallel multiplication and addition operation conforms to the order of the reduction summation, and a third data block is obtained; the elements in the third data block are subjected to parallel accumulation calculation, and the dot product operation result of the first vector and the second vector is obtained.

[0133] By the above step 1041, in the case of eliminating the cumulative sequence change, each first data block and the corresponding second data block can obtain a result of parallel multiplication and addition operation, and the results of parallel multiplication and addition operation of the plurality of first data blocks and second data blocks are merged to obtain the dot product operation result of the first vector and the second vector. When the results of addition operation are merged, the results of parallel multiplication and addition operation are arranged according to the reduction summation sequence, which can improve the efficiency of parallel summation in the register, reduce errors, and better utilize hardware resources. For example, in the process of accumulation, the adjacent elements performing addition calculation are also adjacent in the position in the register, which can reduce the register access delay and improve the read hit rate of elements in the register. In addition, the global synchronization operation usually introduces additional delay, and the reduction summation sequence can reduce the number of global synchronization and improve the efficiency of the overall addition operation.

[0134] In some embodiments, after the step 104 as shown, the step 107-108 is further included, which is specifically explained as follows. Figure 7

[0135] In the step 107, in response to detecting the first data block and the second data block not loaded into the first register, the first data block and the second data block not loaded into the first register are loaded into the first register, and the unloading dot product calculation result is calculated, and the detection of the first data block and the second data block not loaded into the first register is repeated until the first data block and the second data block are all loaded into the first register.

[0136] In some embodiments, the multiplication and addition operation result is stored in the second register; and the second register is a register in the processor distinguished from the first register.

[0137] The step 107 can be implemented by performing the step 1071, which is specifically explained as follows.

[0138] In the step 1071, in response to detecting the first data block and the second data block not loaded into the first register and completing the parallel multiplication and addition operation, the first data block and the second data block not loaded into the first register are loaded into the first register.

[0139] By the above step 1071, since it is clear that the register needs to perform multiple rounds of calculation when calculating large-scale vectors, the calculation result is stored in the second register after the parallel multiplication and addition operation of the data in the first register is completed, so as to empty the first register. In the case that there is no data in the first register, the data needed to be calculated in the next round can be prefetched, so as to improve the calculation efficiency and reduce the time required for dot product operation.

[0140] ​In step 108, the dot product calculation result and the unloaded dot product calculation result are added to obtain a new dot product calculation result.

[0141] In some embodiments, the scale of the first vector and the second vector can be large, so that the processor cannot complete the calculation by loading once through the register, and needs to load and calculate in batches through the register.

[0142] Through steps 107-108, the large-scale first vector and the second vector are calculated in multiple rounds, and the calculation results of multiple rounds are combined, so that the parallel dot product operation of the large-scale vector is realized, and the calculation speed of the large-scale vector dot product operation is improved.

[0143] Next, an exemplary application of the embodiment of the present application in an actual application scenario will be described.

[0144] Referring to Figure 7 , Figure 8 is a flowchart of an actual application scenario of the embodiment of the present application.

[0145] Next, an exemplary application of the embodiment of the present application in an actual application scenario will be described.

[0146] In step 201, data preprocessing is performed.

[0147] In some embodiments, according to the buffer size of the memory storing the A vector and the B vector and the size of the data block that the register can process, the number of layers in which the vector A and the vector B are divided is determined. According to the number of layers, the accumulation order of the divided vector A and vector B is determined. For example, the buffer for storing the A vector and the B vector can store 128 elements, and the register can store 32 elements, so that the dot product accumulation order between the data blocks of the vector A and the vector B is determined as follows:

[0148] P4_0=P3_0+P3_1;

[0149] P3_0=P2_0+P2_1;

[0150] P3_1=P2_2+P2_3;

[0151] Wherein, P4_0 represents the dot product operation result, P2_0, P2_1, P2_2, P2_3 are the granularity sub-vector dot product sums, and each is calculated independently, and P3_0 and P3_1 represent the parallel multiplication addition operation results of the elements in the data block.

[0152] Taking the dot product of sub-vectors and P2_0 as an example, taking the vector represented by the first data block as a, and taking the vector represented by the second data block as b, the dot product accumulation sequence in the data block is as follows:

[0153] P2_0=P1_0+P1_1;

[0154] P1_0=P0_0+P0_1;

[0155] P1_1=P0_2+P0_3;

[0156] Wherein, P0_0, P0_1, P0_2, P0_3 represent the accumulation results of the multiplication operation results of the plurality of elements in the data block:

[0157] P0_0=a0*b0+a2*b2+a4*b4+a6*b6+a8*b8+a10*b10+a12*b12+a14*b14;

[0158] P0_1=a1*b1+a3*b3+a5*b5+a7*b7+a9*b9+a11*b11+a13*b13+a15*b15;

[0159] P0_2=a16*b16+a18*b18+a20*b20+a22*b22+a24*b24+a26*b26+a28*b28+a30*b30;

[0160] P0_3=a17*b17+a19*b19+a21*b21+a23*b23+a25*b25+a27*b27+a29*b29+a31*b31;

[0161] After determining the accumulation sequence, if the vector A and the vector B participating in the dot product are less than 128 elements, the vector A and the vector B participating in the dot product are stored in the buffer, and the buffer is null filled to make the elements stored in the buffer be 128 elements, so as to obtain the aligned vector A and the aligned vector B.

[0162] In step 202, the aligned vector A and the aligned vector B are divided into data blocks according to the parallel operation capability supported by the processor according to the AVX512 instruction level, and are loaded into the ZMM register of the AVX512 through the built-in function (load Intrinsic) of the AVX512. For the aligned vector A and the aligned vector B, 4 data blocks are cut into the ZMM register each time. In the loading process, an efficient prefetch strategy is adopted to reduce data transmission delay and improve data access efficiency.

[0163] In step 203, data rearrangement is performed in the case that the slicing is not completed or the rearrangement of the sliced data blocks is not completed.

[0164] Each group of data blocks from the first vector and the second vector is taken as one group, and four groups of data blocks (a0, b0), (a1, b1), (a2, b2), (a3, b3) are obtained. Wherein, the index of the elements in each data block is denoted as {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31}.

[0165] For the elements in each data block, the AVX512 permutex2var related Intrinsic is used to rearrange the elements in the data block, and the elements in the data block are permuted in the data blocks from the same vector. Taking the first data block slicing into four data blocks as an example, the elements in each data block retain the original row of elements, and the other row elements are permuted into the other three data blocks. For example, the elements in the first data block are rearranged into the elements composed of the first row of the plurality of data blocks from the same vector, and the elements in the second data block are rearranged into the elements composed of the second row of the plurality of data blocks from the same vector.

[0166] For example, the index of the elements in a0 is changed to:

[0167] a02, a00, a03, a01, a018, a016, a019, a017,

[0168] a12, a10, a13, a11, a118, a116, a119, a117,

[0169] a22, a20, a23, a21, a218, a216, a219, a217,

[0170] a32, a30, a33, a31, a318, a316, a319, a317

[0171] };

[0172] The index of the elements in a1 is changed to:

[0173] a06, a04, a07, a05, a022, a020, a023, a021,

[0174] a16, a14, a17, a15, a122, a120, a123, a121,

[0175] a26, a24, a27, a25, a222, a220, a223, a221,

[0176] a36, a34, a37, a35, a322, a320, a323, a321

[0177] };

[0178] a2 in the index of the element is changed to

[0179] a010, a08, a011, a09, a026, a024, a027, a025,

[0180] a110, a18, a111, a19, a126, a124, a127, a125,

[0181] a210, a28, a211, a29, a226, a224, a227, a225,

[0182] a310, a38, a311, a39, a326, a324, a327, a325

[0183] };

[0184] a3 in the index of the element is changed to

[0185] a014, a012, a015, a013, a030, a028, a031, a029,

[0186] a114, a112, a115, a113, a130, a128, a131, a129,

[0187] a214, a212, a215, a213, a230, a228, a231, a229,

[0188] a314, a312, a315, a313, a330, a328, a331, a329

[0189] }

[0190] It can be seen that the elements in the rearranged data block are arranged in the accumulation order, that is, according to the grouping mode of (a0, b0), (a1, b1), (a2, b2), (a3, b3), the elements at the corresponding positions of the two rearranged data blocks are multiplied in parallel and then added, and the addition order can be the same as the initially determined accumulation order. The corresponding position is the position of the two elements used for multiplication when performing dot product calculation on two matrices.

[0191] In step 204, after obtaining the plurality of rearranged first data blocks and the plurality of rearranged second data blocks, the _mms12_dpbf16_ps Intrinsic instruction in the AVX512 instruction set can be used to perform parallel multiplication and addition on the elements in the data blocks stored in the registers, that is, the half-precision floating-point vector elements.

[0192] Referring to Figure 8 , Figure 9 is the schematic diagram provided by the embodiment of the present application for performing parallel multiplication and addition on the half-precision floating-point vector elements in the registers. When performing the parallel multiplication and addition, the ZMM register P0 needs to be initialized first, and 4 vector dot product calculations are performed on the 4 groups of data blocks of (a0, b0), (a1, b1), (a2, b2), (a3, b3). Each vector dot product calculation saves 16 floating-point number results into the ZMM register P0, and the results are accumulated with the results of the next dot product calculation.

[0193] The instruction for initializing the ZMM register P0 can be:

[0194] P0 = _mm512_setzero_ps ().

[0195] The instruction for performing dot product calculation on each group of data blocks can be:

[0196] P0 = _mm512_dpbf16_ps (P0, a0, b0);

[0197] P0 = _mm512_dpbf16_ps (P0, a1, b1);

[0198] P0 = _mm512_dpbf16_ps (P0, a2, b2);

[0199] P0 = _mm512_dpbf16_ps (P0, a3, b3);

[0200] The above instructions represent that the data is divided into new data blocks (a0, b0), new data blocks (a1, b1), new data blocks (a2, b2), and new data blocks (a3, b3) and is accumulated and multiplied in turn.

[0201] In some embodiments, after each parallel operation period ends, the multiplication and addition calculation results in the registers can be written back to the memory, and format conversion and arrangement are performed according to the order required for subsequent data merging.

[0202] In step 205, the parallel multiplication and addition results obtained are reduced and summed.

[0203] After obtaining the parallel multiplication and addition calculation results, the order of the elements in P0 is adjusted so that the indices of the elements in P0 conform to the arrangement of the reduction sum. For example, the order of the elements in P0 before adjustment is {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, and the order of the elements in P0 after adjustment is {0, 8, 4, 12, 2, 10, 6, 14, 1, 9, 5, 13, 3, 11, 7, 15}. Arranging the elements in the order of the reduction sum can improve the calculation efficiency, reduce errors, and better utilize hardware resources.

[0204] After arranging the elements in the order of the reduction sum, the floating-point numbers stored in P0 can be summed in the order of accumulation by the element-by-element accumulation instruction AVX512_mm512_reduce_add_ps Intrinsic, to obtain the accurate result of the dot product of the data block, and then the accurate result of the dot product of the data block is stored in P4_0.

[0205] After the reduction sum is completed, step 202 is returned to detect whether there are vectors that have not been split and loaded into registers. If the splitting is not completed, the data block is continued to be split and input into the registers. If the splitting is completed, the parallel multiplication and addition results obtained are subjected to the reduction sum to obtain the accurate dot product operation result.

[0206] In some embodiments, if the split sub-vectors of the first vector and the second vector are not only 4, it can be determined whether all the split data blocks have been calculated and whether the next round of parallel calculation is needed according to the length of the first vector and the second vector and the processing granularity of the processor. If the next round of parallel calculation is needed, the calculation result is stored in the second register after the floating-point numbers in the ZMM register are calculated, the floating-point numbers that need to be calculated in parallel in the next round are prefetched, and the above parallel calculation process is repeated until the calculation of the data blocks in the first vector and the second vector is completed, to obtain the parallel calculation result.

[0207] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0208] The processor dot product operation method of the present application will be described taking the first vector and the second vector as vectors with less than 64 elements as an example.

[0209] In step 301, the accumulation order of the first vector and the second vector is determined, and the element number of the first vector and the second vector is adjusted to be the same as an integer multiple of the number of elements that can be stored in a register.

[0210] In the embodiment of the present application, the buffer size satisfying the first vector and the second vector is 64, that is, the first vector and the second vector need to be adjusted to the size of 64, and the positions of data less than 64 elements in the buffer for storing the first vector and the second vector in the memory are filled with null values, so that the number of elements of the first vector and the second vector is an integer multiple of the number of elements capable of being stored by the register.

[0211] In some embodiments, by taking the first vector and the second vector as a data block of 32 elements, the dot product order of the data block is determined as follows:

[0212] P4_0=P3_0+P3_1;

[0213] P3_0=P2_0+P2_1;

[0214] P3_1=P2_2+P2_3;

[0215] P2_0, P2_1, P2_2, P2_3 are sub-vector dot product sums with data block granularity, and are calculated independently. When there are remaining 64 elements, P2_2, P2_3 are only filled, and the dot product sum is 0.

[0216] For the sub-vector a and the sub-vector b, the dot product accumulation order in the sub-vector is as follows:

[0217] P2_0=P1_0+P1_1;

[0218] P1_0=P0_0+P0_1;

[0219] P1_1=P0_2+P0_3;

[0220] Where P0_0, P0_1, P0_2, P0_3 are independently summed:

[0221] P0_0=a0*b0+a2*b2+a4*b4+a6*b6+a8*b8+a10*b10+a12*b12+a14*b14;

[0222] P0_1=a1*b1+a3*b3+a5*b5+a7*b7+a9*b9+a11*b11+a13*b13+a15*b15;

[0223] P0_2=a16*b16+a18*b18+a20*b20+a22*b22+a24*b24+a26*b26+a28*b28+a30*b30;

[0224] P0_3 = a17*b17 + a19*b19 + a21*b21 + a23*b23 + a25*b25 + a27*b27 + a29*b29 + a31*b31;

[0225] In the above steps, the two half-precision floating-point vectors participating in the dot product operation are adjusted to be stored according to 64 elements, and the data is filled with necessary null values, so that the processor can completely read the first vector and the second vector, and further ensure the accuracy of subsequent calculation.

[0226] In step 302, the first vector is divided into a plurality of first data blocks according to the accumulation order, and the second vector is divided into a plurality of second data blocks according to the accumulation order, and the first data blocks and the second data blocks are loaded into the first registers of the processor.

[0227] In some embodiments, according to the parallel operation capability supported by the AVX512 instruction set according to the first vector and the second vector, the data blocks are divided, and the preprocessed half-precision floating-point vector data is quickly loaded from the memory to the ZMM registers of the AVX512 using the load intrinsic of the AVX512. As shown in Figure 1 As shown in the figure, since it is stored by 64 elements, when there are 64 remaining elements, two groups of data blocks 0 (a0, b0) and data blocks 1 (a1, b1) with numerical values and two groups of data blocks 2 (a2, b2) and data blocks 3 (a3, b3) with null values can be obtained.

[0228] The storage order of the elements in each data block is {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31}.

[0229] In step 303, the same rearrangement method as in step 204 is used to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks.

[0230] In step 304, the parallel multiplication and addition result is calculated by the same parallel multiplication and addition method as in step 205.

[0231] In step 305, the same rearrangement method as in step 206 is used to make the index of the elements in P0 conform to the placement method of the reduction sum, and the floating-point numbers stored in P0 are summed according to the accumulation order by using the element-by-element accumulation instruction AVX512_mm512_reduce_add_ps Intrinsic, to obtain the accurate result of the data block dot product, and then store the accurate result of the data block dot product in P4_0.

[0232] The following continues to illustrate an exemplary structure of the dot product operation apparatus 131 provided by the embodiments of the present application, which is implemented as a software module, in some embodiments, as shown in Figure 1 The processor 10 can run Figure 3 The dot product operation apparatus 131 in the non-volatile memory 130 shown, the software module in the dot product operation apparatus 131 can include:

[0233] The split loading module 1311 is configured to split the first vector into a plurality of first data blocks and split the second vector into a plurality of second data blocks according to an accumulation order, load the first data blocks and the second data blocks into a first register of the processor, wherein the accumulation order includes an order of accumulating products of elements at corresponding positions in the first vector and the second vector in a single-thread dot product calculation of the first vector and the second vector and a number of data layers, and the first data blocks and the second data blocks are arranged in one-to-one correspondence in the first register.

[0234] The element rearrangement module 1312 is configured to rearrange elements of the first data blocks in the first register and rearrange elements of the second data blocks to obtain a plurality of rearranged first data blocks and a plurality of rearranged second data blocks, wherein the elements in each of the rearranged first data blocks are arranged continuously according to the accumulation order, and the elements in each of the rearranged second data blocks are arranged continuously according to the accumulation order.

[0235] The parallel calculation module 1313 is configured to perform parallel multiplication and addition operations on elements at a plurality of corresponding positions in the rearranged first data blocks and the rearranged second data blocks by a plurality of threads of the processor.

[0236] The result arrangement module 1314 is configured to obtain a dot product operation result of the first vector and the second vector based on a result of the parallel multiplication and addition operations.

[0237] In some embodiments, the element rearrangement module 1312 is further configured to rearrange the elements of the first data blocks in the first register and rearrange the second data blocks to obtain the plurality of rearranged first data blocks and the plurality of rearranged second data blocks by the following manners:

[0238] Rearrange the elements of the first data blocks by a single instruction multiple data hardware instruction set to replace one or more rows of elements of each of the first data blocks to obtain the plurality of rearranged first data blocks, wherein the replacement includes replacement within the first data block or replacement between two first data blocks;

[0239] Rearrange the second data blocks by a single instruction multiple data hardware instruction set to replace one or more rows of elements of each of the second data blocks to obtain the plurality of rearranged second data blocks.

[0240] In some embodiments, the result arrangement module 1314 is further configured to implement the dot product operation of the first vector and the second vector based on the results of the parallel multiplication and addition operations by:

[0241] adjusting the arrangement order of the elements in the results of the parallel multiplication and addition operations to make the arrangement order of the elements in the results of the parallel multiplication and addition operations comply with the order of the reduction summation, to obtain a third data block; and performing parallel accumulation calculation on the elements in the third data block to obtain the dot product operation result of the first vector and the second vector.

[0242] In some embodiments, the number of elements of the first vector and the second vector is an integer multiple of the number of elements that can be stored in the first register.

[0243] The split loading module 1311 is further configured to implement the number of elements of the first vector and the second vector being an integer multiple of the number of elements that can be stored in the first register by performing the following processing:

[0244] writing the first vector into a first buffer area in the memory, and filling the first buffer area with null elements to make the number of elements stored in the first buffer area be an integer multiple of the number of elements that can be stored in the first register;

[0245] writing the second vector into a second buffer area in the memory, and filling the second buffer area with null elements to make the number of elements stored in the second buffer area be an integer multiple of the number of elements that can be stored in the first register.

[0246] In some embodiments, the split loading module 1311 is further configured to implement the splitting of the first vector into a plurality of first data blocks and the splitting of the second vector into a plurality of second data blocks according to the accumulation order by:

[0247] splitting the first vector into a plurality of first data blocks with a size of a processor processing granularity according to the accumulation order, and splitting the second vector into a plurality of second data blocks with a size of a processor processing granularity according to the accumulation order.

[0248] In some embodiments, the split loading module 1311 is further configured to implement the loading of the first data blocks and the second data blocks into the first register of the processor by:

[0249] loading the plurality of first data blocks and the plurality of second data blocks into the plurality of first registers in parallel by a plurality of threads of the processor.

[0250] In some embodiments, the split loading module 1311 is further configured to, in response to detecting that the first data block and the second data block are not loaded into the first register, load the first data block and the second data block that are not loaded into the first register into the first register, calculate the point product calculation result of the loaded data, and repeat the detection of the first data block and the second data block that are not loaded into the first register until the first data block and the second data block are all loaded into the first register.

[0251] The parallel calculation module 1313 is further configured to sum the point product calculation result and the point product calculation result of the loaded data to obtain a new point product calculation result.

[0252] Embodiments of the present application provide an electronic device, comprising:

[0253] a memory configured to store computer executable instructions or computer programs;

[0254] a processor configured to execute the computer executable instructions or computer programs stored in the memory to implement the dot product operation method of the processor provided by embodiments of the present application.

[0255] Embodiments of the present application provide a processing chip configured to execute the dot product operation method of the processor described above.

[0256] Embodiments of the present application provide a computer readable storage medium having computer executable instructions or computer programs stored therein, which, when executed by a processor, will cause the processor to execute the dot product operation method of the processor provided by embodiments of the present application, for example, the dot product operation method of the processor as shown in ​ .

[0257] In some embodiments, the computer readable storage medium can be a RAM, a ROM, a flash memory, a magnetic surface memory, an optical disc, or a CD-ROM, etc. storage medium; or can be various devices comprising one or any combination of the above storage medium.

[0258] In some embodiments, the computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0259] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the dot product operation method of the processor in the embodiment of the present application.

[0260] As an example, the computer executable instructions can be deployed to execute on one electronic device, or on multiple electronic devices located in one place, or on multiple electronic devices distributed in multiple places and interconnected through a communication network.

[0261] In summary, by rearranging the elements in the data block to be accumulated split from the vector in the register in the embodiment of the present application, the elements in each rearranged data block are arranged according to the accumulation order, so that the accumulation order of the elements in the data block is preserved by the way of multi-thread parallel calculation when the multiplication and addition operations required by the dot product algorithm are performed, so that the dot product operation method of the processor can be executed in parallel on the basis of maintaining the calculation precision, and the speed of the dot product operation of the processor is improved. The embodiment of the present application can rearrange by the way of element replacement in the register in the unit of row, compared with the way of writing or rearranging element by element, the speed of rearrangement is improved, and then the speed of dot product operation is improved. The embodiment of the present application fills the null value into the vector to be operated by the dot product operation, so that the vector is aligned with the alignment boundary of the processor, the multi-core processing capability of the processor and the single instruction multiple data characteristics of the SIMD instruction set are utilized, the parallel writing and parallel calculation are realized, and the calculation efficiency is improved. In addition, since the register loading performance of the processor is limited, by splitting the first vector and the second vector into multiple data blocks, it can be ensured that the data block processed each time can be completely put into the register, the number of times of processor loading miss is reduced, and the memory access efficiency is improved.

[0262] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for dot product operation in a processor, characterized in that, include: The first vector is divided into multiple first data blocks according to the accumulation order, and the second vector is divided into multiple second data blocks. The first data blocks and the second data blocks are loaded into the first register of the processor. The accumulation order includes the order and data layer number of accumulating the product of the corresponding elements in the first vector and the second vector when performing single-threaded dot product calculation on the first vector and the second vector. The first data blocks and the second data blocks are arranged in a one-to-one correspondence in the first register. The elements of the first data block and the elements of the second data block are rearranged in the first register to obtain multiple rearranged first data blocks and multiple rearranged second data blocks. The elements in each rearranged first data block are arranged continuously in the accumulation order, and the elements in each rearranged second data block are arranged continuously in the accumulation order. The processor uses multiple threads to perform parallel multiplication and addition operations on elements at multiple corresponding positions in the rearranged first data block and the rearranged second data block. The dot product of the first and second vectors is obtained based on the results of parallel multiplication and addition operations.

2. The dot product operation method for the processor according to claim 1, characterized in that, The step of rearranging the elements of the first data block in the first register and rearranging the second data block to obtain multiple rearranged first data blocks and multiple rearranged second data blocks includes: The elements of the first data block are rearranged by replacing one or more rows of elements in each first data block using a single instruction multiple data hardware instruction set, resulting in multiple rearranged first data blocks. The replacement includes replacement within a data block or replacement between two data blocks. By replacing one or more rows of elements in each of the second data blocks using a single instruction multiple data hardware instruction set, the second data blocks are rearranged to obtain multiple rearranged second data blocks.

3. The dot product operation method for the processor according to claim 1, characterized in that, The result of the dot product operation between the first vector and the second vector, obtained from the parallel multiplication and addition operations, includes: The order of elements in the results of the parallel multiplication and addition operations is adjusted to conform to the order of reduction summation, resulting in a third data block; the elements in the third data block are then subjected to parallel accumulation calculation to obtain the dot product result of the first vector and the second vector.

4. The dot product operation method for the processor according to claim 1, characterized in that, The number of elements in both the first vector and the second vector is an integer multiple of the number of elements that the first register can store; The following process is performed to ensure that the number of elements in the first vector and the second vector is an integer multiple of the number of elements that the first register can store: Write the first vector into the first buffer in memory, and fill the first buffer with null elements so that the number of elements stored in the first buffer is an integer multiple of the number of elements that the first register can store. The second vector is written into the second buffer of the memory, and null elements are filled into the second buffer so that the number of elements stored in the second buffer is an integer multiple of the number of elements that the first register can store.

5. The dot product operation method for the processor according to claim 1, characterized in that, The step of dividing the first vector into multiple first data blocks according to the accumulation order, and dividing the second vector into multiple second data blocks, includes: The first vector is divided into multiple first data blocks of processor-processing granularity according to the accumulation order, and the second vector is divided into multiple second data blocks of processor-processing granularity according to the accumulation order.

6. The dot product operation method for the processor according to claim 1, characterized in that, Loading the first data block and the second data block into the processor's first register includes: Multiple threads of the processor load multiple first data blocks and multiple second data blocks in parallel into multiple first registers.

7. The dot product operation method for the processor according to claim 1, characterized in that, The method further includes: In response to the detection of a first data block and a second data block that are not loaded into the first register, the first data block and the second data block that are not loaded into the first register are loaded into the first register, the calculation result of the unloaded dot product is obtained, and the detection of the first data block and the second data block that are not loaded into the first register is repeated until the first data block and the second data block are both loaded into the first register. The dot product calculation result and the unloaded dot product calculation result are summed to obtain a new dot product calculation result.

8. The dot product operation method for the processor according to claim 7, characterized in that, The results of the parallel multiplication and addition operations are stored in the second register; The step of loading the first data block and the second data block that were not loaded into the first register into the first register in response to detecting that they were not loaded into the first register includes: In response to detecting the first and second data blocks that are not loaded into the first register, and after completing the parallel multiplication and addition operations, the first and second data blocks that are not loaded into the first register are loaded into the first register.

9. The dot product operation method for a processor according to claim 1, characterized in that, Before dividing the first vector into multiple first data blocks according to the accumulation order, and dividing the second vector into multiple second data blocks, the method further includes: The first vector and the second vector are compressed into a compressed first vector and a compressed second vector, wherein the precision of the compressed first vector is lower than that of the first vector, and the precision of the compressed second vector is lower than that of the second vector.

10. A dot product calculation device, characterized in that, include: The segmentation and loading module is used to segment a first vector into multiple first data blocks and a second vector into multiple second data blocks according to the accumulation order, and load the first data blocks and second data blocks into the first register of the processor. The accumulation order includes the order and data layer number of accumulating the product of the corresponding elements in the first vector and the second vector when performing single-threaded dot product calculation on the first vector and the second vector. The first data blocks and the second data blocks are arranged in a one-to-one correspondence in the first register. An element rearrangement module is used to rearrange the elements of the first data block in the first register and to rearrange the elements of the second data block to obtain multiple rearranged first data blocks and multiple rearranged second data blocks, wherein the elements in each rearranged first data block are arranged continuously in the accumulation order, and the elements in each rearranged second data block are arranged continuously in the accumulation order. The parallel computing module is used to perform parallel multiplication and addition operations on elements at multiple corresponding positions in the rearranged first data block and the rearranged second data block through multiple threads of the processor. The result processing module is used to obtain the dot product result of the first vector and the second vector based on the results of parallel multiplication and addition operations.

Images