Matrix operation implementation device and method, processor architecture, chip and equipment
By introducing a matrix buffer structure and control unit into the CPU core, the matrix operation instructions are decomposed and loaded and calculated concurrently, solving the problem of multi-port access to the vector physical register group, and improving the CPU's matrix multiplication execution efficiency and hardware utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENGDU QUNXIN MICROELECTRONICS TECHNOLOGY CO LTD
- Filing Date
- 2025-12-15
- Publication Date
- 2026-05-12
AI Technical Summary
When performing matrix multiplication operations, existing CPUs experience chip area and timing convergence issues due to multi-port access to the vector physical register set, which also blocks the scheduling of other vector instructions and reduces the execution efficiency and hardware utilization of matrix multiplication.
By introducing a matrix buffer structure and a control unit, matrix operation instructions are broken down into loading and calculation micro-instructions. Data loading and operation are carried out concurrently and alternately through the matrix eraser and write board, which relieves the read port pressure of the vector physical register group and avoids access conflicts.
While maintaining the advantages of the CPU's general programming model, it reduces hardware overhead, alleviates access conflicts to the register file in matrix operations, and improves the energy efficiency and performance of the CPU when executing matrix-intensive loads.
Smart Images
Figure CN122018850A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of processor technology, and in particular to a matrix operation implementation device, method, processor architecture, chip, and equipment. Background Technology
[0002] Supporting matrix operations on the CPU side, with a simple programming model and no need for additional heterogeneous accelerator cards, offers significant advantages in AI inference deployment. Therefore, with the widespread application of artificial intelligence technology, neural network-based AI inference has been widely deployed on central processing units (CPUs). The core of such operations is a large number of matrix multiplication operations. To efficiently execute these calculations on general-purpose CPUs, modern CPU instruction set architectures (ISAs) typically introduce dedicated matrix computation instructions and integrate corresponding matrix execution units into the hardware pipeline. This instruction set provides a set of matrix registers as a software programming interface, allowing matrix data to be loaded from main memory into these registers and multiplying and adding operations directly within the core. This avoids the bottleneck of frequent memory accesses in traditional methods, improving computational efficiency.
[0003] In CPU microarchitecture implementations supporting matrix operations, a key design challenge lies in how to efficiently supply operand data to matrix multiplication units, particularly how to manage access to the register file. Currently, many modern CPUs employ a solution that reuses the existing Vector Physical Register File (VPRF) to construct matrix registers (when the CPU core enters matrix operation mode, it logically binds a predefined portion of the vector physical registers in the VPRF to form the required matrix register). Figure 1 The diagram illustrates VPRF (such as) achieved by reusing support vector operations. Figure 1 A typical microarchitecture implementation for matrix multiplication operations on vector matrices. (See reference) Figure 1 As shown, matrix memory access instructions (such as...) Figure 1 The MatrixLoad and MatrixStore in the matrix can be distributed to the Load Store Unit (LSU) through the distribution unit to load matrix data into the matrix physical register set (e.g., ...). Figure 1 Matrix PRF in the matrix); matrix multiplication operation instructions (such as...) Figure 1The matrix data is distributed to the scheduling queue (ScheduleQueue) by the Matrix Multiply Controller (MMC) after the matrix data is ready in the MPRF. The MMC then schedules the matrix data to the matrix multiplication execution unit (VDPB array) to perform the multiplication and addition operations and finally writes it back to the MPRF. Finally, the matrix storage instruction (MatrixStore) writes the result back to main memory through the LSU.
[0004] However, this approach has a significant performance bottleneck: when performing matrix multiplication and addition operations, the computation unit (such as a vector dot product array) typically needs to simultaneously read a row vector of matrix A and multiple column vectors of matrix B within a single cycle. For example, for a result matrix that supports outputting 16 elements per cycle, it may be necessary to simultaneously read one row vector of matrix A and 16 column vectors of matrix B. This means that the vector physical register set needs to support up to 17 read port operations within a single cycle. Implementing such a multi-port register file presents significant design challenges: on the one hand, a multi-port register file drastically increases chip area and wiring complexity; on the other hand, it puts enormous pressure on timing convergence, potentially limiting the increase in CPU clock speed. Moreover, the continuous high-intensity occupation of register ports by matrix operations severely blocks the scheduling of other vector instructions (especially matrix data loading / storage instructions), making it difficult to parallelize computation and data handling, thereby reducing the overall execution efficiency and hardware utilization of matrix multiplication.
[0005] Therefore, there is an urgent need for an innovative microarchitecture solution that can avoid or mitigate access conflicts to the register file during matrix multiplication with lower hardware overhead while maintaining the advantages of the CPU's general programming model, thereby improving the energy efficiency and performance of the CPU when performing matrix-intensive workloads such as AI inference. Summary of the Invention
[0006] The purpose of this application is to provide a matrix operation implementation device, method, processor architecture, chip, and equipment to avoid or mitigate access conflicts to the register file during matrix multiplication with lower hardware overhead.
[0007] To achieve the above objectives, in one aspect, embodiments of this application provide a matrix operation implementation apparatus, which is part of a central processing unit core, and the apparatus includes:
[0008] Vector physical register set; and,
[0009] Matrix operation unit; the matrix operation unit includes:
[0010] Matrix buffer structure;
[0011] The control unit is used to load the matrix data corresponding to the matrix operation instruction from the vector physical register group into the matrix buffer structure, read the matrix data from the matrix buffer structure and provide it to the vector dot product array to perform matrix operations.
[0012] In the matrix operation implementation device of this application embodiment, the device is part of the vector execution unit of the central processing unit core.
[0013] In the matrix operation implementation device of this application embodiment, the matrix buffer structure includes a matrix erasing plate.
[0014] In the matrix operation implementation apparatus of this application embodiment, the control unit is further configured to:
[0015] The matrix operation instructions are broken down into at least one loading microinstruction and at least one calculation microinstruction to adapt to the matrix erasing and writing board.
[0016] In the matrix operation implementation device of this application embodiment, the number of matrix erasing plates is at least two, and the control unit is used to enable at least two matrix erasing plates to participate in matrix data loading and matrix data operation concurrently and alternately.
[0017] In the matrix operation implementation device of this application embodiment, the control unit includes a matrix multiplication controller, and the matrix multiplication controller includes:
[0018] The matrix instruction queue is used to buffer received matrix multiplication operation instructions;
[0019] The microinstruction scheduling queue is used to store the disassembled microinstructions;
[0020] A finite state machine controller is used to schedule the microinstructions and manage the state of the matrix eraser / write pad.
[0021] In the matrix operation implementation device of this application embodiment, the vector dot product array includes multiple vector dot product operation units that support concurrent operation of multi-channel vector data.
[0022] In the matrix operation implementation device of this application embodiment, each vector dot product operation unit includes a two-level calculation structure, the two-level calculation structure including:
[0023] The first-level computational structure is used to calculate the dot product of multi-channel vector data and obtain the vector dot product result.
[0024] The second-level computational structure is used to accumulate the vector dot product results of the multi-channel vector data and output the accumulated results to the matrix physical register group that is separated from the vector physical register group.
[0025] In the matrix operation implementation device of this application embodiment, the finite state machine controller contains an idle global identifier list. The number of bits of the global identifiers in the list can index all different matrix multiplication operation instructions to be executed in the matrix instruction queue and the microinstruction scheduling queue. The target bit of the global identifier indicates the matrix eraser associated with it.
[0026] In the matrix operation implementation apparatus of this application embodiment, the matrix multiplication controller is further configured to:
[0027] When the matrix instruction queue is not full and there is a free global identifier in the list, the matrix multiplication operation instruction output by the central processing unit kernel distribution unit is received, a global identifier is allocated for the matrix multiplication operation instruction from the list, and the matrix multiplication operation instruction associated with the global identifier is written to the tail of the matrix instruction queue.
[0028] In the matrix operation implementation apparatus of this application embodiment, the matrix multiplication controller is further configured to:
[0029] When there is an idle entry in the microinstruction scheduling queue, the matrix multiplication operation instruction located at the head of the matrix instruction queue is decomposed into at least one loading microinstruction and at least one calculation microinstruction. The loading microinstruction, the calculation microinstruction and their respective associated global identifiers are written together into the corresponding idle entry in the microinstruction scheduling queue. When all the microinstructions after the matrix multiplication operation instruction is decomposed are written into the idle entry in the microinstruction scheduling queue, the entry occupied by the matrix multiplication operation instruction in the matrix instruction queue is released.
[0030] In the matrix operation implementation apparatus of this application embodiment, the matrix multiplication controller is further configured to:
[0031] The finite state machine controller listens for a first ready signal and a second ready signal corresponding to the loaded microinstruction. The first ready signal indicates that the matrix physical register group corresponding to the source operand of the loaded microinstruction is ready. The second ready signal indicates that the target matrix eraser / write plate corresponding to the destination operand of the loaded microinstruction is ready. When the first ready signal and the second ready signal corresponding to the loaded microinstruction are received, and the global identifier of the target matrix eraser / write plate of the loaded microinstruction matches its own global identifier, the finite state machine controller outputs a transmit signal for the loaded microinstruction to the microinstruction scheduling queue, so as to retrieve the loaded microinstruction from the microinstruction scheduling queue and dispatch it to the corresponding loading execution unit for execution. After the loaded microinstruction is executed, the microinstruction scheduling queue entry it occupies is released.
[0032] In the matrix operation implementation apparatus of this application embodiment, the matrix multiplication controller is further configured to:
[0033] The finite state machine controller listens for the third and fourth ready signals corresponding to the computation microinstruction. The third ready signal indicates that the source operand register of the computation microinstruction is ready. The fourth ready signal indicates that the vector dot product array corresponding to the destination operand of the computation microinstruction is ready. When the third and fourth ready signals corresponding to the computation microinstruction are received, and the global identifier of the source operand register of the computation microinstruction matches its own global identifier, the finite state machine controller outputs a transmit signal for the computation microinstruction to the microinstruction scheduling queue, so as to retrieve the computation microinstruction from the microinstruction scheduling queue and dispatch it to the corresponding computation execution unit for execution. After the computation microinstruction is executed, the microinstruction scheduling queue entry it occupies is released.
[0034] On the other hand, embodiments of this application also provide a processor architecture, which includes the matrix operation implementation device described above.
[0035] On the other hand, this application embodiment also provides a chip, which includes the matrix operation implementation device described above.
[0036] On the other hand, this application also provides a matrix operation implementation method, applied to the above-mentioned matrix operation implementation apparatus, the method comprising:
[0037] Receive matrix operation instructions;
[0038] Load the matrix data corresponding to the matrix operation instruction from the vector physical register group into the matrix buffer structure;
[0039] The matrix data is read from the matrix buffer structure and provided to the vector dot product array for matrix operations.
[0040] In the matrix operation implementation method of this application embodiment, the matrix buffer structure includes a matrix eraser / writer, and after receiving the matrix operation instruction, it further includes:
[0041] The matrix operation instructions are broken down into at least one loading microinstruction and at least one calculation microinstruction to adapt to the matrix erasing and writing board.
[0042] On the other hand, embodiments of this application also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions of the above-described method.
[0043] As can be seen from the technical solutions provided in the above embodiments of this application, by adding a matrix buffer structure between the vector physical register group and the control unit, the access to matrix data of the vector physical register group in matrix operations (such as matrix multiplication operations) is transferred to the matrix buffer structure. This avoids the area and timing convergence problems caused by the vector physical register group supporting too many read ports, relieves the pressure of read port requirements of the vector physical register group, and achieves the avoidance or mitigation of access conflicts to the register file in matrix operations with lower hardware overhead while maintaining the advantages of the general programming model. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0045] Figure 1 This diagram illustrates the structural block diagram of performing matrix multiplication in the CPU core using conventional techniques.
[0046] Figure 2 The following is a structural block diagram of a matrix operation implementation apparatus in some embodiments of this application;
[0047] Figure 3 The following are structural block diagrams of matrix operation implementation apparatuses in other embodiments of this application;
[0048] Figure 4 This paper shows a schematic diagram illustrating the working principle of the matrix multiplication controller of the matrix operation implementation device in some embodiments of this application;
[0049] Figure 5 This paper illustrates a data flow diagram of the matrix multiplication operation of the matrix multiplication controller of the matrix operation implementation apparatus in some embodiments of this application;
[0050] Figure 6 This paper shows a schematic diagram of the state transition of the matrix eraser / writer plate of the matrix operation implementation device in some embodiments of this application;
[0051] Figure 7 A schematic diagram of a VDPB array supporting concurrent multi-channel vector data operations is shown in some embodiments of this application;
[0052] Figure 8 Flowcharts illustrating matrix operation implementation methods in some embodiments of this application are shown;
[0053] Figure 9A structural block diagram of a computer device in some embodiments of this application is shown.
[0054] [Explanation of Labels in the Attached Image]
[0055] 10. Vector Physical Register Set;
[0056] 20. Matrix operation unit;
[0057] 21. Control unit;
[0058] 22. Matrix buffer structure;
[0059] 23. Vector dot product array;
[0060] 902. Computer equipment;
[0061] 904, Processor;
[0062] 906. Memory;
[0063] 908. Drive mechanism;
[0064] 910. Input / output interfaces;
[0065] 912. Input devices;
[0066] 914. Output devices;
[0067] 916. Presentation equipment;
[0068] 918. Graphical User Interface;
[0069] 920. Network interface;
[0070] 922. Communication link;
[0071] 924. Communication bus. Detailed Implementation
[0072] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.
[0073] It should be noted that in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved are all information and data authorized and agreed upon by the user and fully authorized by all parties. That is, the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.
[0074] Figure 2 The diagram shows a structural block diagram of a matrix operation implementation device in some embodiments of this application. This matrix operation implementation device is part of a central processing unit (CPU) core and may include a vector physical register group 10 and a matrix execution unit (MEX) 20. The matrix execution unit 20 includes a control unit 21, a matrix buffer structure 22, and a vector dot product array 23. The control unit 21 loads matrix data corresponding to matrix operation instructions from the vector physical register group 10 into the matrix buffer structure 22, reads the matrix data from the matrix buffer structure 22, and provides it to the vector dot product array 23 for matrix operation.
[0075] Figure 2 The embodiment shown adds a matrix buffer structure 22 between the vector physical register group 10 and the control unit 21, transferring the matrix data access to the vector physical register group 10 in matrix operations (such as matrix multiplication) to the matrix buffer structure 22. This avoids the area and timing convergence problems caused by the vector physical register group 10 supporting too many read ports, relieves the pressure on the read port requirements of the vector physical register group 10, and achieves the avoidance or mitigation of access conflicts to the register file in matrix operations with lower hardware overhead while maintaining the advantages of the CPU's general programming model.
[0076] Figure 3The diagram shows a structural block diagram of a matrix operation implementation device in some other embodiments of this application. This matrix operation implementation device is located in the Vector Execution Unit (VEX) of the CPU core, and may include a vector PRF (VPRF) and a matrix operation unit (MEX). The matrix operation unit includes: a matrix multiply controller (MMC) as a control unit, a matrix scratch pad (MSP) as a matrix buffer structure, and a vector dot product array (VDPB array). The MMC can load the matrix PRF (Matrix Physical Register File, MPRF) corresponding to the matrix operation instructions, which is segmented from the vector PRF, into the MSP; read matrix data from the MSP and provide it to the VDPB array for matrix operation; and write the calculation result into the MPRF.
[0077] For example, in Figure 3 In the illustrated embodiment, MMC can load a matrix B of N-dimensional vectors (N-vectors) from MPRF to MSP. MMC can also load Matrix B from MSP and vector A from MPRF. m Sum vector C m Loaded into the VDPB array to perform matrix multiplication based on these operands. Specifically, in Figure 3 In this context, Vector op Pipes refers to the vector operation pipeline (p0 to p1 are the vector operation pipeline identifiers).
[0078] Figure 3 The illustrated embodiment adds an MSP to the VEX of the CPU core to transfer matrix data access to the vector physical register set in matrix multiplication operations to the MSP. This avoids the area and timing convergence problems caused by the vector physical register set supporting too many read ports, relieves the pressure on the read port requirements of the vector physical register set, and achieves the avoidance or mitigation of access conflicts to the register file in matrix operations with lower hardware overhead while maintaining the advantages of the CPU's general programming model. This improves the energy efficiency and performance of the CPU when performing matrix-intensive workloads such as AI inference.
[0079] In some embodiments of this application, the MSP is a dedicated storage unit for temporarily storing matrix data and is independent of the VPRF. For example, in... Figure 5In the illustrated embodiment, the MSP can be, for example, a circuit structure storing 8x256 bits of data, used to load 8x8 matrix data of matrix B using a 4-channel INT8 or 2-channel BF16. The data of matrix B in the MSP remains valid throughout the entire lifetime of the matrix multiplication operation instruction and can be read multiple times or repeatedly within this lifetime. Figure 5 In the matrix, matrix B[b0, b1, b2, ..., b7] and matrix A[a0, a1, ..., a...] are... M-1 ] and matrix C[c0 c1、…、c M-1 ] can be used as a matrix operand in matrix multiplication operations; where matrix A and matrix C can be obtained from MPRF.
[0080] In some embodiments of this application, depending on the bit width of the VPRF (128bit / 256bit / 512bit / 1024bit), the MEX can support matrix multiplication operations of different data types such as 4x4, 8x8, 16x16, and 32x32. Therefore, in some embodiments of this application, the MEX based on a 256-bit bit width VPRF and an 8x8 4-channel INT8 data type is used as an example only and should not be construed as a limitation of this application.
[0081] In some embodiments of this application, the number of MSPs is at least two (or an even number when there are multiple MSPs). Thus, under the control of the MMC, two or more even numbers of MSPs can concurrently and alternately participate in loading and calculating matrix data in a ping-pong manner, thereby improving the processing efficiency of matrix multiplication operations. Furthermore, to facilitate the concurrent and alternate participation of multiple MSPs in loading and calculating matrix data in a ping-pong manner, after receiving the matrix multiplication operation instruction (MatrixMad instruction), the MMC can decompose the matrix multiplication operation instruction into multiple microinstructions. These microinstructions include at least one loading microinstruction (movmsp microinstruction) and at least one calculation microinstruction (vdpb microinstruction).
[0082] For example, taking two 8x256-bit MSPs (MSP0 and MSP1) as an example, while MSP0 is loading matrix data from the VPRF to MSP0 using a movmsp microinstruction, MSP1 can concurrently provide the matrix data in MSP1 to the VDPB array for computation using a vdpb microinstruction; while MSP1 is loading matrix data from the VPRF to MSP1 using a movmsp microinstruction, MSP0 can concurrently provide the matrix data in MSP0 to the VDPB array for computation using a vdpb microinstruction; in this way, MSP0 and MSP1 can concurrently and alternately participate in loading and computing matrix data in a ping-pong manner.
[0083] In some embodiments of this application, a MatrixMad instruction (MatrixMad C, A, B) transmits the data of matrix B from the MPRF to the MSP, and then performs operations in the VDPB array using the row vectors of matrices C and A provided by the MPRF and all column vectors of matrix B provided by the MSP. In this embodiment, the matrix multiplication instructions distributed to the MMC have been renamed at the previous stage; therefore, A, B, and C in the above matrix multiplication instructions also represent the renamed matrix physical register indices. Since each physical matrix register consists of a set of consecutive physical vector registers, in the implementation supporting the 8x8 matrix calculation core, the index number of each physical vector register in the physical matrix register satisfies:
[0084] vprn = mpr*8+n, 0≤n≤7, * represents multiplication.
[0085] To enable concurrent execution of MSP data transfer and VDPB array computation between different MatrixMad instructions, the MatrixMad instruction can be decomposed into 4 movmsp micro-instructions and 8 vdpb micro-instructions. Each movmsp micro-instruction is responsible for loading matrix data from the MPRF into the corresponding MSP; each vdpb micro-instruction is responsible for performing matrix multiplication operations and outputting the vector result. The specific decomposition scheme of the MatrixMad instruction is shown in Table 1 below:
[0086] Table 1
[0087]
[0088] In Table 1 above, the source operand registers of the movmsp and vdpb microinstructions are the index numbers of the corresponding vector physical registers in the matrix register; the vector register index number in the implicit source operand MSP of the movmsp microinstruction is the same as the vector index number of the explicit source operand vector register in its matrix register.
[0089] refer to Figure 4As shown in some embodiments of this application, the MMC may include: a MatrixInstruction Queue (MIQ), a Microinstruction Scheduling Queue (MSQ), and a Finite State Machine Controller (FSMC). The MIQ is used to buffer received matrix multiplication instructions; that is, the MIQ is a buffer queue storing MatrixMad instructions output from the CPU core dispatch unit. The MSQ is used to store the decomposed microinstructions; that is, the MSQ is a microinstruction scheduling queue storing the microinstructions decomposed from the MatrixMad instructions. The FSMC is used to schedule the microinstructions (schedule microinstruction execution recording and calculation) and manage the state of the matrix eraser / write pad. The FSMC internally contains a Globally Identifier Queue (GIDQ). The bit coverage of the GID (or gid) in the GIDQ can index all different matrix multiplication instructions to be executed in the MIQ and MSQ. The target bit of the GID indicates (e.g., the least significant bit) its associated matrix eraser / write pad (e.g., MSP0 or MSP1).
[0090] refer to Figure 4 As shown in some embodiments of this application, the working principle of MMC includes the following:
[0091] 1. When MIQ is not full and there are free GIDs in GIDQ, receive the MatrixMad instruction output by the CPU kernel distribution unit, allocate a GID for each MatrixMad instruction from GIDQ, write and associate the MatrixMad instruction with the GID (e.g., ... Figure 4 MatrixMad / gid0, MatrixMad / gid1, etc. are written into the free entry at the tail of the MIQ queue.
[0092] 2. When an idle entry point exists in the MSQ, the MatrixMad instruction at the head of the MIQ queue is decomposed into multiple movmsp micro-instructions and multiple vdpb micro-instructions. The multiple movmsp micro-instructions, multiple vdpb micro-instructions, and their respective associated GIDs (e.g., ...) are then processed. Figure 4 The microinstructions (vdpb / gid0, movmsp / gid0, vdpb / gid1, movmsp / gid1, etc.) are written to the corresponding free entry in the MSQ queue. After all the microinstructions after the MatrixMad instruction is decomposed are written to the free entry in the MSQ queue, the entry occupied by the MatrixMad instruction in the MIQ is released.
[0093] 3. The FSMC listens for the first and second ready signals corresponding to the movmsp microinstruction; the first ready signal (e.g. Figure 4 The first ready signal (the source operand register ready signal) indicates that the matrix physical register group corresponding to the source operand of the movmsp microinstruction is ready; the second ready signal indicates that the target MSP (such as MSP0 or MSP1) corresponding to the destination operand of the movmsp microinstruction is ready; when the first ready signal and the second ready signal corresponding to the movmsp microinstruction are received, and the GID of the target MSP of the movmsp microinstruction matches its own GID, the finite state machine controller outputs a transmit signal (such as...) to the MSQ for the movmsp microinstruction. Figure 4 The MSQ wake-up signal in the FSMC is used to retrieve the movmsp microinstruction from the MSQ and dispatch it to the corresponding load execution unit for execution (the FSMC can output an MSP selection signal based on the second ready signal to select the target MSP and load the movmsp original operand (e.g., the B[m] vector) from the MPRF into the target MSP by executing the movmsp microinstruction); after the movmsp microinstruction is executed, the MSQ entry it occupies is released. Figure 4 In this context, MSPDEMUX is an MSP DEMUX demultiplexer used to demultiplex two MSQs.
[0094] 4. The FSMC monitors the third and fourth ready signals corresponding to the vdpb microinstruction; the third ready signal (e.g. Figure 4 The first signal indicates that the source operand register (source MSP, MPRF) of the VDPB microinstruction is ready; the second signal indicates that the VDPB array corresponding to the destination operand of the VDPB microinstruction is ready; when the third and fourth ready signals corresponding to the VDPB microinstruction are received, and the GID of the source operand register of the VDPB microinstruction matches its own GID, the FSMC outputs a transmit signal for the VDPB microinstruction to the MSQ (e.g., ...). Figure 4The VDPB array uses the MSQ wake-up signal to retrieve the VDPB microinstruction from the microinstruction scheduling queue and dispatch it to the corresponding computation execution unit for execution (the FSMC can output an MSP selection signal based on the fourth ready signal to retrieve the VDPB microinstruction from the MSQ and provide it to the VDPB array for execution). After the VDPB microinstruction is executed, it releases the MSQ entry it occupies. When the VDPB array executes the VDPB microinstruction, the VDPB source operands can include: the MSP array from the source MSP, the A[m] vector from the MPRF, and the C[m] vector from the MPRF. If the calculation logic of the MatrixMad instruction is C += A * B in Table 1 above, then the VDPB array can output the calculated C[m] vector to the MPRF.
[0095] It should be noted that, in the embodiments of this application, for movmsp microinstructions and vdpb microinstructions with different GIDs belonging to different MatrixMad instructions, FSMC can schedule them to be executed concurrently within the same cycle to improve the efficiency of matrix multiplication instruction execution. For example, MatrixMad1 corresponds to vdpb / gid0 and movmsp / gid0, MatrixMad1 corresponds to vdpb / gid1 and movmsp / gid1, and MatrixMad2 corresponds to vdpb / gid2 and movmsp / gid2; FSMC can schedule them to have MSP1 participate in the calculation of vdpb / gid0 and MSP0 participate in the loading of movmsp / gid1 within one cycle T1; and in another cycle T2, have MSP1 participate in the loading of movmsp / gid2 and MSP0 participate in the calculation of vdpb / gid1.
[0096] Combination Figure 6 As shown, in some embodiments of this application, taking two MSPs (i.e., MSP0 and MSP1) as an example, MSP0 and MSP1 each contain a gid0 counter and a gid1 counter (the gid0 and gid1 counters can be... Figure 5 The time step picker in the clock data structure, where Time Step represents a time step (i.e., a clock cycle), is used. gid0 and gid1 are initially assigned 0 and 1 respectively. MSP0 and MSP1 transition between the following states:
[0097] (1) Idle state
[0098] MSP0 / MSP1 are initially in the IDLE state and are assigned initial values of 0 and 1 respectively. The movsmp micro-instructions containing gid=0 and gid=1 in MSQ are assigned to the target MSP in the ready state. When both the target MSP and the source operand register are ready, the MMC issues the ready movmsp micro-instruction, and the target MSP enters the loading state.
[0099] (2) Loading status
[0100] The MSP in the LOADING state continuously receives data for executing movmsp instructions and counts the number of movmsp instructions received. When the count reaches the total number of movmsp microinstructions decomposed from each MatrixMad instruction, the MSP, containing the complete matrix data, enters the COMPUTING state and simultaneously wakes up the VDPB microinstruction in the MSQ that matches this gid with the MSP gid.
[0101] (3) Computing status
[0102] In the COMPUTING state, the MSP sends a matrix data request required to execute the vdpb microinstruction and counts the number of times the vdpb microinstruction is executed. When the vdpb count reaches the total number of vdpb microinstructions decomposed by the MatrixMad instruction, the MatrixMad instruction is completed, the MSP updates gid (gid+2) and re-enters the IDLE state.
[0103] Combination Figure 5 As shown, in some embodiments of this application, the VDPB array may include multiple vector dot product operation units that support concurrent computation of multi-channel vector data. Each vector dot product operation unit includes a two-level computational structure: a first-level computational structure for calculating the dot product of multi-channel vector data to obtain a vector dot product result; and a second-level computational structure for accumulating the vector dot product results of the multi-channel vector data and outputting the accumulated result to a matrix physical register group partitioned from the vector physical register group. Thus, through this hierarchical structure, the dot product operation function can be implemented with relatively low hardware overhead.
[0104] Combination Figure 7 As shown, in some embodiments of this application, the VDPB array can consist of 8 VDPB pipelines, each VDPB pipeline containing 4 VDPB4x8 arithmetic units to compute the two input vectors ([a mThe dot product of [b0, b1, b2, b3, b4, b5, b6, b7] is calculated and output to a corresponding element of matrix C [C0, C1, C2, C3, C4, C5, C6, C7]. In this embodiment, the vector dot integration is performed in two steps: first, the VDPB4x8 dot product unit calculates the INT8 data dot product between the four channels; second, the aforementioned four VDPB4x8 dot product results are accumulated and output. One row vector of the result matrix C is output each cycle. For an 8x8 matrix, a complete matrix multiplication operation requires 8 clock cycles; for a 16x16 matrix, it requires 16 clock cycles.
[0105] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.
[0106] This application also provides a processor architecture that includes the matrix operation implementation device described above. The processor architecture may include, but is not limited to, a microprocessor architecture.
[0107] This application also provides a chip that includes the matrix operation implementation device described above, that is, the chip architecture of the chip includes the processor architecture described above.
[0108] Based on the matrix operation implementation apparatus described above, this application embodiment also provides a matrix operation implementation method, see reference. Figure 8 As shown, in some embodiments of this application, the matrix operation implementation method may include the following steps:
[0109] Step 801: Receive matrix operation instructions.
[0110] Step 802: Load the matrix data corresponding to the matrix operation instruction from the vector physical register group into the matrix buffer structure.
[0111] Step 803: Read the matrix data from the matrix buffer structure and provide it to the vector dot product array to perform matrix operations.
[0112] In some embodiments of the matrix operation implementation method of this application, the matrix buffer structure includes a matrix eraser / writer. After receiving the matrix operation instruction, it may further include: decomposing the matrix operation instruction into at least one loading microinstruction and at least one calculation microinstruction to adapt to the matrix eraser / writer.
[0113] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations that can be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).
[0114] Embodiments of this application also provide a computer device. For example... Figure 9 As shown, in some embodiments of this application, the computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which may implement one or more hardware threads. The computer device 902 may also include any memory 906 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 906 and can run on the processor 904. When the computer program is run by the processor 904, it can execute instructions for the matrix operation implementation method described in any of the above embodiments. Non-limitingly, for example, the memory 906 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes associated instructions stored in any memory or combination of memories, the computer device 902 can perform any operation of the associated instructions. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.
[0115] Computer device 902 may also include an input / output interface 910 (I / O) for receiving various inputs (via input device 912) and providing various outputs (via output device 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output interface 910 (I / O), input device 912, and output device 914 may be omitted, and the device may function solely as a computer device within a network. Computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.
[0116] Communication link 922 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.
[0117] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), computer-readable storage media, and computer program products according to some embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0119] These computer program instructions may also be loaded onto a computer or other programmable data processor, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0120] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0121] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0122] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this application, computer-readable media does not include transient media, such as modulated data signals and carrier waves.
[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0124] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0125] It should also be understood that, in the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.
[0126] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the method embodiments are basically similar to the device embodiments, so the description is relatively simple; relevant parts can be referred to the description of the device embodiments.
[0127] In the description of this application, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this application, as well as the features of different embodiments or examples.
[0128] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.
Claims
1. A matrix operation implementation device, characterized in that, The device is part of a central processing unit core, and the device includes: Vector physical register set; and, Matrix operation unit; The matrix operation unit includes: Matrix buffer structure; Vector dot product arrays; and, The control unit is used to load the matrix data corresponding to the matrix operation instruction from the vector physical register group into the matrix buffer structure, read the matrix data from the matrix buffer structure and provide it to the vector dot product array to perform matrix operations.
2. The matrix operation implementation apparatus as described in claim 1, characterized in that, The device is part of the vector execution unit of the central processing unit core.
3. The matrix operation implementation apparatus as described in claim 1, characterized in that, The matrix buffer structure includes a matrix eraser / write pad.
4. The matrix operation implementation apparatus as described in claim 3, characterized in that, The control unit is also used for: The matrix operation instructions are broken down into at least one loading microinstruction and at least one calculation microinstruction to adapt to the matrix erasing and writing board.
5. The matrix operation implementation apparatus as described in claim 4, characterized in that, The number of matrix erasing plates is at least two, and the control unit is used to enable at least two matrix erasing plates to participate in matrix data loading and matrix data operation concurrently and alternately.
6. The matrix operation implementation apparatus as described in claim 4, characterized in that, The control unit includes a matrix multiplication controller, which comprises: The matrix instruction queue is used to buffer received matrix multiplication operation instructions; The micro-instruction scheduling queue is used to store the disassembled micro-instructions; A finite state machine controller is used to schedule the microinstructions and manage the state of the matrix eraser / write pad.
7. The matrix operation implementation apparatus as described in claim 1, characterized in that, The vector dot product array contains multiple vector dot product operation units that support concurrent operations on multi-channel vector data.
8. The matrix operation implementation apparatus as described in claim 7, characterized in that, Each of the vector dot product operation units comprises a two-level computational structure, the two-level computational structure including: The first-level computational structure is used to calculate the dot product of multi-channel vector data and obtain the vector dot product result. The second-level computational structure is used to accumulate the vector dot product results of the multi-channel vector data and output the accumulated results to the matrix physical register group that is separated from the vector physical register group.
9. The matrix operation implementation apparatus as described in claim 6, characterized in that, The finite state machine controller contains a list of idle global identifiers. The number of bits in the global identifiers in the list covers the index of all different matrix multiplication operations to be executed in the matrix instruction queue and the microinstruction scheduling queue. The target bit of the global identifier indicates the matrix eraser associated with it.
10. The matrix operation implementation apparatus as described in claim 9, characterized in that, The matrix multiplication controller is also used for: When the matrix instruction queue is not full and there is a free global identifier in the list, the matrix multiplication operation instruction output by the central processing unit kernel distribution unit is received, a global identifier is allocated for the matrix multiplication operation instruction from the list, and the matrix multiplication operation instruction associated with the global identifier is written to the tail of the matrix instruction queue.
11. The matrix operation implementation apparatus as described in claim 9, characterized in that, The matrix multiplication controller is also used for: When there is an idle entry in the microinstruction scheduling queue, the matrix multiplication operation instruction located at the head of the matrix instruction queue is decomposed into at least one loading microinstruction and at least one calculation microinstruction. The loading microinstruction, the calculation microinstruction and their respective associated global identifiers are written together into the corresponding idle entry in the microinstruction scheduling queue. When all the microinstructions after the matrix multiplication operation instruction is decomposed are written into the idle entry in the microinstruction scheduling queue, the entry occupied by the matrix multiplication operation instruction in the matrix instruction queue is released.
12. The matrix operation implementation apparatus as described in claim 9, characterized in that, The matrix multiplication controller is also used for: The finite state machine controller listens for a first ready signal and a second ready signal corresponding to the loaded microinstruction. The first ready signal indicates that the matrix physical register group corresponding to the source operand of the loaded microinstruction is ready. The second ready signal indicates that the target matrix eraser / write plate corresponding to the destination operand of the loaded microinstruction is ready. When the first ready signal and the second ready signal corresponding to the loaded microinstruction are received, and the global identifier of the target matrix eraser / write plate of the loaded microinstruction matches its own global identifier, the finite state machine controller outputs a transmit signal for the loaded microinstruction to the microinstruction scheduling queue, so as to retrieve the loaded microinstruction from the microinstruction scheduling queue and dispatch it to the corresponding loading execution unit for execution. After the loaded microinstruction is executed, the microinstruction scheduling queue entry it occupies is released.
13. The matrix operation implementation apparatus as described in claim 9, characterized in that, The matrix multiplication controller is also used for: The finite state machine controller listens for the third and fourth ready signals corresponding to the computation microinstruction. The third ready signal indicates that the source operand register of the computation microinstruction is ready. The fourth ready signal indicates that the vector dot product array corresponding to the destination operand of the computation microinstruction is ready. When the third and fourth ready signals corresponding to the computation microinstruction are received, and the global identifier of the source operand register of the computation microinstruction matches its own global identifier, the finite state machine controller outputs a transmit signal for the computation microinstruction to the microinstruction scheduling queue, so as to retrieve the computation microinstruction from the microinstruction scheduling queue and dispatch it to the corresponding computation execution unit for execution. After the computation microinstruction is executed, the microinstruction scheduling queue entry it occupies is released.
14. A processor architecture, characterized in that, The processor architecture includes the matrix operation implementation apparatus according to any one of claims 1-13.
15. A chip, characterized in that, The chip includes the matrix operation implementation device according to any one of claims 1-13.
16. A method for implementing matrix operations, characterized in that, Applied to the apparatus according to any one of claims 1-13, the method comprises: Receive matrix operation instructions; Load the matrix data corresponding to the matrix operation instruction from the vector physical register group into the matrix buffer structure; The matrix data is read from the matrix buffer structure and provided to the vector dot product array for matrix operations.
17. The matrix operation implementation method as described in claim 16, characterized in that, The matrix buffer structure includes a matrix eraser / writer, and after receiving the matrix operation instruction, it further includes: The matrix operation instructions are broken down into at least one loading microinstruction and at least one calculation microinstruction to adapt to the matrix erasing and writing board.
18. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to claim 16 or 17.