Matrix multiplication and vector calculation implementation fusion method for heterogeneous reconfigurable architecture

Abstract

The invention relates to a fusion method for realizing matrix multiplication and vector calculation for a heterogeneous reconfigurable architecture. The method comprises the steps that a matrix multiplication unit operates a matrix multiplication operator based on input data; under the condition that the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to a vector operation unit; and the vector operation unit takes the partial output result as an input running vector operation operator until a target output result is obtained. By adopting the method, the resource utilization rate can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer architecture technology, and in particular to a method for integrating matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture. Background Art

[0002] In heterogeneous reconfigurable architectures, current reasoning techniques mainly use operator-by-operator computational reasoning methods. Specifically, the computation process of the previous operator is affected by the memory limitations of the chip where the computation unit is located, and data input, computation, and output are usually performed in blocks. When the next operator is computed, this input, computation, and output process is repeated.

[0003] However, the above method has the problem of low resource utilization efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a fusion method for matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture that can improve resource utilization in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for integrating matrix multiplication and vector computing for a heterogeneous reconfigurable architecture, including:

[0006] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0007] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0008] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0009] In one embodiment, the transmitting of part of the output result to the vector operation unit includes:

[0010] The matrix multiplication unit sends part of the output results to the shared memory for storage processing;

[0011] The vector operation unit obtains part of the output results from the shared memory.

[0012] In one embodiment, the matrix multiplication unit includes a plurality of matrix multiplication processing subunits, and the process of determining the partial output results includes:

[0013] Each matrix multiplication processing subunit executes the matrix multiplication operator in different time periods to obtain multiple candidate output results;

[0014] Multiple candidate output results are accumulated to obtain partial output results.

[0015] In one of the embodiments, the vector operation unit is integrated with an activation function mapping function.

[0016] In a second aspect, the present application also provides a fusion device for matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture, the device comprising: a matrix multiplication unit and a vector operation unit, the matrix multiplication unit and the vector operation unit are communicatively connected;

[0017] A matrix multiplication unit, used for running a matrix multiplication operator based on input data, and transmitting a partial output result obtained based on the input data to a vector operation unit;

[0018] The vector operation unit is used to run the vector operation operator with part of the output result as input until the target output result is obtained.

[0019] In one embodiment, the apparatus further comprises a shared memory, and the shared memory is respectively communicatively connected with the matrix multiplication unit and the vector operation unit;

[0020] Matrix multiplication unit, used to send part of the output results to shared memory for storage processing;

[0021] Vector operation unit, used to obtain part of the output results from the shared memory.

[0022] In one of the embodiments, the matrix multiplication unit includes a plurality of matrix multiplication processing subunits;

[0023] Each matrix multiplication processing subunit is used to execute the matrix multiplication operator in different time periods to obtain multiple candidate output results; and accumulate the multiple candidate output results to obtain a partial output result.

[0024] In one embodiment, the vector operation unit includes a basic vector operation subunit, an exponential vector operation subunit and a logarithmic vector operation subunit;

[0025] A basic vector operation subunit, used for performing basic operations on part of the output results based on the vector operation operator;

[0026] An exponential vector operator unit, used for performing exponential operations on part of the output results based on the vector operation operator;

[0027] The logarithmic vector operator unit is used to perform logarithmic operations on part of the output results based on the vector operation operator.

[0028] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0029] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0030] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0031] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0033] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0034] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0035] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0036] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0037] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0038] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0039] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0040] In the above-mentioned fusion method of matrix multiplication and vector computing for heterogeneous reconfigurable architecture, the matrix multiplication unit first runs the matrix multiplication operator based on the input data; when the matrix multiplication unit obtains partial output results based on the input data, the partial output results are transmitted to the vector operation unit; the vector operation unit uses the partial output results as input to run the vector operation operator until the target output result is obtained; in this method, while the matrix multiplication unit generates partial output results, the vector operation unit can start processing these partial output results, thereby realizing the continuity and parallelism of the data flow. In addition, the computing structure with clear division of labor allows different units to play their respective advantages at different time points, thereby improving the utilization of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] FIG1 is a diagram showing the internal structure of a computer device in one embodiment;

[0043] Figure 2 A flowchart of a method for integrating matrix multiplication and vector calculation for a heterogeneous reconfigurable architecture in one embodiment;

[0044] Figure 3 It is a flowchart of a fusion method for matrix multiplication and vector calculation implementation for a heterogeneous reconfigurable architecture in another embodiment;

[0045] Figure 4 It is a flowchart of a fusion method for matrix multiplication and vector calculation implementation for a heterogeneous reconfigurable architecture in another embodiment;

[0046] FIG5 is a schematic diagram of calculation of a partial output result determination process in another embodiment;

[0047] FIG6 is a schematic diagram of the structure of a fusion device for implementing matrix multiplication and vector calculation for heterogeneous reconfigurable architecture in one embodiment;

[0048] Figure 7 It is a schematic diagram of the structure of a fusion device for realizing matrix multiplication and vector calculation for heterogeneous reconfigurable architecture in another embodiment;

[0049] FIG8 is a schematic diagram of the structure of a fusion device for implementing matrix multiplication and vector calculation for heterogeneous reconfigurable architecture in another embodiment;

[0050] FIG9 is a schematic diagram of the structure of a fusion device for realizing matrix multiplication and vector calculation for heterogeneous reconfigurable architecture in another embodiment;

[0051] Fig.10 is a schematic diagram of the internal structure of a vector operation unit in another embodiment;

[0052] FIG11 is a schematic diagram of the structure of a vector operation unit in another embodiment;

[0053] FIG12 is a schematic diagram of the structure of a vector operation unit in yet another embodiment;

[0054] FIG13 is a schematic diagram of calculating an S-shaped activation function sigmoid in another embodiment;

[0055] FIG. 14 is a schematic diagram of calculating a hyperbolic tangent (Tanh) activation function in another embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] In heterogeneous reconfigurable architectures, current reasoning technologies generally use operator-by-operator computational reasoning methods. This method is characterized by the fact that the computation process of each operator is limited by the memory on the chip where it is located, and usually requires data to be divided into blocks for input, calculation, and output. When an operator completes the calculation, the execution of the next operator must repeat the similar input, calculation, and output process, which not only increases latency but may also lead to a waste of resources.

[0058] The operator-by-operator processing method often fails to fully utilize the computing resources within the chip, especially when processing large amounts of data, the efficiency of memory bandwidth and computing units may be greatly reduced. In addition, data transmission and frequent switching between operators will also increase additional energy consumption and latency, further affecting the performance of the overall architecture.

[0059] Therefore, in response to this limitation, the present application provides a fusion method for matrix multiplication and vector calculation implementation for a heterogeneous reconfigurable architecture to solve the above problem.

[0060] In an exemplary embodiment, a computer device is provided, which may be oriented to a heterogeneous reconfigurable architecture, and its internal structure diagram may be shown in FIG1. ​​The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in the fusion process of matrix multiplication and vector calculation for heterogeneous reconfigurable architecture. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fusion method for matrix multiplication and vector calculation for heterogeneous reconfigurable architecture is implemented.

[0061] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0062] In an exemplary embodiment, Figure 2 As shown, a fusion method for matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture is provided, and the method is applied to the computer device in FIG. 1 as an example for explanation, including the following steps 201 to 203. Among them:

[0063] In step 201, a matrix multiplication unit runs a matrix multiplication operator based on input data.

[0064] The input data may include an input matrix and a weight matrix.

[0065] The matrix multiplication unit is responsible for performing matrix multiplication operations. In calculations, the matrix multiplication unit multiplies the input matrix and generates a new matrix as output.

[0066] The matrix multiplication operator is an algorithm or operator that performs matrix multiplication operations by multiplying the elements one by one and accumulating the results to generate the product matrix.

[0067] In the embodiment of the present application, the matrix multiplication unit first receives input data, formats and verifies the input data so that their dimensions and data types match for efficient calculation. Next, the matrix multiplication operator begins to perform the actual matrix multiplication operation.

[0068] Step 202: When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit.

[0069] The vector operation unit is a module responsible for executing vector-related computing tasks. It can perform various mathematical operations such as addition, scaling, and dot product.

[0070] A vector operator is an algorithm or operator that performs specific vector operations.

[0071] In an embodiment of the present application, the matrix multiplication operator multiplies the input data and accumulates them to generate partial output results. During this process, the matrix multiplication operator dynamically manages its internal memory so that the input data can be effectively stored in its on-chip memory. After the partial output results are calculated, the partial output results will be transmitted to the subsequent vector operation unit for further processing.

[0072] After receiving the partial output results, the vector operation unit will perform necessary checks to ensure the integrity and accuracy of the data. Next, the vector operation unit will use these partial output results as input to perform related vector operations, such as addition, scaling or dot product operations.

[0073] In some embodiments, in order to improve the accuracy and effectiveness of calculations, the matrix multiplication unit also monitors the calculation status in real time and implements error detection and correction mechanisms.

[0074] Step 203: the vector operation unit uses part of the output result as input to run the vector operation operator until the target output result is obtained.

[0075] In the embodiment of the present application, the vector operation unit performs format verification on the input data, that is, part of the output result, to ensure that it meets the requirements of the operation operator. Subsequently, the vector operation unit initializes the operation operator and loads the input data into an appropriate storage area for efficient calculation. Next, the vector operation operator starts to perform specific operation operations.

[0076] After each operation, the matrix multiplication unit generates new partial output results, which are used again as inputs for subsequent operations of the vector operation unit. This process will continue to cycle until the predetermined target output conditions are met or the set accuracy requirements are reached. Finally, after a series of operations and adjustments, the vector operation unit will successfully generate the target output results and complete the entire calculation process.

[0077] In the above-mentioned fusion method of matrix multiplication and vector computing for heterogeneous reconfigurable architecture, the matrix multiplication unit first runs the matrix multiplication operator based on the input data; when the matrix multiplication unit obtains partial output results based on the input data, the partial output results are transmitted to the vector operation unit; the vector operation unit uses the partial output results as input to run the vector operation operator until the target output result is obtained; in this method, while the matrix multiplication unit generates partial output results, the vector operation unit can start processing these partial output results, thereby realizing the continuity and parallelism of the data flow. In addition, the computing structure with clear division of labor allows different units to play their respective advantages at different time points, thereby improving the utilization of computing resources.

[0078] In an exemplary embodiment, Figure 3 As shown, in the above embodiment, "transmitting part of the output results to the vector operation unit" includes steps 301 to 302. Among them:

[0079] Step 301: The matrix multiplication unit sends part of the output results to the shared memory for storage processing.

[0080] In the embodiment of the present application, first, the matrix multiplication unit needs to format part of the output results to ensure that the data meets the storage requirements of the shared memory, such as confirming the data type, dimension and arrangement order, so as to facilitate subsequent reading and processing.

[0081] Next, the matrix multiplication unit writes part of the output results to the shared memory. After the data is successfully written to the shared memory, the matrix multiplication unit records the storage address and status so that subsequent operations can accurately access the data. At the same time, the access control of the shared memory can also be updated so that other computing units can read these partial output results in time when needed.

[0082] Step 302: The vector operation unit obtains part of the output result from the shared memory.

[0083] In an embodiment of the present application, first, the vector operation unit needs to confirm the availability and format of the data in the shared memory. Then, the vector operation unit will initiate a read request to extract a portion of the output results from the shared memory. After successfully reading the data, the vector operation unit will load the partial output results into the internal memory and prepare them for subsequent operations. Subsequently, the vector operation unit will use these partial output results to perform corresponding operations, such as addition, scaling or other linear transformations, to gradually approach the final target output result.

[0084] In the above embodiment, shared memory allows multiple computing units to read and write data simultaneously, thereby improving the ability of parallel processing. After the matrix multiplication is completed, the vector operation unit can quickly start the calculation to maximize resource utilization. In addition, by centrally managing some output results, shared memory reduces the complexity of data management.

[0085] In some embodiments, the vector operation unit may also perform a data integrity check to confirm that the extracted data has not been damaged or altered during transmission.

[0086] In an exemplary embodiment, Figure 4 As shown, the matrix multiplication unit includes a plurality of matrix multiplication processing sub-units, and the "determination process of part of the output result" in the above embodiment includes steps 401 to 402. Among them:

[0087] Step 401: Each matrix multiplication processing subunit executes a matrix multiplication operator in different time periods to obtain a plurality of candidate output results.

[0088] In the embodiment of the present application, in the process of each matrix multiplication processing subunit executing the matrix multiplication operator in different time periods, first, each matrix multiplication processing subunit will independently start the operation according to the preset input data and weight matrix. When each matrix multiplication processing subunit executes the matrix multiplication operator, it will gradually complete its own calculation task and generate its own candidate output result. In this stage, the matrix multiplication processing subunit needs to ensure the accuracy and efficiency of the operation, and may adopt parallel computing to shorten the calculation time.

[0089] Step 402: Accumulate multiple candidate output results to obtain partial output results.

[0090] In the embodiment of the present application, after completing the respective calculations, each matrix multiplication processing subunit will generate multiple candidate output results. Next, these candidate output results are summarized and accumulated. In this process, the candidate output results first need to be format-verified and integrated to ensure that their dimensions and data types are consistent. Then, all candidate output results are accumulated item by item and summarized into partial output results.

[0091] In some embodiments, in the matrix multiplication unit, the register of each matrix multiplication processing subunit is used to store the partial sum of the output matrix, so that each matrix multiplication processing subunit can process different parts of the matrix multiplication in parallel. In the process of performing matrix multiplication, all matrix multiplication processing subunits work independently, perform the calculation tasks assigned to themselves, and accumulate the calculation results into their own registers.

[0092] Since the calculation time of each matrix multiplication processing subunit may be different, the time to complete the output accumulation is also different. For example, as shown in Figure 5, the matrix multiplication processing subunit located in the lower left corner of the output matrix is ​​usually the first to complete its calculation because it processes the earlier part of the input matrix. When the matrix multiplication processing subunit completes the output accumulation, in the next clock cycle, the matrix multiplication processing subunits to its right and above will gradually complete their respective output accumulations. In this way, the calculation completion time of each matrix multiplication processing subunit forms a dynamic time series, which gradually promotes the formation of the entire matrix output.

[0093] This parallel computing strategy significantly improves the efficiency of matrix multiplication. In each clock cycle, the entire output matrix can complete up to 32 matrix multiplication outputs, thanks to the collaborative work of multiple matrix multiplication processing subunits. Through parallel processing, the entire calculation process not only speeds up the output speed, but also ensures support for large-scale matrix calculations, greatly improving performance.

[0094] In the above embodiment, by assigning the calculation task of matrix multiplication to multiple matrix multiplication processing subunits and executing them in parallel in different time periods, it is possible to more effectively utilize the computing resources, thereby speeding up the overall computing speed. Multiple matrix multiplication processing subunits perform tasks simultaneously, so that the calculation process can be carried out in parallel in multiple dimensions. This parallel processing can significantly reduce the calculation delay and improve the response speed. In addition, each matrix multiplication processing subunit only needs to process a part of the matrix multiplication, which reduces the calculation burden of each matrix multiplication processing unit, so that each matrix multiplication processing subunit completes its task in a shorter time, which helps to improve the processing efficiency and stability of the unit.

[0095] In an exemplary embodiment, an activation function mapping function is integrated on the vector operation unit.

[0096] In an embodiment of the present application, the activation function mapping allows the vector operation unit to apply a variety of activation functions when processing the hidden layer and output layer in the neural network, such as the Rectified Linear Unit (ReLU), Sigmoid, and Tanh. These activation functions are responsible for introducing nonlinear characteristics in the neural network, enabling the network to learn complex data patterns and features. After integrating the activation function mapping, the vector operation unit can calculate the activation value in real time during the inference process, eliminating the need for additional hardware or software support in the past.

[0097] In the above embodiment, by integrating activation function mapping, the vector operation unit can significantly improve performance and response speed when performing complex deep learning tasks, while reducing computing delays and optimizing resource utilization efficiency. The activation function mapping function integrated on the vector operation unit enables the vector operation unit to not only perform conventional mathematical calculations, but also support key operations in neural network reasoning. By implementing activation function mapping, the vector operation unit can directly perform nonlinear transformations on input data, thereby enhancing expression capabilities and performance.

[0098] In summary of all the above embodiments, the present application further provides a method for integrating matrix multiplication and vector calculation implementation for a heterogeneous reconfigurable architecture, the method comprising:

[0099] Step 1: Each matrix multiplication processing subunit executes a matrix multiplication operator in different time periods to obtain multiple candidate output results.

[0100] Step 2: Accumulate multiple candidate output results to obtain partial output results.

[0101] Step 3: When the matrix multiplication unit obtains a partial output result based on the input data, the matrix multiplication unit sends the partial output result to the shared memory for storage processing.

[0102] Step 4: The vector operation unit obtains part of the output results from the shared memory; the vector operation unit is integrated with an activation function mapping function.

[0103] Step 5: The vector operation unit uses part of the output result as input to run the vector operation operator until the target output result is obtained.

[0104] In an exemplary embodiment, the vector operation unit is integrated with an activation function mapping function.

[0105] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a matrix multiplication and vector calculation implementation device for heterogeneous reconfigurable architecture for implementing the above-mentioned matrix multiplication and vector calculation implementation method for heterogeneous reconfigurable architecture. The implementation solution for solving the problem provided by the device is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more matrix multiplication and vector calculation implementation devices for heterogeneous reconfigurable architecture provided below can refer to the limitations of the matrix multiplication and vector calculation implementation method for heterogeneous reconfigurable architecture above, and will not be repeated here.

[0107] In an exemplary embodiment, as shown in FIG6 , a matrix multiplication and vector calculation implementation device for a heterogeneous reconfigurable architecture is provided, comprising a matrix multiplication unit 01 and a vector operation unit 02 , wherein the matrix multiplication unit 01 is communicatively connected with the vector operation unit 02 ;

[0108] The matrix multiplication unit 01 is used to run the matrix multiplication operator based on the input data, and transmit part of the output result obtained based on the input data to the vector operation unit 02;

[0109] The vector operation unit 02 is used to run the vector operation operator using part of the output result as input until the target output result is obtained.

[0110] The main function of the matrix multiplication unit 01 is to run the matrix multiplication operator based on the input data. The matrix multiplication operator can perform multiplication operations on multiple input matrices to generate new matrix outputs. When performing calculations, the matrix multiplication unit 01 will preprocess and format the data according to the preset operation strategy to ensure the accuracy and consistency of the input data. After the matrix operation is completed, the matrix multiplication unit 01 will transmit the generated partial output results to the vector operation unit 02 through the communication interface in a timely manner for subsequent processing.

[0111] The vector operation unit 02 receives part of the output results from the matrix multiplication unit 01 and uses these results as input to run the vector operation operator. The vector operation unit 02 can perform a series of vector-related calculation tasks, such as addition, scaling, inner product, etc., until the final target output result is generated.

[0112] In the above embodiment, while the matrix multiplication unit generates partial output results, the vector operation unit can start processing these partial output results, thereby achieving the continuity and parallelism of the data flow. In addition, the computing structure with clear division of labor allows different units to play their respective advantages at different time points, thereby improving the utilization rate of computing resources.

[0113] In an exemplary embodiment, Figure 7 As shown, the above device also includes a shared memory 03, and the shared memory 03 is respectively connected to the matrix multiplication unit 01 and the vector operation unit 02 for communication;

[0114] The matrix multiplication unit 01 is used to send part of the output results to the shared memory 03 for storage processing; the vector operation unit 02 is used to obtain part of the output results from the shared memory 03.

[0115] The device also includes a shared memory 03, which is connected to the matrix multiplication unit 01 and the vector operation unit 02 through a high-speed communication interface, respectively, to achieve efficient storage and sharing of data. The shared memory 03 enables the matrix multiplication unit 01 and the vector operation unit 02 to quickly exchange data between different calculation stages, reducing the delay caused by frequent access to the main memory.

[0116] After performing the matrix multiplication operation and obtaining partial output results, the matrix multiplication unit 01 first formats and arranges these results to ensure that the data can be stored in the shared memory 03 in an optimal manner. Then, the matrix multiplication unit 01 sends the partial output results to the shared memory 03 through the shared memory 03 interface for storage processing.

[0117] The vector operation unit 02 is responsible for obtaining the required partial output results from the shared memory 03. In this process, the vector operation unit 02 first checks the data status in the shared memory 03 to ensure that the required data has been successfully stored. Then, it extracts the partial output results into the internal memory through an efficient reading mechanism for subsequent vector operations.

[0118] In the above embodiment, shared memory allows multiple computing units to read and write data simultaneously, thereby improving the ability of parallel processing. After the matrix multiplication is completed, the vector operation unit can quickly start the calculation to maximize resource utilization. In addition, by centrally managing some output results, shared memory reduces the complexity of data management.

[0119] In an exemplary embodiment, as shown in FIG8 , the matrix multiplication unit 01 includes a plurality of matrix multiplication processing subunits 011 ;

[0120] Each matrix multiplication processing subunit 011 is used to execute the matrix multiplication operator in different time periods to obtain multiple candidate output results; and accumulate the multiple candidate output results to obtain a partial output result.

[0121] The matrix multiplication unit 01 includes a plurality of matrix multiplication processing subunits 011, which together form a parallel processing architecture to improve the efficiency and performance of matrix multiplication operations. Each matrix multiplication processing subunit 011 can independently execute a matrix multiplication operator, thereby processing respective input data in different time periods and generating multiple candidate output results.

[0122] Each matrix multiplication processing subunit 011 receives a specific input matrix and performs calculations according to a preset algorithm. When processing a large matrix, each matrix multiplication processing subunit 011 stores the generated candidate output results in a temporary storage area after the calculation is completed for subsequent summary processing.

[0123] After obtaining all candidate output results, the matrix multiplication unit 01 will perform accumulation processing on these candidate output results. This process first requires format verification and integration of the candidate output results to ensure that their dimensions, data types, and arrangement order are consistent. Next, all candidate output results are accumulated item by item and summarized into partial output results.

[0124] In the above embodiment, by assigning the calculation task of matrix multiplication to multiple matrix multiplication processing subunits and executing them in parallel in different time periods, it is possible to more effectively utilize the computing resources, thereby speeding up the overall computing speed. Multiple matrix multiplication processing subunits perform tasks simultaneously, so that the calculation process can be carried out in parallel in multiple dimensions. This parallel processing can significantly reduce the calculation delay and improve the response speed. In addition, each matrix multiplication processing subunit only needs to process a part of the matrix multiplication, which reduces the calculation burden of each matrix multiplication processing unit, so that each matrix multiplication processing subunit completes its task in a shorter time, which helps to improve the processing efficiency and stability of the unit.

[0125] In an exemplary embodiment, as shown in FIG9 , the vector operation unit 02 includes a basic vector operation subunit 021 , an exponential vector operation subunit 022 , and a logarithmic vector operation subunit 023 ;

[0126] The basic vector operator unit 021 is used to perform basic operations on part of the output results based on the vector operator; the exponential vector operator unit 022 is used to perform exponential operations on part of the output results based on the vector operator; the logarithmic vector operator unit 023 is used to perform logarithmic operations on part of the output results based on the vector operator.

[0127] The basic vector operation subunit 021 is responsible for performing basic mathematical operations on some output results. This subunit can perform a variety of basic operations, including vector addition, subtraction, scalar multiplication and dot product operations. These basic operations are the basis of complex algorithms and are usually used for data preprocessing and feature extraction.

[0128] When performing an operation, the basic vector operation subunit 021 first receives part of the output results from the shared memory 03 or the matrix multiplication unit 01. According to the selected vector operation operator, the basic vector operation subunit 021 will perform element-by-element calculations on the input data to generate new output results. Through efficient parallel processing, the basic operation can complete a large number of computing tasks in a short time, thereby supporting real-time data analysis and processing.

[0129] The exponential vector operation subunit 022 is used to perform exponential operation on part of the output results, and can quickly calculate the exponential value of the input data, such as the exponent of other bases.

[0130] In the actual operation process, the exponential vector operation subunit 022 first receives a part of the output result from the basic vector operation subunit 021 or the shared memory 03. Based on the selected vector operation operator, the exponential vector operation subunit 022 will perform an exponential operation.

[0131] The logarithmic vector operation subunit 023 is used to perform logarithmic operations on part of the output results, and can efficiently calculate the logarithmic value of the input data, such as the natural logarithm or the logarithm with base 10.

[0132] When performing an operation, the logarithmic vector operator 023 first receives a portion of the output results from the basic vector operator 021 or the shared memory 03. Based on the selected vector operator, the logarithmic vector operator 023 will perform a logarithmic operation on the input data.

[0133] In some embodiments, Fig.10 As shown, the vector operation unit includes a storage unit and a processing unit. Among them, there are 64 PEs (Processing Element) in the vector operation unit PEA, supporting FP32, Int32 and Int16 precision 4 lines. LSU (Load Store Unit), up to 32 channels of simultaneous reading and writing, provides calculation data for PE, ConfigControl (32-bit control network interface) and Data Control (256-bit data network interface), respectively provide configuration and data for PE. DMA (Direct Memory Access) transports configuration or data, supports Scatter and Gather functions, and transports image data in DDR (Double Data Rate) to a specific internal address according to the configured format. The 32-bit-control network interface and the 256-bit data network interface are connected to the 32-bit-control network and the 256-bit data network, respectively.

[0134] In Figure 11, PEA represents a vector operation unit, and PE represents a processing unit, namely a basic vector operation subunit, an exponential vector operation subunit, and a logarithmic vector operation subunit. In an embodiment of the present application, four vector operation units can be designed to support efficient data processing and calculation. These vector operation units divide the output data into four parts according to the channel, where the data of channels 0 to 7 are designated to be processed by the first vector operation unit. This division not only improves the parallel processing capability, but also ensures that each vector operation unit can focus on a specific data set, reducing computing conflicts and resource competition.

[0135] Further subdividing the structure of each vector operation unit, each vector operation unit consists of 8 columns of processing units, and each column contains 8 processing units. In each cycle, these processing units need to be able to process a matrix multiplication output and complete the activation function calculation, which means that they need to perform not only basic addition, subtraction, multiplication and division operations, but also support more complex nonlinear activation functions such as ReLU or Sigmoid.

[0136] In each column of 8 processing units, as shown in Figure 12, in Figure 12, PEA represents vector operation unit, PE represents processing unit, namely basic vector operation subunit, exponential vector operation subunit and logarithmic vector operation subunit. PEA includes three processing units that can perform ordinary calculations. These processing units can perform basic arithmetic operations (addition, subtraction, multiplication and division), comparison and inversion operations to ensure that basic computing needs are met. In addition, there are three special calculation processing units, which have the ability to perform specific functions. Two of the special calculation processing units can perform exponential operations, which are suitable for processing tasks that require exponential transformations, such as activation functions or data preprocessing in deep learning. The other special calculation processing unit focuses on logarithmic operations.

[0137] In some embodiments, different activation functions can be implemented through different PE mappings and combinations.

[0138] As shown in Figure 13, in Figure 13, MUL represents a multiplier, SFU represents a special function unit, ADD represents an adder, DIV represents a divider, and router represents a router. In Figure 13, by mapping the five PEs, the calculation of the sigmoid activation function for the input can be completed. Assuming that the time for PE to perform various calculations is one cycle, the calculation process is as follows:

[0139] In cycle 0, part of the output result of the matrix multiplication unit is passed to the first PE, where MUL calculation is performed to obtain -X, which is then passed to the next PE.

[0140] Cycle 1, the second PE (SFU) takes -X as input to calculate EXP and obtains , and pass it to the next PE;

[0141] Cycle2, the third PE adds it to the constant 1 and gets +1 and pass it to the next PE;

[0142] Cycle3, the fourth PE uses it as a divisor and divides it by 1 to get , and pass it to the next PE;

[0143] Cycle 4, the fifth PE acts as the router of the previous PE. , the calculation result of the sigmoid activation function is stored in the shared memory.

[0144] As shown in Figure 14, EXP stands for Exponent Unit and SUB stands for Subtractor. Figure 14 can complete the PE mapping of the Tanh activation function. Because its calculation requires the reuse of intermediate calculations, shared memory is used as a temporary storage point. It is still assumed that all calculation operations only require one cycle. The calculation process is as follows:

[0145] In cycle 0, part of the output results of the matrix multiplication unit are passed to the first SFU, and the calculation is , and save it to shared memory.

[0146] Cycle1 is passed to the third PE, which performs MUL calculation to get -X, which is passed to the second SFU;

[0147] Cycle2, the second PE (SFU) takes -X as input to calculate EXP and obtains , and pass it to the next PE;

[0148] Cycle3, the fourth PE, gets , and temporarily store it in the PE register;

[0149] Cycle4, the fifth PE, gets , and temporarily store it in the PE register;

[0150] Cycle5, the sixth PE takes the results of the fourth and fifth PE as input and obtains , the calculation result of the tanh activation function is passed to the next PE;

[0151] Cycle 6, the seventh PE is used as the route of the previous PE and calculates the result Store in shared memory.

[0152] In the above embodiment, by integrating the basic vector operation subunit, the exponential vector operation subunit and the logarithmic vector operation subunit into the same vector operation unit, the device can flexibly respond to various computing requirements and realize diversified computing functions.

[0153] Each module in the above-mentioned fusion device for realizing matrix multiplication and vector calculation for heterogeneous reconfigurable architecture can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0154] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0155] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0156] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0157] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0158] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0159] The matrix multiplication unit sends part of the output results to the shared memory for storage processing;

[0160] The vector operation unit obtains part of the output results from the shared memory.

[0161] In one embodiment, the matrix multiplication unit includes a plurality of matrix multiplication processing subunits, and the processor further implements the following steps when executing the computer program:

[0162] Each matrix multiplication processing subunit executes the matrix multiplication operator in different time periods to obtain multiple candidate output results;

[0163] Multiple candidate output results are accumulated to obtain partial output results.

[0164] In one embodiment, the vector operation unit is integrated with an activation function mapping function.

[0165] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0166] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0167] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0168] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] The matrix multiplication unit sends part of the output results to the shared memory for storage processing;

[0171] The vector operation unit obtains part of the output results from the shared memory.

[0172] In one embodiment, the matrix multiplication unit includes a plurality of matrix multiplication processing subunits, and when the computer program is executed by the processor, the following steps are further implemented:

[0173] Each matrix multiplication processing subunit executes the matrix multiplication operator in different time periods to obtain multiple candidate output results;

[0174] Multiple candidate output results are accumulated to obtain partial output results.

[0175] In one embodiment, the vector operation unit is integrated with an activation function mapping function.

[0176] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0177] The matrix multiplication unit operates a matrix multiplication operator based on the input data;

[0178] When the matrix multiplication unit obtains a partial output result based on the input data, the partial output result is transmitted to the vector operation unit;

[0179] The vector operation unit uses part of the output results as input to run the vector operation operator until the target output result is obtained.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0181] The matrix multiplication unit sends part of the output results to the shared memory for storage processing;

[0182] The vector operation unit obtains part of the output results from the shared memory.

[0183] In one embodiment, the matrix multiplication unit includes a plurality of matrix multiplication processing subunits, and when the computer program is executed by the processor, the following steps are further implemented:

[0184] Each matrix multiplication processing subunit executes the matrix multiplication operator in different time periods to obtain multiple candidate output results;

[0185] Multiple candidate output results are accumulated to obtain partial output results.

[0186] In one embodiment, the vector operation unit is integrated with an activation function mapping function.

[0187] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0188] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fusion method for matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture, characterized in that: The method comprises: The matrix multiplication unit operates a matrix multiplication operator based on the input data; When the matrix multiplication unit obtains a partial output result based on the input data, transmitting the partial output result to the vector operation unit; The vector operation unit uses the partial output result as input to run a vector operation operator until a target output result is obtained.

2. The method according to claim 1, characterized in that The transmitting the part of the output result to the vector operation unit comprises: The matrix multiplication unit sends the partial output result to the shared memory for storage processing; The vector operation unit obtains the partial output result from the shared memory.

3. The method according to claim 1, characterized in that The matrix multiplication unit includes a plurality of matrix multiplication processing subunits, and the process of determining the partial output result includes: Each of the matrix multiplication processing subunits executes the matrix multiplication operator in different time periods to obtain multiple candidate output results; A plurality of the candidate output results are accumulated to obtain the partial output result.

4. The method according to any one of claims 1 to 3, characterized in that: The vector operation unit is integrated with an activation function mapping function.

5. A fusion device for matrix multiplication and vector calculation implementation for heterogeneous reconfigurable architecture, characterized in that: The device comprises: a matrix multiplication unit and a vector operation unit, wherein the matrix multiplication unit is communicatively connected with the vector operation unit; The matrix multiplication unit is used to run a matrix multiplication operator based on input data, and transmit a part of the output result obtained based on the input data to the vector operation unit; The vector operation unit is used to run a vector operation operator using the partial output result as input until a target output result is obtained.

6. The device according to claim 5, characterized in that The device further comprises a shared memory, wherein the shared memory is communicatively connected to the matrix multiplication unit and the vector operation unit respectively; The matrix multiplication unit is used to send the partial output results to the shared memory for storage processing; The vector operation unit is used to obtain the part of the output result from the shared memory.

7. The device according to claim 5, characterized in that The matrix multiplication unit includes a plurality of matrix multiplication processing subunits; Each of the matrix multiplication processing subunits is used to execute the matrix multiplication operator in different time periods to obtain multiple candidate output results; and accumulate the multiple candidate output results to obtain the partial output result.

8. The device according to claim 5, characterized in that The vector operation unit includes a basic vector operation subunit, an exponential vector operation subunit and a logarithmic vector operation subunit; The basic vector operation subunit is used to perform basic operations on the partial output results based on the vector operation operator; The exponential vector operator unit is used to perform an exponential operation on the partial output result based on the vector operation operator; The logarithmic vector operator unit is used to perform logarithmic operation on the partial output results based on the vector operation operator.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images