Processing method of matrix multiplication operation in parallel computing hardware and related device

Abstract

The embodiment of the application provides a processing method for matrix multiplication operation in parallel computing hardware and related equipment, wherein a first initial matrix and a second initial matrix are obtained; then, half-precision processing is performed based on a single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; then, a first difference matrix is obtained based on a difference value of the first initial matrix and the first half-precision matrix, and a second difference matrix is obtained based on a difference value of the second initial matrix and the second half-precision matrix; finally, the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain a first single-precision target matrix, so that a single-precision multiplication operation result with higher accuracy is obtained on a hardware device supporting only half-precision multiplication operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a processing method for matrix multiplication in parallel computing hardware and related equipment. BACKGROUND

[0002] In recent years, with the vigorous development of the fields of machine learning and artificial intelligence, the number of matrix multiplication operations has greatly increased, and the development of special hardware for parallel computing has also been greatly accelerated. Consequently, improving the accuracy of matrix multiplication operations has become the most important thing to note.

[0003] In related technologies, for single-precision matrix multiplication, a computing chip supporting single-precision calculation can be used for calculation, but the computing chip supporting single-precision calculation has a high cost. In addition, in the use of a computing chip supporting only half-precision calculation, the single-precision matrix to be multiplied is converted into a half-precision matrix, and then half-precision multiplication is performed, but the accuracy of the result obtained by this scheme is low. SUMMARY

[0004] The embodiments of the present application provide a processing method for matrix multiplication in parallel computing hardware and related equipment, which can improve the accuracy of single-precision matrix multiplication in the use of a computing chip supporting half-precision calculation.

[0005] To achieve the above object, the first aspect of the embodiments of the present application proposes a processing method for matrix multiplication in parallel computing hardware, comprising:

[0006] obtaining a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices;

[0007] performing half-precision processing based on a single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix;

[0008] obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices;

[0009] accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and taking the first single-precision target matrix as the result of matrix multiplication of the first initial matrix and the second initial matrix.

[0010] In some embodiments, the obtaining a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix comprises:

[0011] performing single-precision processing on the first half-precision matrix to obtain a first intermediate matrix;

[0012] performing half-precision processing on a difference between the first initial matrix and the first intermediate matrix to obtain the first difference matrix.

[0013] In some embodiments, the accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix comprises:

[0014] accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first addition item;

[0015] obtaining a second addition item according to a product of the first difference matrix and the second difference matrix;

[0016] accumulating the first addition item and the second addition item to obtain the first single-precision target matrix.

[0017] In some embodiments, an element in the first half-precision matrix comprises a mantissa; and the accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix further comprises:

[0018] obtaining a preset multiplication value determined according to a bit number of the mantissa;

[0019] multiplying the first difference matrix by the preset multiplication value to obtain a first enlarged matrix, and multiplying the second difference matrix by the preset multiplication value to obtain a second enlarged matrix;

[0020] accumulating a product of the first half-precision matrix and the second enlarged matrix, and a product of the second half-precision matrix and the first enlarged matrix to obtain an intermediate enlarged matrix;

[0021] dividing the intermediate enlarged matrix by the preset multiplication value to obtain an intermediate reduced matrix;

[0022] accumulating the product of the first half-precision matrix and the second half-precision matrix and the intermediate reduced matrix to obtain the first single-precision target matrix.

[0023] In some embodiments, the obtaining the first initial matrix and the second initial matrix comprises:

[0024] obtaining a first target matrix and a second target matrix;

[0025] obtaining a maximum matrix multiplication operation order of the parallel computing hardware;

[0026] dividing the first target matrix to obtain at least one first initial matrix and dividing the second target matrix to obtain at least one second initial matrix based on the maximum matrix multiplication operation order.

[0027] In some embodiments, when the first initial matrix and the second initial matrix are multiple, the method further comprises:

[0028] selecting the second initial matrix in the second target matrix according to the block position of the first initial matrix in the first target matrix, and generating a plurality of matrix sequences; each of the matrix sequences comprises a plurality of matrix groups, and each of the matrix groups comprises the first initial matrix and the second initial matrix;

[0029] accumulating the product of the first half-precision matrix and the second half-precision matrix in each of the matrix groups to obtain a first accumulated matrix of the matrix sequence, and accumulating the sum of the product of the first half-precision matrix and the second difference value matrix and the product of the second half-precision matrix and the second difference value matrix in each of the matrix groups to obtain a second accumulated matrix of the matrix sequence;

[0030] obtaining a second single-precision target matrix according to the first accumulated matrix and the second accumulated matrix, and taking the second single-precision target matrix as the result of the matrix multiplication operation of the first target matrix and the second target matrix.

[0031] In some embodiments, the method further comprises:

[0032] performing double-precision processing based on a single-precision data type to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a verification matrix of the second single-precision target matrix;

[0033] performing multiplication operation on the first double-precision matrix and the second double-precision matrix to obtain an evaluation matrix;

[0034] obtaining a verification result based on the verification matrix and the evaluation matrix;

[0035] comparing the verification result with a preset verification threshold, and outputting the second single-precision target matrix based on the comparison result.

[0036] To achieve the above object, a second aspect of the embodiments of the present application provides a matrix multiplication device, comprising:

[0037] an obtaining module, configured to obtain a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices;

[0038] a half-precision conversion module, configured to perform half-precision processing based on a single-precision data type, to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix;

[0039] a difference processing module, configured to obtain a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix, and obtain a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices;

[0040] a calculation module, configured to accumulate a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix, to obtain a first single-precision target matrix, and use the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix.

[0041] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the processing method of matrix multiplication in parallel computing hardware according to the first aspect when executing the computer program.

[0042] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer-readable storage medium, and stores a computer program, wherein the computer program is executed by a processor to implement the processing method of matrix multiplication in parallel computing hardware according to the first aspect.

[0043] The processing method and related device for matrix multiplication operation in parallel computing hardware provided by the embodiments of the present application obtain a first initial matrix and a second initial matrix, wherein the first initial matrix and the second initial matrix are both single-precision matrices; then, half-precision processing is performed based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; then, a first difference matrix is obtained based on the difference between the first initial matrix and the first half-precision matrix, and a second difference matrix is obtained based on the difference between the second initial matrix and the second half-precision matrix, wherein the first difference matrix and the second difference matrix are both half-precision matrices; finally, the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain a first single-precision target matrix, and the first single-precision target matrix is taken as the result of the matrix multiplication operation of the first initial matrix and the second initial matrix. The embodiments of the present application save the error of the first initial matrix converted to the first half-precision matrix by using the first difference matrix, and save the error of the second initial matrix converted to the second half-precision matrix by using the second difference matrix, so as to add an error compensation term in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operation, and then obtain a single-precision multiplication operation result with higher accuracy on a hardware device supporting only half-precision multiplication operation.

[0044] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the following description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the processing method for matrix multiplication operation in parallel computing hardware provided by an embodiment of the present application.

[0046] Figure 2 is Figure 1 is a flowchart of step S101 in

[0047] Figure 3 is a structure diagram of matrix blocking provided by an embodiment of the present application.

[0048] Figure 4 is a floating-point data structure diagram provided by an embodiment of the present application.

[0049] Figure 5 is Figure 1 is a flowchart of step S103 in

[0050] Figure 6 yes Figure 1 The flowchart for step S104.

[0051] Figure 7 yes Figure 1 Another flowchart of step S104 in the process.

[0052] Figure 8 This is a schematic flowchart of a matrix block multiplication operation provided in one embodiment of this application.

[0053] Figure 9 This is another schematic diagram of a matrix block multiplication operation provided in an embodiment of this application.

[0054] Figure 10 This is another flowchart of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of this application.

[0055] Figure 11 This is a schematic diagram of an improved process for matrix block multiplication provided in an embodiment of this application.

[0056] Figure 12 This is a schematic diagram of another improved process for matrix block multiplication operation provided in an embodiment of this application.

[0057] Figure 13 This is another flowchart of a method for processing matrix multiplication operations in parallel computing hardware provided in an embodiment of this application.

[0058] Figure 14 This is a schematic diagram of the chip structure of the Ascend AI processor provided in one embodiment of this application.

[0059] Figure 15 This is a schematic diagram illustrating the execution of block-based parallel computation provided in an embodiment of this application.

[0060] Figure 16 This is a simulation diagram comparing the accuracy of a matrix multiplication operation processing method in parallel computing hardware provided in one embodiment of this application.

[0061] Figure 17 This is a simulation diagram comparing the working efficiency of a matrix multiplication operation processing method in parallel computing hardware provided in an embodiment of this application.

[0062] Figure 18 This is a simulation data table showing the working efficiency of the matrix multiplication operation processing method in parallel computing hardware provided in one embodiment of this application.

[0063] Figure 19 This is another accuracy comparison simulation diagram of the matrix multiplication operation processing method in the parallel computing hardware provided in one embodiment of this application.

[0064] Figure 20 is a structural schematic diagram of a matrix multiplication operation device provided by an embodiment of the present application.

[0065] Figure 21 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0067] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0069] First, the meanings of several terms involved in the present application are analyzed:

[0070] Single-precision matrix multiplication operation refers to the operation of multiplying two matrices of single-precision floating-point type. Single-precision floating-point type usually uses 32-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 8 bits are used for the exponent, and 23 bits are used for the mantissa. Assuming that the size of matrix A is m x n and the size of matrix B is n x p, then the size of their product C is m x p, where the value of each element C[i][j] of matrix C is the sum of the products of the elements of the i-th row of matrix A and the elements of the j-th column of matrix B.

[0071] Similar to single-precision matrix multiplication operation, half-precision matrix multiplication operation refers to the operation of multiplying two matrices of half-precision floating-point type. Half-precision floating-point type usually uses 16-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 5 bits are used for the exponent, and 10 bits are used for the mantissa. Since the precision of half-precision floating-point type is lower, in actual applications, half-precision matrix multiplication is usually used in scenarios that require less precision, such as calculations in neural networks.

[0072] IEEE-754 is a binary floating-point representation of numbers, which is an international standard developed by the Institute of Electrical and Electronics Engineers (IEEE). The standard specifies the binary representation of floating-point numbers in computers, including the representation of sign bits, exponent bits, and mantissa bits, etc.

[0073] Parallel computing hardware refers to a hardware device that can perform multiple computing tasks simultaneously. These hardware devices usually have high parallel computing capabilities and efficient computing resource management capabilities, and can achieve efficient computing and data processing.

[0074] In recent years, with the rapid development of machine learning, artificial intelligence and other fields, the number of matrix multiplication operations has increased significantly, and the development of specialized hardware for parallel computing has also been accelerated. As a result, improving the accuracy of matrix multiplication operations has become a major concern.

[0075] In related technologies, for single-precision matrix multiplication operations, a computing chip supporting single-precision calculation can be used for calculation, but the computing chip supporting single-precision calculation has a high cost. In addition, usually in the use of a computing chip supporting only half-precision calculation, the single-precision matrix to be multiplied is converted into a half-precision matrix, and then half-precision multiplication is performed, but the accuracy of the result obtained by this scheme is low.

[0076] Based on this, the embodiment of the application provides a processing method for matrix multiplication operation in parallel computing hardware and related equipment, which can improve the accuracy of single-precision matrix multiplication operation in a computing chip supporting half-precision calculation. The processing method for matrix multiplication operation in parallel computing hardware mainly comprises the following steps: obtaining a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; then performing half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; then obtaining a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtaining a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices; finally, accumulating the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and taking the first single-precision target matrix as the result of the matrix multiplication operation of the first initial matrix and the second initial matrix. The embodiment of the application is aimed at saving the error of the first initial matrix converted to the first half-precision matrix by using the first difference matrix, and saving the error of the second initial matrix converted to the second half-precision matrix by using the second difference matrix, so as to increase the error compensation term in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operation, and then obtain a single-precision multiplication operation result with higher accuracy on a hardware device supporting only half-precision multiplication operation.

[0077] The embodiment of the application provides a processing method for matrix multiplication operation in parallel computing hardware and related equipment, which is specifically described through the following embodiments. First, the processing method for matrix multiplication operation in parallel computing hardware in the embodiment of the application is described.

[0078] The embodiment of the application can acquire and process related data based on artificial intelligence technology. Wherein, artificial intelligence is to use digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best result. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0079] The parallel computing hardware matrix multiplication operation processing method provided by the embodiments of the present application relates to the technical field of artificial intelligence, in particular to the field of data computing and processing. The parallel computing hardware matrix multiplication operation processing method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server, and can also be a computer program running in the terminal or the server. For example, the computer program can be a native program or a software module in an operating system; can be a local application program (Application, APP), that is, a program that needs to be installed in an operating system to run, or can be an applet, that is, a program that only needs to be downloaded into a browser environment to run; can also be an applet that can be embedded into any APP. In short, the above computer program can be any form of application program, module or plug-in. The terminal communicates with the server through a network. The parallel computing hardware matrix multiplication operation processing method can be executed by the terminal or the server, or cooperatively executed by the terminal and the server.

[0080] In some embodiments, the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer or a smart watch, etc. In addition, the terminal can also be a smart vehicle-mounted device. The smart vehicle-mounted device applies the parallel computing hardware matrix multiplication operation processing method of the embodiments to provide related services and improve the driving experience. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms; or a service node in a blockchain system, the service nodes in the blockchain system form a peer-to-peer (Peer To Peer, P2P) network among each other, and the P2P protocol is an application layer protocol running on the transmission control protocol (Transmission Control Protocol, TCP) protocol. The server can install a server of a text translation system, which can interact with the terminal through the server, for example, the server installs corresponding software, which can be an application that implements the parallel computing hardware matrix multiplication operation processing method, but is not limited to the above forms. The terminal and the server can be connected through communication connection modes such as Bluetooth, universal serial bus (Universal Serial Bus, USB) or network, which are not limited in the embodiments.

[0081] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0082] In some embodiments, the matrix multiplication processing method in parallel computing hardware provided in this application primarily aims to perform matrix multiplication of two single-precision matrices in parallel computing hardware that only supports half-precision matrix multiplication. The parallel computing hardware that only supports half-precision matrix multiplication can be an Ascend AI chip, etc.

[0083] First, the processing method for matrix multiplication operations in the parallel computing hardware of this application embodiment is described. In this embodiment, the processing method for matrix multiplication operations in the parallel computing hardware can be applied to a matrix multiplication operation device. (Refer to...) Figure 1 This is an optional flowchart of a matrix multiplication operation processing method in parallel computing hardware provided in this application embodiment. Figure 1 The method may include, but is not limited to, steps S101 to S104. It is also understood that this embodiment... Figure 1 The order of steps S101 to S104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0084] Step S101: Obtain the first initial matrix and the second initial matrix.

[0085] In some embodiments, after a single-precision matrix multiplication operation, the matrix multiplication device first obtains a first initial matrix and a second initial matrix, both of which are single-precision matrices. The first initial matrix and the second initial matrix refer to the matrix data to be processed by the matrix multiplication operation. In this embodiment, the source of obtaining the first initial matrix and the second initial matrix is ​​not limited; they can be manually input, generated based on calculations by a machine learning model, extracted from a text database by a computer device, or crawled from the internet by a computer device, etc.

[0086] In some embodiments, the first initial matrix is ​​derived from the first target matrix, and the second initial matrix is ​​derived from the second target matrix. The first target matrix and the second target matrix are two matrices that need to be multiplied. In some cases, the data size of the first target matrix and the second target matrix is ​​large, such as the large size of the input matrix for matrix multiplication operations in some applications of deep learning network models. Therefore, in order to improve the computational efficiency of the matrix multiplication operation device, and also due to the limitation of the running memory size of the parallel computing hardware, it is necessary to divide the first target matrix and the second target matrix into blocks. The process of dividing the first target matrix and the second target matrix into blocks according to the embodiments of this application is described below.

[0087] Therefore, refer to Figure 2 The process of obtaining the first initial matrix and the second initial matrix includes steps S201 to S203.

[0088] Step S201: Obtain the first target matrix and the second target matrix.

[0089] Step S202: Obtain the maximum matrix multiplication order of the parallel computing hardware.

[0090] Step S203: Based on the order of the maximum matrix multiplication operation, the first target matrix is ​​divided to obtain at least one first initial matrix, and the second target matrix is ​​divided to obtain at least one second initial matrix.

[0091] In some embodiments, after a single-precision matrix multiplication operation, the matrix multiplication device first obtains a first target matrix and a second target matrix, both of which are single-precision matrices. Then, it obtains the maximum matrix multiplication order in the parallel computing hardware; this can be understood as being obtainable through the size of the parallel computing hardware's runtime memory.

[0092] In some embodiments, according to the data size of the first target matrix and the second target matrix and the maximum matrix multiplication operation order in the parallel computing hardware, if the data size of the first target matrix and / or the second target matrix is greater than the maximum matrix multiplication operation order in the parallel computing hardware, the first target matrix and the second target matrix are processed as follows.

[0093] Referring to Figure 3 , it is a structure diagram of matrix block provided by an embodiment of the present application, wherein the matrix is a first target matrix with M=32 rows and K=48 columns, is a second target matrix with K=48 rows and N=32 columns. When the maximum matrix multiplication operation order in the current parallel operation hardware is 16x16 order, the first target matrix Figure 3 may be divided into 2x3 first initial matrices A single with 16 rows and 16 columns as shown in ; similarly, the second target matrix may be divided into 3x2 second initial matrices B single with 16 rows and 16 columns.

[0094] Step S102: based on single-precision data type, half-precision processing is performed to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix.

[0095] In some embodiments, due to the limitation of part of the parallel computing hardware, the matrix multiplication operation device only supports half-precision matrix multiplication operation, and therefore, based on the single-precision data type of the first initial matrix A single and the second initial matrix B single , half-precision processing is performed on the first initial matrix A single and the second initial matrix B single to obtain a first half-precision matrix A single of the first initial matrix A half and a second half-precision matrix B single of the second initial matrix B half , so that the subsequent matrix multiplication operation device performs matrix multiplication operation B half according to the first half-precision matrix A half and the second half-precision matrix.

[0096] In some embodiments, after the first initial matrix and the second initial matrix are converted into the first half-precision matrix and the second half-precision matrix, since the half-precision floating-point number data cannot fully represent the single-precision floating-point number data, i.e., the data truncation error is introduced when the single-precision data (FP32) input is converted into the half-precision data (FP16) because the mantissa part cannot be fully represented. Therefore, directly taking the structure after the multiplication operation of the first half-precision matrix and the second half-precision matrix as the result of the first initial matrix and the second initial matrix at this time will result in low accuracy. In order to explain the above-mentioned data truncation error and at the same time explain the present application, reference is made to Figure 4 The method provided in the present application is further introduced.

[0097] Reference is made to Figure 4 is a floating-point number data structure diagram provided by an embodiment of the present application. Wherein, the first initial matrix A single is a single-precision matrix, and therefore for each element data, there are 32 bits of data, of which 1 bit is used for the sign bit, 8 bits are used for the exponent, and 23 bits are used for the mantissa; in addition, the first half-precision matrix A half is a half-precision matrix, and therefore for each element data, there are 16 bits of data, of which 1 bit is used for the sign bit, 5 bits are used for the exponent, and 10 bits are used for the mantissa. Therefore, the first half-precision matrix A half cannot fully store all the mantissa data of the first initial matrix A single , i.e., the first half-precision matrix A half can only store the first mantissa part l1 of the first initial matrix A single , thereby inevitably leading to data truncation error. Therefore, in the embodiment of the present application, the first difference matrix R A-half is introduced, which is used to store the remaining second mantissa part l2 of the first initial matrix A single . It can be understood that since the first difference matrix is also a half-precision matrix, its mantissa part can also store 10 bits of data. In addition, considering the hidden mantissa bit in the IEEE-754 floating-point number rule, the first difference matrix actually saves the last ten mantissas of the mantissa part that is not 0 in the second mantissa part l2. Therefore, theoretically, for the worst case, i.e., the first data in the second mantissa part l2 is not 0, the mantissa part data that can be stored by the first half-precision matrix and the first difference matrix reaches 21 bits, which is very close to the 23-bit mantissa data part of the first initial matrix. Based on this, the data truncation error existing after the first initial matrix and the second initial matrix are converted into the first half-precision matrix and the second half-precision matrix can be effectively solved.

[0098] Therefore, the error of the first initial matrix converted to the first half-precision matrix is saved by using the first difference matrix, and the error of the second initial matrix converted to the second half-precision matrix is saved by using the second difference matrix, so that an error compensation term is added in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operation, and then a single-precision multiplication operation result with higher accuracy is obtained on a hardware device supporting only half-precision multiplication operation.

[0099] Step S103: obtaining the first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtaining the second difference matrix based on the difference between the second initial matrix and the second half-precision matrix.

[0100] In some embodiments, in order to solve the data truncation error after the first initial matrix and the second initial matrix are converted to the first half-precision matrix and the second half-precision matrix, the matrix multiplication operation device obtains the first difference matrix based on the difference between the first initial matrix and the first half-precision matrix, and obtains the second difference matrix based on the difference between the second initial matrix and the second half-precision matrix. It can be understood that due to the limitation of partial parallel computing hardware, the matrix multiplication operation device only supports multiplication operation of half-precision matrix, and therefore the first difference matrix and the second difference matrix are both half-precision matrices.

[0101] In some embodiments, in order to more effectively improve the accuracy of single-precision matrix multiplication operation in a computing chip supporting half-precision calculation, it is necessary to more accurately generate the first difference matrix and the second difference matrix. The process of generating the first difference matrix and the second difference matrix provided by the embodiments of the present application is described as follows.

[0102] Reference Figure 5 The first difference matrix is obtained based on the difference between the first initial matrix and the first half-precision matrix, including steps S501 to S502.

[0103] Step S501: performing single-precision processing on the first half-precision matrix to obtain a first intermediate matrix.

[0104] Step S502: performing half-precision processing on the difference between the first initial matrix and the first intermediate matrix to obtain the first difference matrix.

[0105] In some embodiments, in order to more accurately generate the first difference matrix R A-half , it is necessary to accurately determine the difference between the first half-precision matrix A half and the first initial matrix A single , and therefore it is necessary to perform single-precision processing on the first half-precision matrix A half to obtain a first intermediate matrix to_single(A half ). Then, according to the first initial matrix Ahalf to_single(A half ) of the first intermediate matrix to_half(A half -to_single(A half ) to obtain a first difference matrix with higher precision as follows:

[0106] R A-half = to_half(A half -to_single(A half )) (1)

[0107] In some embodiments, similar to the first difference matrix, in order to more accurately generate the second difference matrix R B-half , it is necessary to accurately determine the difference between the second half-precision matrix B half and the second initial matrix B single , so it is necessary to perform single-precision processing on the second half-precision matrix B half to obtain a second intermediate matrix to_single(B half ). Then, according to the difference B half -to_single(B half ) between the second initial matrix B half and the second intermediate matrix to_single(B half ), half-precision processing is performed to obtain a second difference matrix with higher precision as follows:

[0108] R B-half = to_half(B half -to_single(B half )) (2)

[0109] Step S104: Accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and the first single-precision target matrix is used as the result of the matrix multiplication operation of the first initial matrix and the second initial matrix.

[0110] In some embodiments, after obtaining the first difference matrix and the second difference matrix, in order to more effectively improve the precision of single-precision matrix multiplication operation in the calculation chip supporting half-precision calculation, the multiplication operation of the first initial matrix and the second initial matrix can be further converted as follows:

[0111]

[0112] It can be understood that the formula (3) includes the product of the first half-precision matrix and the second half-precision matrix A half Bhalf Also included is a compensation term A half R B-half +B half R A-half +R A-half R B-half .

[0113] In some embodiments, the first difference matrix R A-half is used to store the difference between the first initial matrix A single The first half-precision matrix A half , so the value of the first difference matrix R half will be relatively small compared to the first half-precision matrix A A-half ; Similarly, the second difference matrix R B-half is used to store the difference between the second initial matrix B single The second half-precision matrix B half , so the value of the second difference matrix R half will be relatively small compared to the second half-precision matrix B B-half . Based on this, for formula (3), the product of the first difference matrix and the second difference matrix R A-half R B-half is very small compared to other terms, so in the embodiments of the present application, in order to improve the processing efficiency of the processing method of the matrix multiplication operation in parallel computing hardware, the product of the first difference matrix and the second difference matrix R A-half R B-half in formula (3) can be ignored as a redundant term. Based on this, the result of the matrix multiplication operation of the first initial matrix and the second initial matrix, the first single-precision target matrix, can be obtained by accumulating the product of the first half-precision matrix A half and the second half-precision matrix B half , the product of the first half-precision matrix A half and the second difference matrix R B-half , and the product of the second half-precision matrix A half and the first difference matrix R A-half :

[0114]

[0115] In some embodiments, for some application scenarios that pay more attention to the processing efficiency of single-precision matrix multiplication operation, in order to more effectively improve the accuracy of single-precision matrix multiplication operation in the use of computing chips supporting half-precision calculation, the product of the first difference matrix and the second difference matrix R A- half R B-half (i.e. the redundant term) in formula (3) still needs to be considered.

[0116] Therefore, referring to Figure 6 , the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain the first single-precision target matrix, including steps S601 to S603.

[0117] Step S601: The product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain the first additive term.

[0118] Step S602: The second additive term is obtained according to the product of the first difference matrix and the second difference matrix.

[0119] Step S603: The first additive term and the second additive term are accumulated to obtain the first single-precision target matrix.

[0120] In some embodiments, in order to more effectively improve the accuracy of single-precision matrix multiplication in a computing chip supporting half-precision calculation, based on formula (3), the product of the first half-precision matrix and the second half-precision matrix A half B half , the product of the first half-precision matrix and the second difference matrix A half R B-half , and the product of the second half-precision matrix and the first difference matrix B half R A-half are accumulated to obtain the first additive term as follows:

[0121] C single_1 =A half B half +A half R B-half +B half R A-half (5)

[0122] Then, according to the product of the first difference matrix and the second difference matrix R A-half R B-half , the second additive term is obtained as follows:

[0123] C single_2 =R A-half R B-half (6)

[0124] The first additive term C single_1 and the second additive term C single_2 are accumulated to obtain the first single-precision target matrix as follows:

[0125]

[0126] In some embodiments of the IEEE-754 standard, if two floating-point numbers differ significantly in magnitude, the smaller floating-point number may be ignored during addition, a phenomenon known as "floating-point underflow." For example, with an F32 subarray of 23 bits, 8388608 has a length of 7, meaning that an FP32 subarray can represent a maximum of 7 significant digits (the number of digits after the decimal point). The 7th digit may not represent all digits, but the 6th digit is always significant. Suppose a = 112345.1 and b = 0.00001. Mathematically, a + b = 112345.10001. However, the program calculation results in a + b = 112345.101562, indicating that a small floating-point number is partially ignored. This occurs because 112345.10001, when converted to a single-precision floating-point number in IEEE-754, must first be converted to 1.1234510001. Since the maximum number of significant digits after the decimal point is 7, 1.1234510001 becomes inaccurate starting from the 8th decimal place. Furthermore, if two numbers are close, subtracting them can result in a very small difference. If the result is too small to be represented, it is rounded to zero.

[0127] Analyzing formula (7), on the one hand, if the first initial matrix A single and the first half-precision matrix A half Being very close will result in the first difference matrix R. A-half It is very small, which indirectly leads to B in formula (7). half R A-half It is very small; similarly, if the second initial matrix B is very small. single Second half-precision matrix B half Being very close will result in the second difference matrix R. B-half It is very small, which indirectly leads to A in formula (7). half R B-half Very small; on the other hand, using the first difference matrix R A-half Additional records of the first initial matrix A single The remaining second mantissa portion l2 has a smaller data volume than the second half-precision matrix B. half The data volume is more than 10 orders of magnitude smaller, which will further lead to B... half R A-half During calculations, floating-point underflow issues may occur; similarly, the second difference matrix R may be used. B-half Additional records for the second initial matrix B single The remaining second tail part l B-2 The amount of data is greater than that of the first half-precision matrix A half The data volume is more than 10 orders of magnitude smaller, which will further lead to the need for A. halfR B-half During the operation, the floating point underflow problem may occur.

[0128] Therefore, in order to solve the floating point underflow problem, so as to more effectively improve the precision of single-precision matrix multiplication operation in the calculation chip supporting half-precision calculation, the following further processing is needed for the first difference matrix R A-half and the second difference matrix R B-half .

[0129] Therefore, referring to Figure 7 , the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix are accumulated to obtain the first single-precision target matrix, including steps S701 to S705.

[0130] Step S701: Obtain a preset multiplication value determined according to the number of bits of the mantissa.

[0131] Step S702: Multiply the first difference matrix by the preset multiplication value to obtain a first enlarged matrix, and multiply the second difference matrix by the preset multiplication value to obtain a second enlarged matrix.

[0132] Step S703: Accumulate the product of the first half-precision matrix and the second enlarged matrix, and the product of the second half-precision matrix and the first enlarged matrix to obtain an intermediate enlarged matrix.

[0133] Step S704: Divide the intermediate enlarged matrix by the preset multiplication value to obtain an intermediate reduced matrix.

[0134] Step S705: Accumulate the product of the first half-precision matrix and the second half-precision matrix and the intermediate reduced matrix to obtain the first single-precision target matrix.

[0135] In some embodiments, since the first half-precision matrix A half stores the first mantissa part l1 of the first initial matrix A single , the first difference matrix R A-half stores the remaining second mantissa part l2 of the first initial matrix A single , and due to the hidden mantissa bit in the IEEE-754 floating point number rule. Therefore, the data value of the first initial matrix A single and the data value of the first difference matrix R A-half actually differ by at least 11 orders of magnitude. Based on the above reasons, the preset multiplication value is determined to be 2 11 according to the number of bits of the mantissa of the half-precision data structure. Then, the first difference matrix R 11 is multiplied by the preset multiplication value 2 A-half to obtain the first enlarged matrix:

[0136] 2 11 R A-half = to_half(A half -to_single(A half )×2 11 ) (8)

[0137] At this time, the data values of the first amplification matrix 2 11 R A-half and the data values of the second half-precision matrix B half are similar in order of magnitude, which can effectively prevent the floating-point underflow problem of B half (2 11 R A-half ). Similarly, by multiplying the second difference matrix by a preset multiplication value, the second amplification matrix is obtained as follows:

[0138] 2 11 R B-half = to_half(B half -to_single(B half )×2 11 ) (9)

[0139] At this time, the data values of the second amplification matrix 2 11 R B-half and the data values of the first half-precision matrix A half are similar in order of magnitude, which can effectively prevent the floating-point underflow problem of A half (2 11 R B-half ). Next, based on the intermediate term A half R B-half +B half R A-half of formula (7), the product of the first half-precision matrix and the second amplification matrix and the product of the second half-precision matrix and the first amplification matrix are further accumulated, and the intermediate amplification matrix is obtained as follows:

[0140]

[0141] Since the intermediate amplification matrix is amplified by a preset multiplication value 2 11 times compared with formula (4), after solving the floating-point underflow problem, the intermediate amplification matrix needs to be divided by the preset multiplication value 2 11 , and the intermediate reduction matrix is obtained as follows:

[0142] (A half (2 11 R B-half )+B half (2 11 R A-half )) / 2 11(11)

[0143] At this time, based on formula (4), the product of the first half-precision matrix and the second half-precision matrix and the intermediate reduced matrix are accumulated to obtain the first single-precision target matrix as follows:

[0144] C single ≈A half B half +(A half (2 11 R B-half )+B half (2 11 R A-half )) / 2 11 (12)

[0145] At this time, the floating point underflow condition in the A A-half R half operation due to the too large data value gap between the first difference matrix R half and the second half-precision matrix B B-half can be effectively avoided; meanwhile, the floating point underflow condition in the B half R A-half operation due to the too large data value gap between the second difference matrix R B-half and the first half-precision matrix A half can also be effectively avoided, thereby more effectively improving the precision of the single-precision matrix multiplication operation in the computing chip supporting half-precision calculation.

[0146] In some embodiments, when the data size of the first target matrix and the second target matrix is large, the first target matrix needs to be divided into a plurality of first initial matrices, and the second target matrix needs to be divided into a plurality of second initial matrices, at which time the matrix multiplication operation process of the first target matrix and the second target matrix is shown in FIG. 8. Figure 3 First, the second initial matrix is selected in the second target matrix according to the block position of the first initial matrix in the first target matrix, and a plurality of matrix groups are generated as follows:

[0147]

[0148] wherein k represents the number of the first initial matrix and the second initial matrix. Then, the first initial matrix and the second initial matrix are subjected to matrix operation to obtain a plurality of first single-precision target matrices C single ; finally, all the first single-precision target matrices C single are accumulated to obtain the second single-precision target matrix At this time, the second single-precision target matrix can be obtained by the following formula:

[0149]

[0150] At this time, combining formula (12) and formula (14), when the first target matrix is divided into a plurality of first initial matrices, and the second target matrix is divided into a plurality of second initial matrices, referring to Figure 8 , it is a flowchart of a matrix block multiplication operation provided by an embodiment of the present application, wherein for the first initial matrix and the second initial matrix in each matrix sequence, formula (12) is sequentially operated to obtain the first single-precision matrix C single , and then the first single-precision matrix C single is accumulated to obtain the second single-precision matrix C single .

[0151] It can be understood that, in order to further improve the accuracy of the matrix multiplication operation, Figure 8 the flowchart shown in A-half R B-half , that is, as shown in Figure 9 . Figure 9 is another flowchart of a matrix block multiplication operation provided by an embodiment of the present application, wherein compared with Figure 8 , a plurality of multiplication operations of the redundant term R A-half R B-half need to be further performed.

[0152] In some embodiments, when the data size of the first target matrix and the second target matrix is small, that is, the maximum matrix multiplication order in the parallel operation hardware is larger than the data size of the first target matrix and the second target matrix, it is further explained that the multiplication operation of the first target matrix and the second target matrix can be immediately processed in the parallel operation hardware. At this time, the first target matrix is directly taken as the first initial matrix, and the second target matrix is also taken as the second initial matrix. It can be understood that, at this time, the first single-precision target matrix C single is equal to the second single-precision target matrix C single .

[0153] In some embodiments, when the operation flow as shown in Figure 8 is used, there is still a floating point underflow situation caused by a large order of magnitude difference between different first single-precision matrices C single . In addition, considering that the data values of A half B half in formula (13) and (A half (2 11 R B-half )+B half (2 11 R A-half )) / 2 11The data values ​​differ by a significant order of magnitude, therefore, in parallel computing hardware, these two items need to be allocated different sized storage cache spaces. If we still use... Figure 8 In the illustrated computational flow, when performing matrix multiplication of the first and second target matrices, the parallel computing hardware needs to divide different storage cache spaces multiple times to store the data sequentially. Data and This data will limit the efficiency of matrix multiplication operations in parallel computing hardware.

[0154] Therefore, in order to improve the accuracy of single-precision matrix multiplication and also to improve the efficiency of matrix multiplication in parallel computing hardware, it is necessary to... Figure 8 The operational flow shown is further optimized as follows. (Refer to...) Figure 10 The matrix multiplication operation processing method in parallel computing hardware provided in this application embodiment further includes the following steps S1001 to S1003.

[0155] Step S1001: Select a second initial matrix in the second target matrix according to the block position of the first initial matrix in the first target matrix, and generate multiple matrix sequences.

[0156] Step S1002: Accumulate the product of the first half-precision matrix and the second half-precision matrix in each matrix group to obtain the first cumulative matrix of the matrix sequence. Accumulate the product of the first half-precision matrix and the second difference matrix in each matrix group to obtain the second cumulative matrix of the matrix sequence.

[0157] Step S1003: Based on the first cumulative matrix and the second cumulative matrix, obtain the second single-precision target matrix, and use the second single-precision target matrix as the result of matrix multiplication of the first target matrix and the second target matrix.

[0158] In some embodiments, as described above, when the first target matrix is ​​divided into multiple first initial matrices and the second target matrix is ​​divided into multiple second initial matrices, then referring to formula (1) and Figure 3 As shown, a second initial matrix is ​​selected in the second target matrix according to the block position of the first initial matrix in the first target matrix, and multiple matrix sequences are generated. Each matrix sequence includes multiple matrix groups, and each matrix group includes the first initial matrix and the second initial matrix corresponding to the first initial matrix. The multiple matrix groups are shown in formula (14).

[0159] Next, refer to Figure 11is a flowchart of an improved process of matrix block multiplication provided by an embodiment of the present application. In the flowchart, the product of the first half-precision matrix and the second half-precision matrix in each matrix group is accumulated to obtain a first accumulated matrix of the matrix sequence, as shown below:

[0160]

[0161] Next, the sum of the product of the first half-precision matrix and the second difference matrix in each matrix group and the product of the second half-precision matrix and the second difference matrix in each matrix group is accumulated to obtain a second accumulated matrix of the matrix sequence, as shown below:

[0162]

[0163] Then, the first accumulated matrix and the second accumulated matrix are accumulated to obtain a second single-precision target matrix C single . Thus, the floating-point underflow caused by the large difference in magnitude between different first single-precision matrices C single is effectively avoided, thereby improving the accuracy of single-precision matrix multiplication. Meanwhile, the parallel computing hardware only needs to divide the storage cache space twice in the matrix multiplication of the first target matrix and the second target matrix, and in addition, the parallel computing capability of the parallel computing hardware can be better utilized, i.e., multiple operations of and multiple operations of can be calculated simultaneously, thereby effectively improving the work efficiency of single-precision matrix multiplication.

[0164] It can be understood that, in order to further improve the accuracy of matrix multiplication, Figure 11 the flowchart shown in A-half R B-half should be added with a redundant term R Figure 12 , as shown in Figure 12 is another improved flowchart of matrix block multiplication provided by an embodiment of the present application, wherein compared with Figure 11 , multiple multiplication operations of the redundant term R A-half R B-half are also required.

[0165] In some embodiments, in order to improve the reliability of the processing method of matrix multiplication in the parallel computing hardware, a detection comparison link needs to be added. Therefore, with reference to Figure 11 , the processing method of matrix multiplication in the parallel computing hardware further includes the following steps S1301 to S1304.

[0166] Step S1301: based on single-precision data, double-precision processing is performed to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a check matrix of the second single-precision target matrix.

[0167] Step S1302: the first double-precision matrix and the second double-precision matrix are subjected to multiplication operation to obtain an evaluation matrix.

[0168] Step S1303: based on the check matrix and the evaluation matrix, a check result is obtained.

[0169] Step S1304: the check result is compared with a preset check threshold, and based on the comparison result, the second single-precision target matrix is output.

[0170] In some embodiments, in order to improve the reliability of the processing method of the matrix multiplication operation in the parallel computing hardware, and also to verify the accuracy of the second single-precision target matrix C single obtained by using the processing method of the matrix multiplication operation in the parallel computing hardware provided by the present application. By using the high accuracy of the double-precision data structure, after the matrix multiplication operation device obtains the second single-precision target matrix C single , based on the single-precision data structure of the second single-precision target matrix C single , double-precision processing is performed on the second single-precision target matrix C single to obtain a check matrix to_double(C single ) of the second single-precision target matrix C single ; at the same time, based on the single-precision data structure of the first target matrix A single , double-precision processing is performed on the first target matrix A single to obtain a first double-precision matrix to_double(A single ) of the first target matrix A single , and based on the single-precision data structure of the second target matrix B single , double-precision processing is performed on the second target matrix B single to obtain a second double-precision matrix to_double(B single ) of the second target matrix B single , then the first double-precision matrix and the second double-precision matrix are subjected to double-precision matrix multiplication operation to obtain an evaluation matrix to_double(A single )·to_double(B single ). Next, based on the check matrix and the evaluation matrix, the check is performed as follows:

[0171]

[0172] Wherein, V is used to store the relative residual error between the check matrix and the evaluation matrix, which is used to represent the actual error between directly performing the matrix multiplication operation of the first target matrix and the second target matrix, and using the processing method of the matrix multiplication operation in the parallel computing hardware provided by the embodiment. F is the Euclidean norm of the matrix, which can be understood as the maximum singular value of the matrix, also known as the 2-norm of the matrix, which is used to evaluate the condition number of the matrix, i.e. the stability of the matrix and the reliability of the numerical solution.

[0173] When the relative residual error V is less than the preset check threshold, it can be determined that the processing method of the matrix multiplication operation in the parallel computing hardware provided by the embodiment is effective, and therefore the second single-precision target matrix can be taken as the result of the matrix multiplication operation of the first target matrix and the second target matrix and output. In the embodiment of the application, the setting of the preset check threshold is not subject to any constraints, i.e. it can be artificially pre-set or obtained by the matrix multiplication operation device according to historical operation rules.

[0174] It can be understood that the check steps shown in steps S1301 to S1304 are to check the reliability of the processing method of the matrix multiplication operation in the parallel computing hardware provided by the embodiment, and the processing method of the matrix multiplication operation in the parallel computing hardware provided by the embodiment is mainly used for the matrix multiplication operation of two single-precision matrices (i.e. the first target matrix and the second target matrix) in parallel computing hardware that only supports half-precision matrix multiplication operation. Therefore, in this case, the check steps shown in steps S1301 to S1304 will be placed in other parallel computing hardware that can support double-precision matrix multiplication operation and executed, and have no any influence on the processing method of the matrix multiplication operation in the parallel computing hardware provided by the embodiment.

[0175] In some examples, the processing method of the matrix multiplication operation in the parallel computing hardware provided by the application can be applied to the Ascend AI processor, which is a chip adapted to a specific field, and the core of the Ascend AI processor is an artificial intelligence chip. As shown in FIG. 6, the Ascend AI processor is composed of a plurality of processing units, and each processing unit is composed of a plurality of processing cores. Figure 14A chip structure diagram of the Ascend AI processor is shown. The Ascend AI processor provides three basic computing units: a matrix computing unit (CUBE), a vector computing unit (Vector), and a scalar computing unit (Scalar), which form three independent pipelines for corresponding calculations to complete the corresponding calculations. Among them, the L1 buffer is used to store the first target matrix and the second target matrix, and then perform a matrix conversion operation in the cache conversion unit and save it, wherein the conversion operation includes dividing the first target matrix into a plurality of first initial matrices, and converting the first initial matrix into a first half-precision matrix, and generating a first difference matrix and a first amplification matrix, and similarly, dividing the second target matrix into a plurality of second initial matrices, and converting the second initial matrix into a second half-precision matrix, and generating a second difference matrix and a second amplification matrix. The buffer L0A and the buffer L0B are used to store the matrices that are about to perform the matrix multiplication operation. The matrix computing unit is used to receive the matrices of the buffer L0A and the buffer L0B to perform the matrix multiplication operation, combined with Figure 12 The execution flow diagram is shown, wherein the matrix multiplication operation includes: and The accumulator is used to accumulate the data obtained by the matrix computing unit, that is, the addition operation as Figure 12 shown. The buffer L0C is used to store the data obtained by the accumulator calculation, and finally obtain the second single-precision target matrix C single .

[0176] In the artificial intelligence chip of Ascend AI processor, the unified buffer is an important part of its internal, used to store the data shared between different computing units. The system control module is a module in the Ascend chip, responsible for controlling the overall operation and scheduling of the chip. It contains the coordination and management between various functional modules, as well as processing external input and output tasks. The bus interface module is an interface module used to connect the Ascend chip with other external devices. It provides an interface for data transmission and communication with the host system or other external devices, realizing the interaction with external systems. The instruction cache is a high-speed cache in the Ascend chip used to store instructions. It is used to improve the reading speed of instructions, reduce instruction access delay, and improve the execution efficiency of instructions. The scalar instruction processing queue is a module in the Ascend chip, used to process scalar instructions. Scalar instructions are instructions that operate on single data, such as addition, multiplication, etc. The scalar instruction processing queue is responsible for receiving and decoding scalar instructions and distributing them to the corresponding functional units for execution. The instruction distribution module is a module in the Ascend chip, used to distribute decoded instructions to the corresponding functional units for execution. It is responsible for distributing instructions to the appropriate functional units according to their types and operands, realizing the parallel execution of instructions and efficient use of computing resources. In summary, the unified buffer in the Ascend chip is used to store data shared between different computing units, the system control module is responsible for overall scheduling and management, the bus interface module realizes communication with external devices, the instruction cache improves instruction reading speed, the scalar instruction processing queue processes scalar instructions, and the instruction distribution module distributes decoded instructions to the corresponding functional units for execution. These modules together form part of the functions in the Ascend chip. The CUBE queue is a hardware module in the Ascend chip, used to manage and schedule the execution of tasks. The CUBE queue can execute multiple tasks in parallel and switch and schedule tasks according to certain scheduling strategies to improve computing efficiency. The Vector queue is another hardware module in the Ascend chip, used to support vector computation. The Vector queue can efficiently perform vector operations by processing multiple vector data in parallel, improving the efficiency and performance of vector computation. The memory conversion queue is a hardware module in the Ascend chip, used to handle data storage and conversion. The memory conversion queue is responsible for managing data transmission and conversion between different storage media, such as data reading and writing between memory and external storage devices. The time synchronization module is a hardware module in the Ascend chip, used to ensure time synchronization between multiple Ascend chips. The time synchronization module ensures that multiple Ascend chips have consistent time references when executing tasks through precise clock synchronization mechanisms, supporting distributed computing and collaborative work. The vector computation unit is a hardware module in the Ascend chip, used to perform vector computation. The vector computation unit can efficiently perform large-scale vector data operations, providing parallel computing capabilities to accelerate vector computation-intensive tasks.The scalar calculation unit is a hardware module in the Ascend chip, used to perform scalar calculations. The scalar calculation unit is responsible for processing single data element calculation tasks, including addition, subtraction, multiplication, division, logical operations, etc., to support scalar computation-intensive tasks. The configuration port is an interface in the Ascend chip, used to configure and manage various parameters and settings of the chip. The configuration port provides a communication channel with the internal control logic of the chip, used to read and write configuration registers of the chip to achieve flexible configuration of chip functions and performance. The bus is a communication channel in the Ascend chip, used to connect various modules inside the chip and external devices. The bus is responsible for transmitting data and control signals to enable communication and collaboration between various modules inside the chip. The L2 cache area is a high-speed cache storage area in the Ascend chip, located between the chip core and the memory. The L2 cache area is used to store frequently accessed data and instructions, providing fast data read / write and access speed to speed up computation and data processing. DDR (Double Data Rate) is a type of memory in the Ascend chip, also one of the common memory types in computer systems. DDR memory uses double data rate technology, capable of transmitting twice the amount of data in one clock cycle, providing higher memory bandwidth and faster data access speed for storing and reading large-scale data.

[0177] It can be understood that the main improvement points of the present application are mainly implemented in the L1 buffer area, the cache conversion unit, the buffer L0A, the buffer L0B, the buffer L0C, the matrix calculation unit, and the accumulator in the artificial intelligence chip of the Ascend AI processor.

[0178] In some embodiments, the processing method for matrix multiplication operation in parallel computing hardware provided by the present application can be applied to block execution processing in ordinary parallel computing hardware, which usually includes multiple artificial intelligence chips, so that the multiplication operation of each block can be processed in parallel using an artificial intelligence chip. For example, in the execution flow shown in Figure 12 , the multiplication operation of the block matrix is executed using the first artificial intelligence chip. Similarly, each block matrix multiplication can be processed using the corresponding artificial intelligence chip, i.e., the and are executed in parallel, and then the obtained data is added up, thereby improving the work efficiency of executing matrix multiplication operation. Taking the Ascend AI processor as an example, the Ascend AI processor supports 32 artificial intelligence chips, which can theoretically handle 32 block matrix multiplication operations simultaneously. In the following, four artificial intelligence chips are selected in a certain Ascend AI processor for example description, and further description is made with reference to Figure 15 . Figure 15Figure 1 is an execution schematic diagram of the block parallel operation provided by an embodiment of the present application. The first target matrix and the second target matrix are divided into four blocks, the data length calculated by each artificial intelligence chip and the index value of the block are calculated, then the four artificial intelligence chips are used for parallel calculation, and finally the calculation results of each artificial intelligence chip are summarized, so that the effect of accelerating the matrix calculation time is achieved by using multiple artificial intelligence chips for parallel calculation. Compared with using one artificial intelligence chip, the work efficiency can be theoretically improved by four times by using four artificial intelligence chips for parallel calculation.

[0179] In some embodiments, the input data is obtained from Figure 14 As can be seen from the above, when the matrix calculation is performed in the artificial intelligence chip of the Ascend AI processor, the input data is required to be placed on the buffer L0, the output finally uses the cache unit UB, and the memory applied by the host side can be transmitted to the Ascend CUBE core for use only through the Global Memory (GM). Therefore, the influence of data transfer on the calculation performance should be fully considered.

[0180] When the data amount after the block division is small, the data transfer is the main factor affecting the performance. In order to fully utilize the 256KB high-speed internal cache UB on the Ascend AI processor, the calculation in one block needs to apply different memory sizes on the UB multiple times, which limits the size of the data block placed on the UB, and causes the data on the UB to be exchanged to the temporary GM for storage, thereby reducing the execution efficiency. In order to solve this problem, the same calculation module can be modularized, that is, the calculation of α [k] and β [k] is separated, so that the same size UB can be uniformly applied, the number of UB applications is reduced, and the data transfer frequency between the GM and the UB is also reduced, thereby improving the execution efficiency.

[0181] In some embodiments, when the input data amount participating in the matrix multiplication is particularly large, the time cost of the calculation exceeds the time cost of the data transfer, and the block division can only solve the size limitation problem of the UB, but cannot solve the time consumption problem. At this time, the multi-core processing capability of the Ascend AI should be fully utilized. For example, assuming that the calculation of K for the test matrix (M, N, K) is performed on the multi-core for parallel calculation, this involves accumulating the calculation results of all cores after the multi-core calculation for the same block, so the intermediate results of the two calculations are stored in advance.

[0182] The Ascend AI processor realizes acceleration through large-scale parallel computing capability, and uses the Bisheng C++ language tool to express the mapping to the computing core of the device through the concept of a parallel operation workgroup. Each parallel operation workgroup is mapped to a certain core. Each parallel operation workgroup has the same instruction code but different identification ids, which is similar to an SPMD (Single Program Multiple Data) technology.

[0183] SPMD is a parallel computing model that refers to multiple processors or computing units executing the same program simultaneously but corresponding to different data. In the SPMD model, different processors or computing units have their own data sets and independently execute the same instruction sequence to process these data. The SPMD model is a task-based parallel model that is suitable for many parallel computing applications such as scientific computing, image processing, and data analysis. In the SPMD model, programmers need to divide the computing problem into multiple independent tasks and assign different data sets to each task. Then, each processor or computing unit will execute the same program in parallel but corresponding to different data sets.

[0184] Reference Figure 16 As shown in FIG. 6, which is a precision comparison simulation diagram of the matrix multiplication operation processing method in the parallel computing hardware provided by the embodiments of the present application. In order to further verify the reliability of the results obtained by adopting the parallel computing hardware matrix multiplication operation processing method provided by the embodiments of the present application, the NPU verification result (the present patent scheme), that is, the matrix multiplication operation processing scheme provided by the present application is compared with other two schemes, which are: 1, NPU verification result (without optimization scheme): that is, the scheme of the results obtained by directly dividing the first target matrix and the second target matrix into a plurality of first initial matrices and a plurality of second initial matrices, and then converting the plurality of first initial matrices and the plurality of second initial matrices into corresponding plurality of half-precision matrices and then performing half-precision matrix multiplication operation in the related art; 2, sgemm verification result scheme of OpenBLAS library: that is, the scheme of the results obtained by directly performing single-precision matrix multiplication operation on the first target matrix and the second target matrix using sgemm of the OpenBLAS library in the related art. It can be understood that the OpenBLAS library is an open-source basic linear algebra subroutine (BLAS) library for high-performance numerical computing. BLAS is a set of standard interfaces and functions for performing common linear algebra operations such as matrix multiplication, vector addition, etc. In addition, in the OpenBLAS library, sgemm is a function for performing matrix multiplication.

[0185] Figure 16The horizontal coordinate in the figure refers to the size of k of the input matrix (i.e. the number size of the column number K of the first target matrix and the row number K of the second target matrix), and the vertical coordinate refers to the precision test difference. It can be obviously obtained that, compared with the NPU verification result (without the optimization scheme), the relative residual V of the second single-precision target matrix C single obtained through the NPU verification result (the scheme of the present application) is very low, and the precision of the second single-precision target matrix C single obtained through the NPU verification result (the scheme of the present application) is very close to the sgemm verification result scheme of the OpenBLAS library, thereby verifying the reliability of the result obtained through the processing method for matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application.

[0186] Reference Figure 17 and Figure 18 is a working efficiency comparison simulation diagram and simulation data table of the working efficiency of the processing method for matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application. In order to further verify the reliability of the working efficiency improved through the processing method for matrix multiplication operation in parallel computing hardware provided by the embodiment of the present application, the running time of the optimization calculation execution flow (i.e. the execution flow shown in Figure 11 , i.e. the NPU verification result (the scheme of the present application) in Figure 17 , and the running time of the sgemm verification result scheme of the OpenBLAS library are simulated and compared. Figure 17 The horizontal coordinate in the figure refers to the size of k of the input matrix (i.e. the number size of the column number K of the first target matrix and the row number K of the second target matrix), and the vertical coordinate refers to the calculation delay. It can be obviously obtained that, with the increase of the number of the column number K of the first target matrix and the row number K of the second target matrix, the running time of the sgemm verification result scheme of the OpenBLAS library will greatly increase, while the running time of the NPU verification result (the scheme of the present application) (i.e. the execution flow shown in Figure 11 ) will increase, but the increase is very small. In addition, in combination with the speedup ratio data in Figure 18 , it can be obviously obtained that when the size of k of the input matrix (i.e. the number size of the column number K of the first target matrix and the row number K of the second target matrix) reaches 2 11At this time, the running time of the NPU verification result (the patent scheme) is much less than that of the sgemm verification result scheme of the OpenBLAS library. It can be understood that the acceleration data is the ratio of the running time of the sgemm verification result scheme of the OpenBLAS library to the running time of the NPU verification result (the patent scheme). Thus, the reliability of the improved work efficiency by using the optimization calculation execution flow (i.e., the execution flow as shown in Figure 11 ) adopted in the parallel computing hardware matrix multiplication operation processing method provided by the embodiment of the present application is verified.

[0187] Figure 19 is another precision comparison simulation diagram of the parallel computing hardware matrix multiplication operation processing method provided by an embodiment of the present application. In order to further verify the reliability of the improved work efficiency by using the parallel computing hardware matrix multiplication operation processing method provided by the embodiment of the present application. This embodiment simulates and compares the precision (i.e., relative residual) obtained by the two schemes as shown in Figure 11 and Figure 12 , i.e., the redundant item scheme and the non-redundant item scheme. Figure 19 The horizontal coordinate in is the size of k of the input matrix (i.e., the number size of the column number K of the first target matrix and the row number K of the second target matrix), and the vertical coordinate is the relative residual. It can be seen that when the maximum difference of the relative residual of the two schemes is about 10 -8 , i.e., the precision of the results obtained by the two schemes is basically the same. Therefore, when the execution flow scheme as shown in Figure 11 is adopted, the calculation amount can be effectively reduced by one quarter, thereby the cache usage amount can be reduced by one quarter, and the work efficiency of the matrix multiplication operation is further improved.

[0188] The embodiment of the present application provides a processing method for matrix multiplication operation in parallel computing hardware, which can improve the accuracy of single-precision matrix multiplication operation in a computing chip supporting half-precision calculation. The first target matrix and the second target matrix, both of which are single-precision matrices, are divided into at least one first initial matrix and at least one second initial matrix, and a plurality of corresponding matrix groups are obtained. Then, based on the single-precision data type, half-precision processing is performed to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix in each matrix group. Then, based on the difference between the first initial matrix and the first half-precision matrix in each matrix group, a first difference matrix is obtained, and based on the difference between the second initial matrix and the second half-precision matrix, a second difference matrix is obtained. Then, the first difference matrix and the second difference matrix are multiplied by a preset multiplication value to obtain a first amplification matrix and a second amplification matrix. Next, the product of the first half-precision matrix and the second amplification matrix and the product of the second half-precision matrix and the first amplification matrix are accumulated to obtain an intermediate amplification matrix of each matrix group, and the intermediate amplification matrices of all matrix groups are accumulated to obtain a first cumulative matrix. Meanwhile, the product of the first half-precision matrix and the second half-precision matrix of all matrix groups is accumulated to obtain a second cumulative matrix, and the first cumulative matrix and the second cumulative matrix are added to obtain a second single-precision target matrix, which is used as the result of the matrix multiplication operation of the first target matrix and the second target matrix.

[0189] The embodiment of the present application is aimed at the process of single-precision matrix multiplication operation using a computing chip supporting half-precision calculation. First, based on the hardware parameters of parallel computing hardware, the first target matrix and the second target matrix are divided to obtain a plurality of first initial matrices and second initial matrices, and the first initial matrices and the second initial matrices are processed in half-precision to obtain first half-precision matrices and second half-precision matrices, so as to facilitate subsequent execution of the parallel computing hardware. Then, the first difference matrix is used to save the error after the first initial matrix is converted to the first half-precision matrix, and the second difference matrix is used to save the error after the second initial matrix is converted to the second half-precision matrix, so as to add an error compensation term in the multiplication operation of the first half-precision matrix and the second half-precision matrix to perform corresponding half-precision multiplication operation, thereby effectively improving the accuracy of half-precision multiplication operation on single-precision matrix. Then, the first difference matrix and the second difference matrix are multiplied by a preset multiplication value and then corresponding operation is performed, so as to solve the floating point underflow problem and improve the accuracy of matrix multiplication operation. Finally, each item of multiplication operation of each matrix group is separated to adjust the operation order, so as to further solve the floating point underflow problem and improve the working efficiency of the matrix multiplication operation device. Furthermore, a single-precision multiplication operation result with high accuracy can be obtained on a hardware device supporting only half-precision multiplication operation, and the working efficiency of multiplication operation is improved.

[0190] The embodiment of the application further provides a matrix multiplication operation device, which can implement the processing method of the matrix multiplication operation in the parallel computing hardware. Figure 20 The device 2000 comprises:

[0191] The acquisition module 2010 is configured to acquire a first initial matrix and a second initial matrix, wherein the first initial matrix and the second initial matrix are both single-precision matrices.

[0192] The half-precision conversion module 2020 is configured to perform half-precision processing based on the single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix.

[0193] The difference processing module 2030 is configured to obtain a first difference matrix based on the difference between the first initial matrix and the first half-precision matrix and obtain a second difference matrix based on the difference between the second initial matrix and the second half-precision matrix, wherein the first difference matrix and the second difference matrix are both half-precision matrices.

[0194] The calculation module 2040 is configured to accumulate the product of the first half-precision matrix and the second half-precision matrix, the product of the first half-precision matrix and the second difference matrix, and the product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and use the first single-precision target matrix as the result of the matrix multiplication operation of the first initial matrix and the second initial matrix.

[0195] The specific implementation of the matrix multiplication operation device of the embodiment is basically the same as that of the processing method of the matrix multiplication operation in the parallel computing hardware, and will not be described here.

[0196] The embodiment of the application further provides an electronic device, which comprises:

[0197] at least one memory;

[0198] at least one processor;

[0199] at least one program;

[0200] The program is stored in the memory, and the processor executes the at least one program to implement the processing method of the matrix multiplication operation in the parallel computing hardware according to the embodiment of the application. The electronic device can be any intelligent terminal, such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0201] Please refer to Figure 21 , Figure 21 which illustrates the hardware structure of the electronic device of another embodiment, and the electronic device comprises:

[0202] The processor 2101 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0203] The memory 2102 can be implemented by a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 2102 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 2102 and are called and executed by the processor 2101 to implement the processing method for matrix multiplication in parallel computing hardware according to the embodiments of the present application.

[0204] The input / output interface 2103 is configured to realize information input and output.

[0205] The communication interface 2104 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0206] The bus 2105 is configured to transmit information between various components (for example, the processor 2101, the memory 2102, the input / output interface 2103, and the communication interface 2104) of the device.

[0207] The processor 2101, the memory 2102, the input / output interface 2103, and the communication interface 2104 are connected to each other through the bus 2105 to realize the communication connection between them in the device.

[0208] The embodiments of the present application further provide a storage medium, which is a computer readable storage medium. The storage medium stores a computer program. The computer program is executed by a processor to implement the processing method for matrix multiplication in parallel computing hardware.

[0209] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely from the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0210] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0211] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps or different steps.

[0212] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0213] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0214] The terms "first", "second", "third", "fourth" and the like used in the specification of the present application and the above-described drawings (if any) are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0215] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0216] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0217] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0218] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0219] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0220] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A processing method of a matrix multiplication operation in parallel computing hardware, characterized by, The method comprises the following steps: obtaining a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; performing half-precision processing based on a single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; obtaining a first difference matrix based on a difference between the first initial matrix and the first half-precision matrix and obtaining a second difference matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein the first difference matrix and the second difference matrix are both half-precision matrices; accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and taking the first single-precision target matrix as a result of matrix multiplication of the first initial matrix and the second initial matrix; the parallel computing hardware comprises an L1 buffer, a buffer L0A, a buffer L0B, and a matrix calculation unit; wherein the L1 buffer is used to store a first target matrix and a second target matrix, then perform a matrix conversion operation in a cache conversion unit and save, the buffer L0A and the buffer L0B are used to store matrices that are about to perform matrix multiplication operation, and the matrix calculation unit is used to receive the matrices of the buffer L0A and the buffer L0B to perform a matrix multiplication operation.

2. The processing method of matrix multiplication operation in parallel computing hardware according to claim 1, wherein, The first difference matrix is obtained based on a difference between the first initial matrix and the first half-precision matrix, and the second difference matrix is obtained based on a difference between the second initial matrix and the second half-precision matrix, comprising: performing single-precision processing on the first half-precision matrix to obtain a first intermediate matrix; performing half-precision processing on a difference between the first initial matrix and the first intermediate matrix to obtain the first difference matrix.

3. The processing method of matrix multiplication operation in parallel computing hardware according to claim 1, wherein, The first single-precision target matrix is obtained by accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix, comprising: accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix to obtain a first additive term; obtaining a second additive term according to a product of the first difference matrix and the second difference matrix; accumulating the first additive term and the second additive term to obtain the first single-precision target matrix.

4. The processing method of matrix multiplication operation in parallel computing hardware according to claim 1, wherein, The elements in the first half-precision matrix comprise a mantissa; the first single-precision target matrix is obtained by accumulating a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix, further comprising: obtaining a preset multiplication value determined according to a number of bits of the mantissa; multiplying the first difference matrix by the preset multiplication value to obtain a first amplified matrix, and multiplying the second difference matrix by the preset multiplication value to obtain a second amplified matrix; accumulate a product of the first half-precision matrix and the second amplification matrix, a product of the second half-precision matrix and the first amplification matrix, to obtain an intermediate amplification matrix; divide the intermediate amplification matrix by the preset multiplication value to obtain an intermediate reduction matrix; accumulate a product of the first half-precision matrix and the second half-precision matrix and the intermediate reduction matrix to obtain the first single-precision target matrix.

5. The processing method of matrix multiplication operation in parallel computing hardware according to claim 1, wherein, The method further comprises: obtaining a first target matrix and a second target matrix; obtaining a maximum matrix multiplication operation order of the parallel computing hardware; based on the maximum matrix multiplication operation order, dividing the first target matrix to obtain at least one first initial matrix, and dividing the second target matrix to obtain at least one second initial matrix.

6. The processing method of matrix multiplication operation in parallel computing hardware according to claim 5, wherein, When the first initial matrix and the second initial matrix are multiple, the method further comprises: selecting the second initial matrix in the second target matrix according to the block position of the first initial matrix in the first target matrix, and generating a plurality of matrix sequences; each of the matrix sequences comprises a plurality of matrix groups, and each of the matrix groups comprises the first initial matrix and the second initial matrix; accumulating a product of the first half-precision matrix and the second half-precision matrix in each of the matrix groups to obtain a first accumulated matrix of the matrix sequence, and accumulating a sum of a product of the first half-precision matrix and the second difference value matrix and a product of the second half-precision matrix and the second difference value matrix in each of the matrix groups to obtain a second accumulated matrix of the matrix sequence; obtaining a second single-precision target matrix according to the first accumulated matrix and the second accumulated matrix, and taking the second single-precision target matrix as a result of matrix multiplication of the first target matrix and the second target matrix.

7. The processing method of matrix multiplication operation in parallel computing hardware according to claim 6, wherein, The method further comprises: performing double-precision processing based on a single-precision data type to obtain a first double-precision matrix of the first target matrix, a second double-precision matrix of the second target matrix, and a verification matrix of the second single-precision target matrix; performing multiplication operation on the first double-precision matrix and the second double-precision matrix to obtain an evaluation matrix; obtaining a verification result based on the verification matrix and the evaluation matrix; comparing the verification result with a preset verification threshold, and outputting the second single-precision target matrix based on a comparison result.

8. A matrix multiplication operation device, characterized by comprising: The device comprises: an obtaining module configured to obtain a first initial matrix and a second initial matrix; wherein the first initial matrix and the second initial matrix are both single-precision matrices; a half-precision conversion module configured to perform half-precision processing based on a single-precision data type to obtain a first half-precision matrix of the first initial matrix and a second half-precision matrix of the second initial matrix; a difference processing module configured to obtain a first difference value matrix based on a difference between the first initial matrix and the first half-precision matrix, and obtain a second difference value matrix based on a difference between the second initial matrix and the second half-precision matrix; wherein the first difference value matrix and the second difference value matrix are both half-precision matrices. The computing module is configured to accumulate a product of the first half-precision matrix and the second half-precision matrix, a product of the first half-precision matrix and the second difference matrix, and a product of the second half-precision matrix and the first difference matrix to obtain a first single-precision target matrix, and to take the first single-precision target matrix as a result of the matrix multiplication operation of the first initial matrix and the second initial matrix. The matrix multiplication device further comprises an L1 buffer, a buffer L0A, a buffer L0B, and a matrix computing unit. The L1 buffer is configured to store a first target matrix and a second target matrix, and then perform a matrix conversion operation in a cache conversion unit and save the result; the buffer L0A and the buffer L0B are configured to store matrices to be subjected to a matrix multiplication operation; and the matrix computing unit is configured to receive the matrices of the buffer L0A and the buffer L0B to perform a matrix multiplication operation.

9. An electronic device, comprising: The computer program is executed by the processor to implement the processing method for the matrix multiplication operation in the parallel computing hardware according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the processing method for the matrix multiplication operation in the parallel computing hardware according to any one of claims 1 to 7.

Images