2 results about "Transpose" patented technology

This application discloses a method and related apparatus for determining the output result of a model, belonging to the field of data processing technology. The network model includes an attention mechanism module; the method includes: determining a query matrix, a key matrix, and a value matrix through the attention mechanism module based on the current input information of the network model; processing the transpose of the key matrix to obtain an intermediate matrix, wherein the intermediate matrix has the same number of rows and columns as the transpose matrix, and the average value of each column element in the intermediate matrix is ​​less than the average value of the corresponding column elements in the transpose matrix; determining an output matrix through the attention mechanism module based on the query matrix, the intermediate matrix, and the value matrix; and determining the current output result of the network model based on the output matrix. Therefore, this application reduces the value of elements in the transpose matrix by processing it, thereby preventing numerical overflow and computational instability at the computational source.

The present application belongs to the technical field of data processing. Disclosed are a method for determining an output result of a model, and a related apparatus. A network model comprises an attention mechanism module. The method comprises: on the basis of current input information of a network model, determining a query matrix, a key matrix and a value matrix by means of an attention mechanism module; processing a transposed matrix of the key matrix, so as to obtain an intermediate matrix, wherein the intermediate matrix has the same number of rows and columns as those in the transposed matrix, and the average value of elements in each column in the intermediate matrix is less than the average value of elements in a corresponding column in the transposed matrix; on the basis of the query matrix, the intermediate matrix and the value matrix, determining an output matrix by means of the attention mechanism module; and on the basis of the output matrix, determining a current output result of the network model. It can be seen therefrom that in the present application, a transposed matrix of a key matrix is processed, so as to decrease the values of elements in the transposed matrix, thereby preventing a computational source from the problems of numerical overflow and computational instability.