Answer text generation method and device

Abstract

The application discloses a reply text generation method and device, and relates to the technical field of model optimization, which comprises the following steps: performing quantized tensor column decomposition on a feature training matrix in a model training process to calculate a weight matrix and a bias; in a pre-filling stage of model inference, performing matrix multiplication and addition operation on the feature matrix after the matrix tensor decomposition operation by using the weight matrix and the bias in the form of tensor decomposition after the model training; in a decoding stage, performing tensor decomposition on a one-dimensional feature vector to perform matrix multiplication and addition with the weight matrix and the bias in the form of tensor decomposition, so as to generate a target reply text based on the corresponding calculation result, thereby solving the technical problem that the selection requirement of the rank in the matrix decomposition calculation process is high in the related art, the complexity is difficult to effectively reduce, or the error between the calculation result obtained by using the decomposition form and the original result is large, and the technical effect of greatly reducing the storage and calculation complexity and effectively improving the inference performance is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model optimization, and particularly relates to a reply text generation method and device. BACKGROUND

[0002] The calculation complexity and memory occupation of a self-attention module in a Transformer model grow with the length of an input sequence, which poses a great challenge to the calculation performance and energy efficiency ratio of a traditional general hardware platform.

[0003] At present, the Transformer model needs to perform dense matrix multiplication calculation in the training stage and the inference stage, and the time length of the matrix multiplication calculation is relatively long. Therefore, the related technology can accelerate the multiplication calculation of the matrix by using a matrix decomposition method to reduce the complexity of storage and calculation.

[0004] However, in the related technology, the selection of the rank in the matrix decomposition calculation process has high requirements, which makes it difficult to effectively reduce the complexity, or easily leads to a large error between the calculation result obtained by using the decomposition form and the original result, and thus needs to be urgently solved. SUMMARY

[0005] The present application provides a reply text generation method and device to at least solve the technical problem that the selection of the rank in the matrix decomposition calculation process has high requirements in the related technology, which makes it difficult to effectively reduce the complexity, or easily leads to a large error between the calculation result obtained by using the decomposition form and the original result.

[0006] The application provides a reply text generation method, comprising the following steps: obtaining initial text information of a target user, and inputting the initial text information into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and performing a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result; obtaining a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model, and performing a matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in a pre-padding stage of an inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; in a decoding stage of the inference process, generating a one-dimensional feature vector of a current word element based on the pre-padding matrix multiplication and addition result, and performing a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and performing a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result; multiplying the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculating a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0007] The application also provides a reply text generation device, comprising: a pre-padding module configured to obtain initial text information of a target user, and input the initial text information into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and perform a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result; a matrix multiplication and addition module configured to obtain a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model, and perform a matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in a pre-padding stage of an inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; a decoding module configured to, in a decoding stage of the inference process, generate a one-dimensional feature vector of a current word element based on the pre-padding matrix multiplication and addition result, and perform a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and perform a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result; and a text generation module configured to multiply the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculate a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0008] The application further provides an electronic device, comprising a memory for storing a computer program, and a processor for implementing the steps of any of the reply text generation methods when executing the computer program.

[0009] The application further provides a non-volatile computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the reply text generation methods.

[0010] The application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the reply text generation methods.

[0011] Through the application, the initial text information of the target user can be obtained, and the initial text information is input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and a preset matrix quantization tensor column decomposition operation is performed on the feature matrix to obtain a corresponding feature matrix array decomposition result; a weight inference matrix and a bias inference tensor in a tensor decomposition form corresponding to the autoregressive language model are obtained, and a matrix multiplication and addition operation is performed on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in the pre-padding stage of the inference process to obtain a pre-padding matrix multiplication and addition result corresponding to the autoregressive language model; in the decoding stage of the inference process, based on the pre-padding matrix multiplication and addition result, a one-dimensional feature vector of the current word element is generated, and a preset quantization processing is performed on the one-dimensional feature vector to obtain a target feature tensor meeting a preset dimension requirement, and a preset vector quantization tensor column decomposition operation is performed on the target feature tensor to obtain a corresponding vector column decomposition result; the vector column decomposition result and the weight inference matrix are multiplied to obtain a corresponding vector matrix product result, and based on the vector matrix product result and the bias inference tensor, a decoding matrix multiplication and addition result corresponding to the autoregressive language model is calculated to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result, so that the technical problem that in the related art, the selection requirement of the rank in the matrix decomposition calculation process is high, the complexity is difficult to effectively reduce, or the error between the calculation result obtained by using the decomposition form and the original result is large can be solved, and the technical effects that the storage and calculation complexity are greatly reduced, the occupation of the data on the storage space is reduced, and the inference performance is effectively improved are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 A flow chart of a reply text generation method according to an embodiment of the application is provided.

[0014] Figure 2 A schematic diagram of a one-dimensional feature vector quantized tensor column decomposition process according to an embodiment of the application is provided.

[0015] Figure 3 An example diagram of a reply text generation device according to an embodiment of the application is provided.

[0016] Among them, 10 is a reply text generation device, 100 is a pre-filling module, 200 is a matrix multiplication and addition module, 300 is a decoding module, and 400 is a text generation module. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0018] It should be noted that in the description of the application, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms "first", "second" and the like in the application are used to distinguish similar objects, not to describe a specific order or sequence.

[0019] In order to make those skilled in the art better understand the application, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0020] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the reply text generation method depends, the specific application environment architecture or specific hardware architecture is described here.

[0021] Embodiments of the application provide a reply text generation method.

[0022] As shown in Figure 1 A flow chart of a reply text generation method according to an embodiment of the application is provided, wherein the reply text generation method comprises the following steps:

[0023] In step S101, initial text information of a target user is acquired, and the initial text information is input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and a preset matrix quantization tensor column decomposition operation is performed on the feature matrix to obtain a corresponding feature matrix array decomposition result.

[0024] As appreciated by those skilled in the art, at present, the model based on the Transformer structure needs to perform dense matrix multiplication calculation in both the training phase and the inference phase, and the time consumption of the matrix multiplication calculation is relatively long. In the training phase, the calculation complexity required by the dense matrix calculation is the third power of the matrix dimension, and the calculation complexity and the occupied memory in the inference phase increase in the second power with the input sequence length.

[0025] However, in the related art, although the storage and calculation complexity are reduced when performing the matrix multiplication calculation, the selection of the rank will affect the final calculation result. If the rank is too large, the complexity reduction is relatively small, and the additional matrix decomposition form for solving the weight matrix will cause the total calculation amount to increase. If the rank is too small, the error between the calculation result obtained by using the decomposition form and the original result will be relatively large, and with the increase of the number of training or inference iterations, the error will gradually accumulate, and eventually cause the calculation result to have a serious error.

[0026] For the autoregressive language model, it usually includes two parts of calculation process, i.e., the self-attention mechanism and the feedforward neural network. Both parts of the calculation process are matrix multiplication calculation in the pre-padding phase of training and inference, and are vector multiplication matrix calculation in the decoding phase of inference. Therefore, based on the quantization tensor column decomposition operation of the vector and the matrix, the embodiments of the present application can solve the large consumption of calculation resources in the matrix multiplication step during the training and inference of the large model, and simultaneously relieve the pressure on the demand for large storage space in the inference process.

[0027] Specifically, in the actual execution process, the embodiments of the present application can first acquire the original text of the user and input it into the pre-trained autoregressive language model to generate the corresponding feature matrix, and perform quantization tensor column decomposition on the feature matrix to obtain the corresponding feature matrix array decomposition result.

[0028] Therefore, the embodiments of the present application can extract the feature matrix corresponding to the text by the autoregressive language model, and perform quantization decomposition thereon, so as to efficiently process the text information input by the user, simplify the feature dimension, and improve the efficiency and accuracy of subsequent text analysis or application.

[0029] Optionally, in one embodiment of this application, a preset matrix quantization tensor column decomposition operation is performed on the feature matrix to obtain the corresponding feature matrix column decomposition result, including: performing a preset zero-padding operation on the feature matrix to obtain a feature zero-padding matrix that meets the preset matrix dimension requirements, and performing a preset rearrangement process on the elements in the feature zero-padding matrix to generate a target dimension tensor operator corresponding to the feature matrix; obtaining the dimension information of the target dimension tensor operator, determining the matrix quantization tensor column decomposition strategy based on the dimension information, and decomposing the target dimension tensor operator through the matrix quantization tensor column decomposition strategy to obtain the feature matrix column decomposition result.

[0030] In the specific implementation process, the steps of performing quantization tensor column decomposition on the feature matrix in this embodiment are as follows:

[0031] Step 1: Extract the feature matrix Perform a zero-padding operation to expand its dimension to (i.e., preset matrix dimension requirements), to obtain the corresponding feature zero-padding matrix (where, Representation of the characteristic matrix For a real number field OK A matrix of columns; The dimension representing the direction of the row number of the feature zero-padding matrix; The dimension representing the direction of the column number of the feature zero-padding matrix; This represents the size of the basic unit in the row direction decomposition of the target dimension tensor operator; This represents the size of the basic unit in the column direction decomposition of the target dimension tensor operator; (This represents the number of levels in the decomposition or the order of the tensor).

[0032] Step 2: Perform element rearrangement on the feature zero-padding matrix to obtain a rearrangement size of ( The row dimension and having indivual or The column-dimensional tensor operator, the mathematical description of this step is: eigenvalue zero-padding matrix From elements The composition is as shown in the following formula:

[0033]

[0034] right The high-dimensional tensor operator obtained after element rearrangement A From elements The composition is as shown in the following formula:

[0035]

[0036] and elements corresponding to the feature zero-padded matrix The element of the feature zero-padded matrix , and the corresponding relationship therebetween is shown in the following formula:

[0037]

[0038] wherein, denotes the element of the i-th row and the j-th column in the feature zero-padded matrix , the value range of i is 1 to , and the value range of j is 1 to ; denotes the element in the high-dimensional tensor operator A , and denotes the index of each level after the row direction decomposition of the high-dimensional tensor operator, the value range of which is 1 to ; denotes the index of each level after the column direction decomposition of the high-dimensional tensor operator, the value range of which is 1 to ;

[0039] Step 3: decomposing the high-dimensional tensor operator A by using the tensor column decomposition method to obtain the feature matrix array decomposition result, as shown in the following formula:

[0040]

[0041] wherein, denotes a small-scale 4-dimensional tensor operator with a scale of ;

[0042] denotes the intermediate index in the decomposition process, the value range of which is .

[0043] That is to say, if the feature matrix is expressed in the form of matrix array decomposition, the feature matrix array decomposition result is referred to as the quantized tensor column decomposition expression form of the feature matrix with a rank of . Through the quantized tensor column decomposition, the complexity of the feature matrix can be reduced from to , wherein denotes the maximum value in each level rank parameter.

[0044] It can be understood that the value of is small, and thus compared with the original data scale of the tensor, the storage complexity of the data after the quantized tensor column decomposition is greatly reduced, so that the storage space occupied is less, and the demand for large storage space in the inference process is greatly reduced.

[0045] In addition, the quantized tensor column decomposition operation on the feature matrix in the above reasoning process is also applicable to different types of matrices (such as weight matrices) in the reasoning or training process.

[0046] Therefore, in the embodiments of the present application, based on the matrix tensor decomposition method, the matrix (such as the feature matrix or the weight matrix) involved in the corresponding matrix multiplication operation in the model reasoning or training process is decomposed by tensor to obtain a decomposition result with smaller storage complexity, thereby greatly reducing the storage and calculation complexity and effectively improving the reasoning performance.

[0047] Optionally, in an embodiment of the present application, before obtaining the initial text information of the target user and inputting the initial text information into the pre-trained autoregressive language model to generate the feature matrix corresponding to the initial text information, it further includes: collecting different types of text data, and performing format conversion operation on the text data to generate corresponding standard text data, and constructing a text training data set through the standard text data; inputting the text training data in the text training data set into the autoregressive language model to generate a feature training matrix corresponding to the text training data; performing a pre-set zero padding operation on the feature training matrix to obtain a corresponding feature zero padding training matrix, and performing a pre-set element rearrangement processing on the feature zero padding training matrix to generate a corresponding feature training tensor operator, and performing a pre-set matrix quantized tensor column decomposition operation on the feature training tensor operator to obtain a corresponding feature training decomposition matrix; obtaining a pre-set weight matrix and a pre-set bias tensor in the form of tensor decomposition corresponding to the autoregressive language model, and performing a matrix multiplication and addition operation on the feature training decomposition matrix, the pre-set weight matrix and the bias tensor to obtain a corresponding training process matrix multiplication and addition result; based on the feature training matrix and the feature training decomposition matrix, calculating the matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the decomposition of the feature training matrix, and based on the matrix multiplication and addition error and the training process matrix multiplication and addition result respectively, iteratively training the autoregressive language model.

[0048] In actual execution process, the embodiments of the present application can first collect multi-domain text data (such as news, dialogue, professional literature, etc.), and uniformly convert the multi-domain different types of text data into standard text data in a pre-set encoding format to construct a text training data set through the standard text data.

[0049] Secondly, the embodiments of the present application can input the text training data into the autoregressive language model to generate a feature training matrix containing semantic features and context association, and perform a dynamic zero padding (such as adaptive padding according to the longest text length) processing on the feature training matrix to obtain a corresponding feature zero padding training matrix.

[0050] Again, the embodiments of the present application can perform element rearrangement operation on the feature zero-padding training matrix to generate a feature training tensor operator, and perform matrix quantization tensor column decomposition operation on the feature training tensor operator to obtain a feature training decomposition matrix.

[0051] Then, the embodiments of the present application can pre-set the weight matrix and bias tensor of the autoregressive language model into a quantization tensor column decomposition form (such as a combination of low-dimensional matrices with a fixed rank number), so as to directly perform matrix multiplication and addition operation on the feature training decomposition matrix and the pre-set weight matrix and bias tensor of the quantization tensor column decomposition form during the model training process, to obtain the corresponding training process matrix multiplication and addition result.

[0052] Finally, the embodiments of the present application can calculate the training process matrix multiplication and addition results before and after the matrix decomposition of the feature training matrix, and calculate the matrix multiplication and addition errors corresponding to the training process matrix multiplication and addition results before and after the matrix decomposition, to calculate the corresponding loss function based on the matrix multiplication and addition errors and the training process matrix multiplication and addition results, and perform back propagation, so that the autoregressive language model learns the compensation mechanism of the quantization decomposition error in the iteration.

[0053] It can be understood that the weight of the autoregressive language model obtained after the training (i.e. the weight inference matrix) is highly compressed lightweight format data, which can be directly used for corresponding matrix multiplication and addition calculation in the pre-padding stage and decoding stage of model inference, not only greatly reducing the storage requirement and memory access bandwidth, but also better adapting to the hardware accelerator with low rank operation optimization requirement, and improving the end-to-end model inference efficiency.

[0054] Therefore, the embodiments of the present application train the weight and bias data into tensor decomposition form during the training process, so that the error caused by the quantization decomposition can be learned and adapted at the beginning of the model training, greatly simplifying the model deployment process and avoiding the precision loss caused by additional decomposition in the inference stage.

[0055] Optionally, in an embodiment of the present application, the preset weight matrix and the preset bias tensor corresponding to the tensor decomposition form of the autoregressive language model are obtained, including: obtaining hierarchical structure information of the autoregressive language model, and parsing the hierarchical structure information to determine input and output feature dimensions and inter-level connection relationships corresponding to the autoregressive language model according to the parsed hierarchical structure information; determining a context window length of the autoregressive language model, and determining an action range and a matrix operation dimension constraint of an initial weight matrix and an initial bias tensor corresponding to the autoregressive language model based on the input and output feature dimensions, the context window length and the inter-level connection relationships; determining a number of hidden layer units and a number of attention heads of the autoregressive language model, and determining a row and column scale of the initial weight matrix according to the number of hidden layer units and the number of attention heads, to construct the initial weight matrix and the initial bias tensor based on the row and column scale, the action range and the matrix operation dimension constraint; performing a preset dimension expansion operation on the initial weight matrix and the initial bias tensor respectively to obtain corresponding expanded matrices, and performing a quantized tensor column decomposition operation on the expanded matrices to obtain a weight decomposition matrix and a bias decomposition matrix corresponding to the expanded matrices, and determining the preset weight matrix and the preset bias tensor according to the weight decomposition matrix and the bias decomposition matrix.

[0056] It should be noted that, in the embodiment of the present application, the hierarchical structure information of the autoregressive language model is first obtained, and the input and output feature dimensions (such as 640 dimensions of the word embedding layer 640 and 1280 dimensions of the attention layer) and the inter-level connection relationships (such as residual connection and layer normalization sequence) are extracted by a structured hierarchical parsing algorithm; further, the embodiment of the present application can generate a dimension operation dimension constraint graph in combination with a preset context window length (such as 1024 tokens), to clearly define the action boundary (such as acting only within the attention head or cross-layer interaction) and the dimension matching rule (such as ensuring the dimension compatibility of matrix multiplication) of the initial weight matrix and the bias tensor at different levels.

[0057] Secondly, the embodiment of the present application can calculate a basic row and column scale (such as 16 heads x 64 dimensions / head = 1024 columns) of the initial weight matrix according to the number of model hidden layer units (such as 2048) and the number of attention heads (such as 16), and dynamically adjust the scale (such as expanding the cross-layer connection matrix to 2048 x 1280) in combination with the dimension constraint graph, to construct the initial weight matrix and the initial bias tensor.

[0058] Afterwards, the embodiment of the present application can dynamically zero-fill the initial weight matrix and the initial bias tensor (adaptively fill zero elements based on the dimension difference of each layer feature), and generate an initial tensor operator through diagonal block rearrangement (aggregating attention head related parameters into a diagonal block structure) and other operations, and adopts a hierarchical quantization decomposition strategy to decompose the attention layer weight into 16 low-rank matrices according to the head, and decomposes the feedforward layer weight according to the hidden layer unit grouping, so as to obtain the weight decomposition matrix and the bias decomposition matrix, and take them as the preset weight matrix and the preset bias tensor of the autoregressive language model training.

[0059] Therefore, the embodiment of the present application can accurately match the quantization decomposition initial parameters corresponding to the autoregressive language model structure, guarantee the compatibility of model training, reduce the storage and calculation cost, and lay a solid foundation for efficient deployment of the model.

[0060] Optionally, in an embodiment of the present application, the initial weight matrix and the initial bias tensor are respectively subjected to preset dimension expansion operations to obtain a weight expansion matrix and a bias expansion tensor, and the weight expansion matrix and the bias expansion tensor are subjected to a quantized tensor column decomposition operation to obtain a corresponding weight decomposition matrix and a bias decomposition matrix, comprising: determining the dimension expansion requirement corresponding to the initial weight matrix and the initial bias tensor, and performing a preset zero-fill operation on the initial weight matrix and the initial bias tensor according to the dimension expansion requirement to obtain a weight expansion matrix corresponding to the initial weight matrix and a bias expansion tensor corresponding to the initial bias tensor; performing a preset element rearrangement operation on the weight expansion matrix and the bias expansion tensor to obtain a weight tensor operator corresponding to the weight expansion matrix and a bias tensor operator corresponding to the bias expansion tensor, and performing a quantized tensor column decomposition operation on the weight tensor operator and the bias tensor operator to obtain a corresponding weight decomposition matrix and a bias decomposition matrix.

[0061] In actual execution process, the embodiment of the present application can adopt the quantized tensor column decomposition processing mode of the above-mentioned feature matrix to perform tensor column decomposition processing on the initial weight matrix and the initial bias tensor.

[0062] Specifically, the embodiment of the present application can first perform a 0-fill operation on the initial weight matrix and the initial bias tensor respectively to obtain a weight expansion matrix corresponding to the initial weight matrix and a bias expansion tensor corresponding to the initial bias tensor; secondly, the embodiment of the present application can perform a preset element rearrangement operation on the weight expansion matrix and the bias expansion tensor to obtain a corresponding weight tensor operator and a bias tensor operator.

[0063] Afterwards, the embodiment of the present application can perform tensor column decomposition on the weight tensor operator and the bias tensor operator to obtain a corresponding weight decomposition matrix and a bias decomposition matrix.

[0064] For example, embodiments of this application may first pad the initial weight matrix B with zeros to expand its dimensionality. The weight expansion matrix (where, This indicates the row number and directional dimension of the weight expansion matrix; (This represents the column number and directional dimension of the weight expansion matrix). Secondly, in this embodiment, the elements of the zero-padded weight expansion matrix can be rearranged to obtain a matrix of size (...). Weight tensor operator .

[0065] Subsequently, embodiments of this application can be applied to the weight tensor operator. Perform quantization tensor column decomposition to obtain the corresponding weight decomposition matrix, as shown in the following equation:

[0066]

[0067] in, Indicates the dimension size as Small-scale 4D tensor operators; Represents the elements in the weight tensor operator; This represents the indexes at each level after column-wise decomposition; This represents the index of each level after decomposition along another dimension; Indicates an intermediate index during the initial weight matrix decomposition process; This represents the rank parameter of each level in the column decomposition of the initial weight matrix tensor.

[0068] Therefore, the embodiments of this application process the initial weight matrix and initial bias tensor through zero-padding, element rearrangement and tensor column decomposition operations to obtain parameters in a low-rank decomposition form, thereby reducing model storage and computational overhead and providing reliable data and technical support for efficient model training and deployment.

[0069] Optionally, in one embodiment of this application, the autoregressive language model is iteratively trained based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process, respectively. This includes: calculating the feature error gradient of each iteration of the training process based on the matrix multiplication and addition results of the training process, and determining the gradient direction of the feature error gradient, so as to adjust the preset weight matrix and preset bias tensor according to the gradient direction, so as to obtain the weight adjustment matrix and bias adjustment tensor after each iteration of training; iteratively training the autoregressive language model through the weight adjustment matrix and bias adjustment tensor until the autoregressive language model meets the preset convergence requirements, so as to obtain the final weight adjustment matrix and bias adjustment tensor, and determining the weight inference matrix and bias inference tensor according to the final weight adjustment matrix and bias adjustment tensor, and generating the trained autoregressive language model.

[0070] As an implementable manner, the embodiment of the present application can calculate the feature error gradient of each iteration by using the adaptive gradient clipping algorithm based on the matrix multiplication and addition result of the training process, and identify and smooth the abnormal gradient (such as the mutation value) through the gradient direction vector clustering.

[0071] Secondly, the embodiment of the present application can dynamically adjust the preset weight matrix and bias tensor according to the modified gradient direction, in combination with the quantization decomposition constraint (such as the low-rank matrix row and column number being unchanged), to use a hierarchical learning rate (the attention head parameter learning rate is higher than the feedforward layer) for the weight decomposition matrix, and apply L1 regularization constraint to the bias decomposition tensor, so as to suppress the redundant parameters.

[0072] After each iteration, the embodiment of the present application can evaluate the model state by the weighted sum of the feature reconstruction error (the difference between the feature matrices before and after decomposition) and the predicted loss value, and if the weighted sum decreases by less than a threshold value for 5 consecutive iterations, the learning rate decay mechanism is triggered.

[0073] When the model meets the preset convergence requirement (such as the predicted loss value being stable for 3 iteration periods), the final weight adjustment matrix and bias adjustment tensor are subjected to quantization accuracy calibration (such as mapping the floating-point parameters to the 8-bit integer range), to generate the weight inference matrix and the bias inference tensor, thereby obtaining the trained autoregressive language model.

[0074] Thus, the embodiment of the present application optimizes the iterative training process, thereby taking into account the quantization decomposition constraint and the model precision, and improving the efficiency and precision of the model inference.

[0075] Optionally, in an embodiment of the present application, based on the feature training matrix and the feature training decomposition matrix, the matrix multiplication and addition error corresponding to the training process matrix multiplication and addition result before and after the decomposition of the feature training matrix is calculated, and based on the matrix multiplication and addition error and the training process matrix multiplication and addition result respectively, the self-recurrent language model is iteratively trained, including: calculating the decomposition-before matrix multiplication result of the feature training matrix and the initial weight matrix, and calculating the decomposition-before matrix multiplication and addition result according to the decomposition-before matrix multiplication result and the initial bias tensor; calculating the decomposition-after matrix multiplication result of the feature training decomposition matrix and the preset weight matrix, and calculating the decomposition-after matrix multiplication and addition result according to the decomposition-after matrix multiplication result and the preset bias tensor; based on the decomposition-before matrix multiplication and addition result and the decomposition-after matrix multiplication and addition result, the corresponding matrix multiplication and addition error is calculated, and the matrix multiplication and addition error is input into the pre-constructed error evaluation model to output the corresponding element-level difference information; the error size and the error distribution position corresponding to the element-level difference information are determined, and the error vector corresponding to the error size and the error distribution position is constructed, and the operation link in the error vector that meets the preset error contribution requirement is marked to generate a labeled error vector; based on the labeled error vector, the element mapping rule of the preset element rearrangement operation is adjusted, and the adjusted element mapping rule is used to perform the preset matrix quantization tensor column decomposition operation on the feature training matrix, the initial weight matrix and the initial bias tensor again to obtain corresponding decomposition data, and based on the decomposition data, the self-recurrent language model is trained.

[0076] Specifically, the embodiments of the present application can also include the error generated in the decomposition process into the back propagation link of the model training, realizing the cooperation of the decomposition accuracy and the model parameter optimization, and the specific process is as follows:

[0077] 1. Obtain the feature training matrix, the initial weight matrix and the initial bias tensor, perform the conventional matrix multiplication on the feature training matrix and the initial weight matrix, and superimpose the initial bias tensor to obtain the decomposition-before matrix multiplication and addition result;

[0078] 2. Perform the preset element rearrangement operation on the feature training matrix and then perform the quantization tensor column decomposition to obtain the feature training decomposition matrix; perform the preset matrix quantization tensor column decomposition on the initial weight matrix and the initial bias tensor respectively to obtain the preset weight matrix and the preset bias tensor;

[0079] 3. Calculate the decomposition-after matrix product of the feature training decomposition matrix and the preset weight matrix, and superimpose the preset bias tensor to obtain the decomposition-after matrix multiplication and addition result;

[0080] 4. Calculate the difference value of the decomposition-before and decomposition-after matrix multiplication and addition results to obtain the matrix multiplication and addition error, input the matrix multiplication and addition error into the error evaluation model, and output the element-level difference information;

[0081] 5. Determine the error size and distribution location according to the element level difference information, construct an error vector, and mark the operation link whose error contribution exceeds the preset threshold to generate a marked error vector;

[0082] 6. Reverse pass the marked error vector along the original operation link to the vector quantized tensor column decomposition step and the matrix quantized tensor column decomposition step, locate the corresponding quantization rearrangement link and decomposition link according to the error mark, and output the error reverse distribution result;

[0083] 7. Based on the error reverse distribution result, adjust the element mapping rule of quantization rearrangement (such as optimizing the element grouping method and adjusting the rearrangement order), and correct the initialization parameters of the decomposition kernel (such as adjusting the initial value distribution of the low-rank kernel), output the corrected quantization rule and decomposition kernel initialization parameters, which are used as the input of the next round of vector and matrix quantized tensor column decomposition steps, realize dynamic correction of decomposition error, and iteratively train the autoregressive language model.

[0084] It should be noted that in the iterative training process of the autoregressive language model of the embodiments of the present application, the matrix multiplication error-based model training process and the model training process based on the matrix multiplication result of the training process complement each other, thereby greatly improving the training performance of the model; for example, after performing the matrix quantized tensor column decomposition operation on the feature training matrix, the initial weight matrix and the initial bias tensor based on the matrix multiplication error, the embodiments of the present application can calculate a new matrix multiplication result according to the decomposition data; further, the embodiments of the present application can calculate the corresponding feature error gradient using the new matrix multiplication result, to adjust the corresponding weight matrix and bias tensor, and train the autoregressive language model through the adjusted weight matrix and bias tensor (i.e., complete one iteration training process), until the autoregressive language model meets the preset convergence requirement, to obtain the final weight adjustment matrix and bias adjustment tensor, and generate the trained autoregressive language model.

[0085] Therefore, the embodiments of the present application improve the decomposition operation accuracy and enhance the stability and efficiency of model training by dynamically adjusting the mapping rule based on the analysis of the quantized decomposition error.

[0086] In step S102, the weight inference matrix and the bias inference tensor corresponding to the tensor decomposition form of the autoregressive language model are obtained, and the matrix multiplication operation of the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor is performed in the pre-filling stage of the inference process to obtain the pre-filled matrix multiplication result corresponding to the autoregressive language model.

[0087] Further, the embodiment of the present application also needs to perform matrix multiplication and addition operation on the feature matrix array decomposition result and the weight inference matrix and bias inference tensor in the form of tensor decomposition after model training, to obtain the pre-padding matrix multiplication and addition result corresponding to the autoregressive language model.

[0088] Specifically, the embodiment of the present application can calculate the product of the feature matrix A after quantized tensor column decomposition and the weight inference matrix B in the form of tensor decomposition, to obtain the matrix multiplication result of the model in the pre-padding stage , as shown in the following formula:

[0089] ;

[0090] Then, the embodiment of the present application can perform matrix addition operation on the matrix multiplication result in the pre-padding stage and the bias inference tensor in the form of tensor decomposition, to obtain the pre-padding matrix multiplication and addition result corresponding to the autoregressive language model.

[0091] It can be understood that the embodiment of the present application can reduce the calculation complexity of the matrix product from the original to , wherein, represents the number of rows of the feature matrix, and herein represents the number of rows of the weight inference matrix, represents the number of columns of the weight inference matrix; is the rank of the operator, and its value is small after decomposition, so the calculation complexity of the matrix product is also greatly reduced due to the tensor decomposition, so that the matrix multiplication calculation after tensor decomposition is linearly dependent on the dimension of the tensor operator, rather than the dimension of the matrix.

[0092] In step S103, in the decoding stage of the inference process, a one-dimensional feature vector of the current token is generated based on the pre-padding matrix multiplication and addition result, and a preset quantization processing is performed on the one-dimensional feature vector to obtain a target feature tensor satisfying a preset dimension requirement, and a preset vector quantized tensor column decomposition operation is performed on the target feature tensor to obtain a corresponding vector column decomposition result.

[0093] Then, in the decoding stage of the model inference process, the embodiment of the present application takes the matrix multiplication and addition result (i.e. the pre-padding matrix multiplication and addition result) output in the pre-padding stage as the core basis, which is generated by the matrix multiplication and addition operation of the feature matrix array decomposition result of the model in the pre-padding process, the weight inference matrix and the bias inference tensor, and has fused the global context association features and hierarchical semantic information of the text.

[0094] Subsequently, in this embodiment, based on the lexical index of the current decoding position, the local feature subset of the corresponding position can be accurately extracted from the multiplication and addition result of the pre-filled matrix. Through the dynamic attention weight allocation mechanism, the correlation strength between the current lexical and the key information in the context (such as core nouns and logical connectors) is strengthened, and the interference of redundant information is weakened.

[0095] Furthermore, embodiments of this application may also introduce an adaptive feature fusion module, which weights and fuses the extracted local features with the feature vectors of historical decoded words, and then removes invalid dimensions through a dimensionality compression algorithm (such as feature filtering based on information entropy), ultimately generating a one-dimensional feature vector that combines global contextual relevance, historical dependency features, and current word specificity, laying the foundation for subsequent quantization and decomposition operations.

[0096] Finally, embodiments of this application can quantize the one-dimensional feature vector (i.e., perform a dimension expansion operation) to obtain a target feature tensor that meets the preset dimension requirements (such as the same dimension as the weight inference matrix), and perform a preset vector quantization tensor column decomposition operation on it to obtain the corresponding vector column decomposition result.

[0097] Therefore, the embodiments of this application can greatly simplify the feature processing flow of inference decoding, reduce computational complexity and storage overhead, while ensuring the effectiveness of feature representation, which helps to improve the efficiency of model inference.

[0098] Optionally, in one embodiment of this application, a one-dimensional feature vector is subjected to a preset quantization process to obtain a target feature tensor that meets the preset dimension requirements, and a preset vector quantization tensor column decomposition operation is performed on the target feature tensor to obtain the corresponding vector column decomposition result. This includes: determining the vector scale information of the one-dimensional feature vector, and performing a preset element rearrangement operation on the one-dimensional feature vector based on the vector scale information to obtain a target feature tensor that meets the preset dimension requirements; performing a preset tensor column decomposition operation on the target feature tensor to obtain multiple sub-vectors corresponding to the one-dimensional feature vector, and determining the vector column decomposition result based on the multiple sub-vectors.

[0099] As one possible approach, such as Figure 2 As shown, for a dimension of size One-dimensional feature vector The embodiments of this application can first perform quantization (i.e., element rearrangement) to obtain a scale of (Total) indivual A high-dimensional tensor (i.e., the target feature tensor) that is multiplied (i.e., meets the preset dimension requirements). As shown in the following formula:

[0100]

[0101] wherein, denotes the basic unit size of quantized decomposition; denotes an element in the high-dimensional tensor denotes an element in the one-dimensional feature vector and the following equation can be obtained:

[0102]

[0103] Secondly, the embodiments of the present application can perform tensor train decomposition (TTD) on the high-dimensional tensor to obtain a plurality of sub-vectors (one-dimensional tensors); wherein, the high-dimensional tensor The mathematical expression of the tensor train decomposition is:

[0104]

[0105] wherein, denotes a three-dimensional tensor with a size of denotes an intermediate index in the decomposition process, and the value range is 1, 2, , is the rank of the quantized tensor train decomposition; is the core of the quantized tensor train decomposition; denotes a dimension parameter related to the basic unit size of the quantized decomposition.

[0106] It can be understood that, by performing the quantized tensor train decomposition operation on the one-dimensional feature vector , the complexity of the one-dimensional feature vector can be reduced from to , wherein, , so that the storage complexity of the data after the quantized tensor train decomposition is greatly reduced, and the demand of the model on large storage space in the inference process is reduced.

[0107] In step S104, the vector train decomposition result and the weight inference matrix are multiplied to obtain a corresponding vector matrix product result, and based on the vector matrix product result and the bias inference tensor, a decoding matrix multiplication and addition result corresponding to the autoregressive language model is calculated to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0108] ​​​​Further, the embodiment of the present application can calculate the vector matrix product result between the vector column decomposition result and the weight inference matrix, and operate the product result with the bias inference tensor to obtain the decoding matrix multiplication and addition result of the autoregressive language model, and generate the target reply text corresponding to the initial text information of the user based on the decoding matrix multiplication and addition result.

[0109] Therefore, the embodiment of the present application can guarantee the efficient connection of the model decoding process through the accurate operation of vectors and matrices, thereby ensuring the accuracy and fluency of the reply text generation while controlling the model calculation cost.

[0110] Optionally, in an embodiment of the present application, the vector column decomposition result and the weight inference matrix are multiplied to obtain the corresponding vector matrix product result, comprising: obtaining a plurality of historical vector matrix product results and a plurality of historical vectors of the historical input data of the autoregressive language model in the historical decoding stage, and performing semantic correlation degree evaluation on the plurality of historical vector matrix product results and the plurality of historical vectors to obtain corresponding correlation degree evaluation data, and determining the historical feature correlation factor of the autoregressive language model through the correlation degree evaluation data; performing element-by-element superposition operation on the vector matrix product result and the bias inference tensor to obtain the decoding matrix multiplication and addition result, and calculating the target decoding matrix multiplication and addition result of the autoregressive language model based on the decoding matrix multiplication and addition result and the historical feature correlation factor; performing feature mapping on the target decoding matrix multiplication and addition result to obtain a feature matching degree sequence corresponding to the preset text database, and selecting at least one reply text satisfying the preset matching degree requirement from the preset text database based on the feature matching degree value of the feature matching degree sequence, and splicing the at least one reply text to generate the target reply text.

[0111] As a kind of can be realized way, the embodiment of the present application can first obtain the historical vector corresponding to the historical vector matrix product result and the historical input data of the autoregressive language model in the historical decoding stage, and the way of combining semantic similarity matching with attention weight calculation is used to evaluate the semantic correlation degree between the two, that is, the correlation degree evaluation data is generated by calculating vector cosine similarity, key feature overlap rate, and then the different historical records are given differential weights based on the evaluation data to determine the historical feature correlation factor of the model.

[0112] Secondly, the embodiment of the present application can perform element-by-element superposition operation on each row of the current vector matrix product result and the bias inference tensor to obtain the decoding matrix multiplication and addition result , and the calculation process is as follows:

[0113]

[0114] wherein, denotes the current vector matrix product result denotes the element of the 1st level after tensor column decomposition, is the index of the level; denotes the bias inference tensor denotes the element of the 1st level after tensor column decomposition; denotes the element of the 1st level after tensor column decomposition of the decoding matrix multiplication and addition junction D denotes the element of the 1st level after tensor column decomposition of the decoding matrix multiplication and addition junction D

[0115] Again, the embodiments of the present application can weight and fuse the decoding matrix multiplication and addition result and the historical feature correlation factor (the correlation factor weight is dynamically adjusted with the decoding step number) to generate a target decoding matrix multiplication and addition result that takes into account the current semantics and historical dependence.

[0116] After that, the embodiments of the present application can use a multi-level feature mapping algorithm to map the target decoding matrix multiplication and addition result into a feature matching degree sequence consistent with the feature dimension of the preset text database, and sort the matching degree values in descending order to filter out multiple candidate reply texts that meet the preset threshold, and remove conflicting texts through semantic coherence verification (such as syntax fluency and logic consistency detection), and finally splice the target reply text in a smooth and coherent manner according to the matching degree priority.

[0117] Thus, the embodiments of the present application improve the coherence and accuracy of the reply text generation by fusing historical feature correlation and precise feature matching operations, optimize the feature mapping and text screening efficiency, and ensure the quality of the generated reply text.

[0118] Optionally, in an embodiment of the present application, after generating the target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result, it further includes: collecting multi-modal feedback data of the target user on the target reply text, and analyzing the multi-modal feedback data to obtain corresponding analysis data, and extracting multiple text evaluation information corresponding to the analysis data, wherein the multiple text evaluation information includes semantic fit degree, content completeness and logical coherence; extracting multiple text features of the target reply text, and performing feature correlation operation on the multiple text features and the multiple text evaluation information to determine the accuracy score and feature matching difference level of the target reply text, wherein the multiple text features include semantic features, syntax structure features and content coherence features; adjusting the weight inference matrix and the bias inference tensor of the autoregressive language model according to the accuracy score and the feature matching difference level to obtain the corresponding weight optimization matrix and the bias optimization tensor, and calculating the new decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the weight optimization matrix and the bias optimization tensor, to regenerate a new target reply text using the new decoding matrix multiplication and addition result.

[0119] Specifically, after generating the target reply text, the embodiment of the application can first collect multi-modal feedback data such as text evaluation, voice feedback, and semantic annotation of the target user based on a multi-modal feedback collection strategy, and extract subjective satisfaction scores from the text evaluation, extract emotional tendency features from the voice feedback, and capture key information missing points from the semantic annotation.

[0120] Secondly, the embodiment of the application can perform cross-modal fusion analysis on the multi-modal feedback data, extract text evaluation information such as semantic fit degree, content completeness, and logical coherence through natural language processing, and supplement emotional tendency scores by means of voice emotion recognition technology to form a multi-dimensional evaluation data set; at the same time, the semantic features, grammatical structure features, content coherence features, and information density features of the target reply text are extracted to construct a text feature vector library.

[0121] Thirdly, the embodiment of the application can use a feature correlation algorithm driven by an attention mechanism to perform dimension-by-dimension matching between the text feature vector and the multi-dimensional evaluation data, and calculate the accuracy comprehensive score of the target reply text by weighted summation, and combine the preset difference threshold to divide the feature matching difference level into three levels of high, medium, and low (such as high-level difference corresponding to semantic feature mismatch).

[0122] After that, the embodiment of the application can dynamically allocate and adjust the weight based on the difference level; in actual execution, the embodiment of the application can use large-step gradient descent to adjust the core layer parameters of the weight reasoning matrix for high-level difference; for medium and low-level difference, the edge layer parameters and bias reasoning tensor are fine-tuned and optimized through small-step fine-tuning. Recalculate the multiplication and addition results of the decoding matrix to generate a new target reply text that better meets the user's needs.

[0123] Thus, the embodiment of the application can rely on multi-modal feedback to realize dynamic optimization of the autoregressive language model, accurately match user needs, and continuously improve the accuracy and fit of the target reply text generation, thereby enhancing the self-adaptation ability of the model.

[0124] It should be noted that in actual execution, the method for calculating and optimizing and accelerating the autoregressive language model based on the tensor decomposition strategy of the embodiment of the application not only accelerates the calculation of the autoregressive language model, but also optimizes other models with a large number of matrix multiplication or vector multiplication matrix operations to accelerate their calculation process, thereby achieving the purpose of performance improvement.

[0125] In addition, the embodiment of the application can also adjust the rank parameter by dynamically adapting the data features to reduce redundant calculation while ensuring decomposition accuracy, improve decomposition efficiency, and adaptability of different feature data, the specific process is as follows:

[0126] 1. Perform a quantization rearrangement operation on a vector or matrix to convert it into a high-dimensional tensor, which is input to a feature complexity evaluation module as the original object of feature extraction;

[0127] 2. The feature complexity evaluation module receives the high-dimensional tensor, extracts the element distribution density, non-zero element aggregation degree, local feature variance, and other multi-dimensional features corresponding to the high-dimensional tensor, and generates a feature set, which is input to a complexity scoring model;

[0128] 3. The complexity scoring model calculates a comprehensive complexity score based on the input feature set through a pre-set feature weight distribution rule, converts the score into a dynamic rank adjustment coefficient according to a mapping relationship, and inputs the coefficient as a reference for setting the dimensions of the tensor column decomposition stage;

[0129] 4. The tensor column decomposition stage sets the initial dimension range of the low-rank core based on the dynamic rank adjustment coefficient, and simultaneously monitors the feature complexity changes of the local blocks of the tensor in real time to generate local block complexity data, which is used as a judgment basis for dimension adjustment;

[0130] 5. According to the local block complexity data, if the local block complexity is higher than a pre-set threshold, the dimensions of the corresponding low-rank core are increased by a proportion of the dynamic rank adjustment coefficient; if the local block complexity is lower than the threshold, the dimensions are decreased by a proportion, and a dynamic low-rank core set adapted to different regional features is generated, which is input to the subsequent matrix or vector multiplication step to replace the fixed rank core for operation.

[0131] Therefore, the embodiments of the present application can break the fixed low-rank setting, so that the rank in the decomposition process dynamically adapts to the data features, improving the balance between decomposition accuracy and efficiency.

[0132] As an implementable way, the embodiments of the present application can also realize the reuse and migration of decomposition core parameters between different levels of the autoregressive language model, reduce redundant calculation, and the specific process is as follows:

[0133] Step 1, in the model training stage, collect the low-rank cores of the weight matrices of each level after decomposition, extract the dimension parameters, element distribution features and operation response modes of each low-rank core, form a core feature data set, and input the data set to step 2 for similarity calculation;

[0134] Step 2, based on the core feature data set of step 1, calculate the feature matching degree of the low-rank cores of adjacent levels, set a matching threshold, select low-rank cores that meet the sharing conditions, construct a core parameter pool according to the level correlation, and mark the reuse priority, and input the parameter pool to step 3 as a search resource;

[0135] Step 3, when performing quantized tensor column decomposition on a high-level matrix, first search the core parameter pool of step 2 according to the structural characteristics of the current matrix, match the shared low-rank core to output the core matching result, which can be used as the decomposition basis input of step 4;

[0136] Step 4, based on the core matching result of step 3, embed the shared low-rank core into the decomposition framework of the high-level matrix, and only perform targeted decomposition on the unmatched feature area to generate a composite low-rank core that fuses the shared core and the new decomposition core, which can be used as the operation input of step 5;

[0137] Step 5, use the composite low-rank core of step 4 for matrix multiplication operation, record the reuse effect of the shared core, and update the priority of the parameter pool. The updated parameter pool can be used as the resource input for the next round of cross-layer decomposition.

[0138] Therefore, the embodiments of the present application can reduce repeated decomposition calculation by sharing the adaptive decomposition core across layers, reduce the storage requirement, and at the same time guarantee the continuity of hierarchical feature transmission.

[0139] In summary, the embodiments of the present application can effectively reduce the storage amount of the matrix through tensor decomposition operation, and the operator multiplication calculation amount after tensor decomposition is very small. Moreover, with the calculation iteration of the model containing a large number of matrix multiplication calculations (such as the autoregressive language model) in the training and inference process, the effect of reducing the calculation amount is more significant.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0141] The embodiments of the present application also provide a reply text generation device.

[0142] As shown in Figure 3 , the reply text generation device 10 comprises a pre-filling module 100, a matrix multiplication module 200, a decoding module 300, and a text generation module 400.

[0143] The pre-filling module 100 is configured to obtain initial text information of a target user, and input the initial text information into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information, and perform a preset matrix quantized tensor column decomposition operation on the feature matrix to obtain a corresponding feature matrix array decomposition result.

[0144] The matrix multiplication and addition module 200 is configured to obtain the weight inference matrix and the bias inference tensor in the tensor decomposition form of the autoregressive language model, and perform matrix multiplication and addition operation on the feature matrix array decomposition result, the weight inference matrix and the bias inference tensor in the pre-padding stage of the inference process, to obtain the pre-padding matrix multiplication and addition result corresponding to the autoregressive language model.

[0145] The decoding module 300 is configured to generate a one-dimensional feature vector of the current word element based on the pre-padding matrix multiplication and addition result in the decoding stage of the inference process, and perform preset quantization processing on the one-dimensional feature vector to obtain a target feature tensor satisfying a preset dimension requirement, and perform preset vector quantization tensor column decomposition operation on the target feature tensor to obtain a corresponding vector column decomposition result.

[0146] The text generation module 400 is configured to multiply the vector column decomposition result and the weight inference matrix to obtain a corresponding vector matrix product result, and calculate a decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the vector matrix product result and the bias inference tensor, to generate a target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result.

[0147] Optionally, in an embodiment of the present application, the reply text generation device 10 further comprises a data set construction module, a feature extraction module, a zero padding module, a quantized tensor column decomposition module and an iterative training module.

[0148] The data set construction module is configured to collect different types of text data before obtaining the initial text information of the target user, and perform format conversion operation on the text data to generate corresponding standard text data, and construct a text training data set through the standard text data, and input the initial text information into the pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information.

[0149] The feature extraction module is configured to input the text training data in the text training data set into the autoregressive language model to generate a feature training matrix corresponding to the text training data.

[0150] The zero padding module is configured to perform a preset zero padding operation on the feature training matrix to obtain a corresponding feature zero padding training matrix, perform a preset element rearrangement processing on the feature zero padding training matrix to generate a corresponding feature training tensor operator, and perform a preset matrix quantized tensor column decomposition operation on the feature training tensor operator to obtain a corresponding feature training decomposition matrix.

[0151] The quantized tensor column decomposition module is configured to obtain a preset weight matrix and a preset bias tensor in the tensor decomposition form of the autoregressive language model, and perform matrix multiplication and addition operation on the feature training decomposition matrix, the preset weight matrix and the bias tensor to obtain a corresponding training process matrix multiplication and addition result.

[0152] The iteration training module is configured to calculate matrix multiplication and addition error corresponding to a training process matrix multiplication and addition result before and after decomposition of the feature training matrix based on the feature training matrix and the feature training decomposition matrix, and iteratively train the autoregressive language model based on the matrix multiplication and addition error and the training process matrix multiplication and addition result, respectively.

[0153] Optionally, in an embodiment of the present application, the pre-filling module 100 comprises a rearrangement unit and a first decomposition unit.

[0154] The rearrangement unit is configured to perform a preset zero padding operation on the feature matrix to obtain a feature zero-padded matrix satisfying a preset matrix dimension requirement, and perform a preset rearrangement process on elements in the feature zero-padded matrix to generate a target dimension tensor operator corresponding to the feature matrix.

[0155] The first decomposition unit is configured to obtain dimension information of the target dimension tensor operator, determine a matrix quantized tensor column decomposition strategy according to the dimension information, and decompose the target dimension tensor operator through the matrix quantized tensor column decomposition strategy to obtain the feature matrix array decomposition result.

[0156] Optionally, in an embodiment of the present application, the decoding module 300 comprises a determination unit and a second decomposition unit.

[0157] The determination unit is configured to determine vector scale information of the one-dimensional feature vector, and perform a preset element rearrangement operation on the one-dimensional feature vector based on the vector scale information to obtain a target feature tensor satisfying a preset dimension requirement.

[0158] The second decomposition unit is configured to perform a preset tensor column decomposition operation on the target feature tensor to obtain a plurality of sub-vectors corresponding to the one-dimensional feature vector, and determine a vector column decomposition result according to the plurality of sub-vectors.

[0159] Optionally, in an embodiment of the present application, the text generation module 400 comprises an evaluation unit, a superposition unit and a feature mapping unit.

[0160] The evaluation unit is configured to obtain a plurality of historical vector matrix product results of the autoregressive language model in a historical decoding stage and a plurality of historical vectors of historical input data, perform semantic correlation degree evaluation on the plurality of historical vector matrix product results and the plurality of historical vectors to obtain corresponding correlation degree evaluation data, and determine a historical feature correlation factor corresponding to the autoregressive language model through the correlation degree evaluation data.

[0161] The superposition unit is configured to perform an element-wise superposition operation on the vector matrix product result and the bias inference tensor to obtain a decoding matrix multiplication and addition result, and calculate a target decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the decoding matrix multiplication and addition result and the historical feature correlation factor.

[0162] The feature mapping unit is configured to perform feature mapping on the target decoding matrix multiplication and addition result to obtain a feature matching degree sequence corresponding to the preset text database, and select at least one reply text satisfying a preset matching degree requirement from the preset text database based on a feature matching degree value of the feature matching degree sequence, and splice the at least one reply text to generate a target reply text.

[0163] Optionally, in an embodiment of the present application, the reply text generation apparatus 10 further comprises a collection module, a feature correlation module and an adjustment module.

[0164] The collection module is configured to collect multi-modal feedback data of the target user on the target reply text after generating the target reply text corresponding to the initial text information according to the decoding matrix multiplication and addition result, and analyze the multi-modal feedback data to obtain corresponding analysis data, and extract a plurality of text evaluation information corresponding to the analysis data, wherein the plurality of text evaluation information includes semantic matching degree, content completeness and logical coherence.

[0165] The feature correlation module is configured to extract a plurality of text features of the target reply text, and perform feature correlation operation on the plurality of text features and the plurality of text evaluation information to determine an accuracy score of the target reply text and a feature matching difference level, wherein the plurality of text features include semantic features, syntactic structure features and content coherence features.

[0166] The adjustment module is configured to adjust the weight inference matrix and the bias inference tensor of the autoregressive language model according to the accuracy score and the feature matching difference level to obtain a corresponding weight optimization matrix and a bias optimization tensor, and calculate a new decoding matrix multiplication and addition result corresponding to the autoregressive language model based on the weight optimization matrix and the bias optimization tensor, to regenerate a new target reply text using the new decoding matrix multiplication and addition result.

[0167] Optionally, in an embodiment of the present application, the quantized tensor column decomposition module comprises an acquisition unit, a window length determination unit, a construction unit and a dimension expansion unit.

[0168] The acquisition unit is configured to acquire hierarchical structure information of the autoregressive language model, and analyze the hierarchical structure information to determine input-output feature dimensions and inter-level connection relationships corresponding to the autoregressive language model according to the analyzed hierarchical structure information.

[0169] The window length determination unit is configured to determine a context window length of the autoregressive language model, and determine an action range and a matrix operation dimension constraint of an initial weight matrix and an initial bias tensor corresponding to the autoregressive language model based on an input-output feature dimension, the context window length and an inter-level connection relationship.

[0170] The construction unit is configured to determine a number of hidden layer units and a number of attention heads of the autoregressive language model, and determine a row and column size of the initial weight matrix according to the number of hidden layer units and the number of attention heads, so as to construct the initial weight matrix and the initial bias tensor based on the row and column size, the action range and the matrix operation dimension constraint.

[0171] The dimension expansion unit is configured to perform a preset dimension expansion operation on the initial weight matrix and the initial bias tensor respectively to obtain a corresponding expanded matrix, perform a quantized tensor column decomposition operation on the expanded matrix to obtain a corresponding weight decomposition matrix and a bias decomposition matrix, and determine a preset weight matrix and a preset bias tensor according to the weight decomposition matrix and the bias decomposition matrix.

[0172] Optionally, in an embodiment of the present application, the iterative training module comprises a first calculation unit, a second calculation unit, an error evaluation unit, a marking unit and a training unit.

[0173] The first calculation unit is configured to calculate a pre-decomposition matrix product result of the feature training matrix and the initial weight matrix, and calculate a corresponding pre-decomposition matrix multiplication-addition result according to the pre-decomposition matrix product result and the initial bias tensor.

[0174] The second calculation unit is configured to calculate a post-decomposition matrix product result of the feature training decomposition matrix and the preset weight matrix, and calculate a corresponding post-decomposition matrix multiplication-addition result according to the post-decomposition matrix product result and the preset bias tensor.

[0175] The error evaluation unit is configured to calculate a corresponding matrix multiplication-addition error based on the pre-decomposition matrix multiplication-addition result and the post-decomposition matrix multiplication-addition result, and input the matrix multiplication-addition error into a pre-constructed error evaluation model to output corresponding element-level difference information.

[0176] The marking unit is configured to determine a corresponding error size and an error distribution position according to the element-level difference information, construct a corresponding error vector through the error size and the error distribution position, and mark an operation link in the error vector that meets a preset error contribution requirement, so as to generate a marked error vector.

[0177] The training unit is configured to adjust an element mapping rule of the preset element rearrangement operation based on the labeled error vector, and perform a preset matrix quantization tensor column decomposition operation on the feature training matrix, the initial weight matrix and the initial bias tensor by using the adjusted element mapping rule to obtain corresponding decomposition data, and train the autoregressive language model based on the decomposition data.

[0178] Optionally, in an embodiment of the present application, the iterative training module further comprises a feature error gradient calculation unit and a convergence judgment unit.

[0179] The feature error gradient calculation unit is configured to calculate a feature error gradient of each iteration training process based on a training process matrix multiplication and addition result, and determine a gradient direction of the feature error gradient, so as to adjust the preset weight matrix and the preset bias tensor according to the gradient direction to obtain a weight adjustment matrix and a bias adjustment tensor after each iteration training.

[0180] The convergence judgment unit is configured to perform iterative training on the autoregressive language model by using the weight adjustment matrix and the bias adjustment tensor until the autoregressive language model meets a preset convergence requirement, so as to obtain a final weight adjustment matrix and a final bias adjustment tensor, determine a weight inference matrix and a bias inference tensor according to the final weight adjustment matrix and the final bias adjustment tensor, and generate a trained autoregressive language model.

[0181] The description of the features in the embodiment of the reply text generation device can refer to the related description of the embodiments of the reply text generation method, which will not be repeated here.

[0182] Embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above reply text generation method embodiments.

[0183] Embodiments of the present application also provide a non-volatile computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above reply text generation method embodiments when running.

[0184] In an exemplary embodiment, the above non-volatile computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0185] The embodiment of the application further provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the steps in any of the reply text generation method embodiments.

[0186] The embodiment of the application further provides another computer program product, comprising a non-volatile computer readable storage medium, the non-volatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps in any of the reply text generation method embodiments.

[0187] Those skilled in the art will further appreciate that the steps of the example methods described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or both. The disclosure is not limited to any one form of implementation. The example embodiments described herein are to be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the disclosure is not intended to be limited to any one form of implementation but is intended to cover any form of implementation consistent with the principles of the disclosure.

[0188] The above provides a reply text generation method, device, equipment and medium. The principle and implementation of the application are described by applying specific examples. The above description of the embodiments is only used to help understand the method and its core idea. It should be pointed out that for those skilled in the art, without departing from the principle of the application, some improvements and modifications can be made to the application, and these improvements and modifications also fall within the protection scope of the claims of the application.

Claims

1. A method for generating response text, characterized in that, Includes the following steps: The initial text information of the target user is obtained and input into a pre-trained autoregressive language model to generate a feature matrix corresponding to the initial text information. The feature matrix is ​​then subjected to a preset matrix quantization tensor column decomposition operation to obtain the corresponding feature matrix column decomposition result. Obtain the weight inference matrix and bias inference tensor in the tensor decomposition form corresponding to the autoregressive language model, and perform matrix multiplication and addition operations on the feature matrix column decomposition result, the weight inference matrix and the bias inference tensor in the pre-filling stage of the inference process to obtain the pre-filled matrix multiplication and addition result corresponding to the autoregressive language model; During the decoding stage of the reasoning process, a one-dimensional feature vector of the current word is generated based on the multiplication and addition results of the pre-filled matrix, and the one-dimensional feature vector is subjected to preset quantization processing to obtain a target feature tensor that meets the preset dimension requirements. Furthermore, the target feature tensor is subjected to preset vector quantization tensor column decomposition operation to obtain the corresponding vector column decomposition results. The vector column decomposition result is multiplied by the weight inference matrix to obtain the corresponding vector matrix product result. Based on the vector matrix product result and the bias inference tensor, the decoding matrix multiplication result corresponding to the autoregressive language model is calculated to generate the target response text corresponding to the initial text information according to the decoding matrix multiplication result.

2. The response text generation method according to claim 1, characterized in that, Before acquiring the initial text information of the target user and inputting the initial text information into a pre-trained autoregressive language model to generate the feature matrix corresponding to the initial text information, the method further includes: Collect different types of text data, perform format conversion operations on the text data to generate corresponding standard text data, and construct a text training dataset using the standard text data; The text training data in the text training dataset is input into the autoregressive language model to generate the feature training matrix corresponding to the text training data; The feature training matrix is ​​padded with zeros to obtain the corresponding feature zero-padding training matrix. The feature zero-padding training matrix is ​​then rearranged with elements to generate the corresponding feature training tensor operator. Finally, the feature training tensor operator is decomposed into a matrix quantization tensor column to obtain the corresponding feature training decomposition matrix. Obtain the preset weight matrix and preset bias tensor in the tensor decomposition form corresponding to the autoregressive language model, and perform matrix multiplication and addition operations on the feature training decomposition matrix, the preset weight matrix and the preset bias tensor to obtain the corresponding training process matrix multiplication and addition results; Based on the feature training matrix and the feature training decomposition matrix, the matrix multiplication and addition error corresponding to the matrix multiplication and addition results of the training process before and after the feature training matrix decomposition is calculated, and the autoregressive language model is iteratively trained based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process, respectively.

3. The response text generation method according to claim 1, characterized in that, The step of performing a preset matrix quantization tensor column decomposition operation on the feature matrix to obtain the corresponding feature matrix column decomposition result includes: The feature matrix is ​​padded with zeros in a preset manner to obtain a feature zero-padding matrix that meets the preset matrix dimension requirements. The elements in the feature zero-padding matrix are then rearranged in a preset manner to generate the target dimension tensor operator corresponding to the feature matrix. The dimension information of the target dimension tensor operator is obtained, and a matrix quantization tensor column decomposition strategy is determined based on the dimension information. The target dimension tensor operator is then decomposed using the matrix quantization tensor column decomposition strategy to obtain the feature matrix column decomposition result.

4. The response text generation method according to claim 1, characterized in that, The step of performing a preset quantization process on the one-dimensional feature vector to obtain a target feature tensor that meets the preset dimension requirements, and performing a preset vector quantization tensor column decomposition operation on the target feature tensor to obtain the corresponding vector column decomposition result, includes: Determine the vector size information of the one-dimensional feature vector, and based on the vector size information, perform a preset element rearrangement operation on the one-dimensional feature vector to obtain a target feature tensor that meets the preset dimension requirements; A preset tensor column decomposition operation is performed on the target feature tensor to obtain multiple sub-vectors corresponding to the one-dimensional feature vector, and the vector column decomposition result is determined based on the multiple sub-vectors.

5. The response text generation method according to claim 1, characterized in that, The step of calculating the decoding matrix multiplication result corresponding to the autoregressive language model based on the vector matrix multiplication result and the bias inference tensor, and generating the target response text corresponding to the initial text information according to the decoding matrix multiplication result, includes: The autoregressive language model obtains multiple historical vector matrix product results and multiple historical vectors of historical input data during the historical decoding stage, and performs semantic relevance evaluation on the multiple historical vector matrix product results and the multiple historical vectors to obtain corresponding relevance evaluation data, and determines the historical feature relevance factor corresponding to the autoregressive language model through the relevance evaluation data. The vector matrix multiplication result and the bias inference tensor are superimposed element-wise to obtain the decoding matrix multiplication result. Based on the decoding matrix multiplication result and the historical feature association factor, the target decoding matrix multiplication result corresponding to the autoregressive language model is calculated. The target decoding matrix multiplication and addition results are subjected to feature mapping to obtain a feature matching degree sequence corresponding to a preset text database. Based on the feature matching degree value of the feature matching degree sequence, at least one response text that meets the preset matching degree requirement is selected from the preset text database. The at least one response text is then concatenated to generate the target response text.

6. The response text generation method according to claim 1, characterized in that, After generating the target response text corresponding to the initial text information based on the multiplication and addition results of the decoding matrix, the method further includes: Collect multimodal feedback data of the target user's response text to the target, and parse the multimodal feedback data to obtain corresponding parsed data. Extract multiple text evaluation information corresponding to the parsed data, wherein the multiple text evaluation information includes semantic fit, content completeness and logical coherence. Extract multiple text features from the target response text, and perform feature association operations on the multiple text features and the multiple text evaluation information to determine the accuracy score and feature matching difference level of the target response text. The multiple text features include semantic features, grammatical structure features and content coherence features. The weight inference matrix and the bias inference tensor of the autoregressive language model are adjusted according to the accuracy score and the feature matching difference level to obtain the corresponding weight optimization matrix and bias optimization tensor. Based on the weight optimization matrix and the bias optimization tensor, a new decoding matrix multiplication and addition result corresponding to the autoregressive language model is calculated to regenerate a new target response text using the new decoding matrix multiplication and addition result.

7. The response text generation method according to claim 2, characterized in that, The step of obtaining the preset weight matrix and preset bias tensor of the tensor decomposition form corresponding to the autoregressive language model includes: Obtain the hierarchical structure information of the autoregressive language model and parse the hierarchical structure information to determine the input and output feature dimensions and inter-hierarchical connection relationships of the autoregressive language model based on the parsed hierarchical structure information; Determine the context window length of the autoregressive language model, and based on the input and output feature dimensions, the context window length, and the inter-level connectivity, determine the scope of action and matrix operation dimension constraints of the initial weight matrix and initial bias tensor corresponding to the autoregressive language model. The number of hidden layer units and the number of attention heads of the autoregressive language model are determined, and the row and column size of the initial weight matrix is ​​determined based on the number of hidden layer units and the number of attention heads. Based on the row and column size, the scope of action and the matrix operation dimension constraints, the initial weight matrix and the initial bias tensor are constructed. The initial weight matrix and the initial bias tensor are subjected to a preset dimension expansion operation to obtain the corresponding weight expansion matrix and bias expansion tensor. The weight expansion matrix and the bias expansion tensor are then subjected to a quantization tensor column decomposition operation to obtain the corresponding weight decomposition matrix and bias decomposition matrix. The preset weight matrix and the preset bias tensor are then determined based on the weight decomposition matrix and the bias decomposition matrix.

8. The response text generation method according to claim 7, characterized in that, The step of calculating the matrix multiplication and addition error corresponding to the matrix multiplication and addition results of the training process before and after the feature training matrix decomposition based on the feature training matrix and the feature training decomposition matrix, and iteratively training the autoregressive language model based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process respectively, includes: Calculate the undecomposed matrix product of the feature training matrix and the initial weight matrix, and calculate the corresponding undecomposed matrix multiplication and addition result based on the undecomposed matrix product and the initial bias tensor; Calculate the product of the decomposed matrix of the feature training decomposition matrix and the preset weight matrix, and calculate the corresponding product of the decomposed matrix based on the product of the decomposed matrix and the preset bias tensor; Based on the matrix multiplication and addition results before and after decomposition, the corresponding matrix multiplication and addition error is calculated, and the matrix multiplication and addition error is input into the pre-constructed error evaluation model to output the corresponding element-level difference information. The corresponding error magnitude and error distribution location are determined based on the element-level difference information, and a corresponding error vector is constructed using the error magnitude and error distribution location. The computational steps in the error vector that meet the preset error contribution requirements are marked to generate a marked error vector. Based on the labeled error vector, the element mapping rules of the preset element rearrangement operation are adjusted, and the preset matrix quantization tensor column decomposition operation is performed again on the feature training matrix, the initial weight matrix, and the initial bias tensor using the adjusted element mapping rules to obtain the corresponding decomposed data. Based on the decomposed data, the autoregressive language model is trained.

9. The response text generation method according to claim 2, characterized in that, The iterative training of the autoregressive language model based on the matrix multiplication and addition error and the matrix multiplication and addition results of the training process includes: Based on the matrix multiplication and addition results of the training process, the feature error gradient of each iteration of the training process is calculated, and the gradient direction of the feature error gradient is determined. The preset weight matrix and the preset bias tensor are adjusted according to the gradient direction to obtain the weight adjustment matrix and bias adjustment tensor after each iteration of training. The autoregressive language model is iteratively trained using the weight adjustment matrix and the bias adjustment tensor until it meets the preset convergence requirements, so as to obtain the final weight adjustment matrix and bias adjustment tensor. The weight inference matrix and the bias inference tensor are then determined based on the final weight adjustment matrix and bias adjustment tensor, and the trained autoregressive language model is generated.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the response text generation method as described in any one of claims 1 to 9 when executing the computer program.

Images